Is the nuclear non-proliferation pact a model for regulating runaway AI?

Brendan Smialowski/AFP via Getty Images The potential existential threat of artificial intelligence has leapt into the headlines after a departing safety researcher at AI corporation Anthropic posted on social media he believed there was more than a one-in-ten chance AI could kill everyone.

Other Anthropic insiders went public in support of his views, saying the technology could become advanced and dangerous enough to cause human extinction within ten years.

This all followed renewed warnings from Turing Award-winning computer scientist Yoshua Bengio, who has spent the past year arguing that increasingly autonomous AI systems could develop self-preservation goals humans cannot easily switch off.

Even if the current development of AI falls short of artificial superintelligence or artificial general intelligence , many have warned it might be used by bad actors for nefarious ends.

This includes biological weapons design , cyber-attacks against critical infrastructure and financial systems, and assisting hybrid warfare efforts.

More conventionally, we already know AI was used in the planning and execution of the United States raid on Venezuela at the start of this year.

A number of sceptics have argued the latest claims remain highly speculative .

But there are still sound reasons to apply the cautionary principle to a technology that is developing at an exponential rate.

Of particular concern is that AI is emerging as central to growing competition between the US and China.

Echoing past space and nuclear races, both superpowers fear falling behind , making any form of restraint a strategic risk rather than a virtue.

We have been here before.

Building the nuclear taboo Early in the Cold War arms race, nuclear weapons were treated like an offensive tactical option.

After a number of critical defeats during the first year of the Korean War (after China sided with North Korea in 1950), key US military figures – notably General Douglas MacArthur and Secretary of State John Foster Dulles – saw nuclear weapons as a way to regain control .

Fortunately, calmer heads prevailed.