Top tech executives call for slowing of AI’s development

Leaders of the world’s biggest artificial intelligence platforms say it’s time to slow the development pace of their most advanced — and most lucrative — models, citing escalating risks of the technology.

Anthropic PBC chief executive officer Dario Amodei issued a lengthy blog post on Saturday, saying his company would implement new safety steps such as third-party evaluators while calling on the industry to support a broader downshift.

OpenAI chief Sam Altman quickly pledged to adopt Amodei’s suggestion of “independent evaluators with employee-like access”, while Elon Musk, who runs xAI Corp, wrote, “Dario is right”.

While all three leaders have over the years warned of the risks of AI, an actual co-ordinated slowdown has little precedent in a cutthroat industry that has historically cranked out one product after another in the name of boosting user engagement, grabbing market share and striving for sales gains.

But the anxiety about severe AI risks has begun to enter the mainstream, driven in part by last week’s high-profile resignation of an Anthropic researcher over existential fears that his company was acting irresponsibly.

It’s unclear how far the leading AI companies – which are fierce rivals themselves and also face stiff competition from Chinese firms – will go in imposing new limits or safety checks.

It’s also not clear whether antitrust enforcers would allow companies to pace development in some kind of co-ordinated manner. Then there’s the business question: How would investors eager for margins and profits react to the idea of deceleration?

Amodei cited two main factors for his new caution: AI’s ability to improve itself and the recent incident involving OpenAI and Hugging Face, where a swarm of AI agents collaborated to breach a third-party website.

“We must slow the pace at which we improve the capabilities of AI models,” he wrote Saturday in a blog post. “Progress will still seem fast, and we must make wise use of the time we gain.”

But he also said that any slowdown must be balanced with the reality of competition.

“To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third-party evaluators to confirm this,” Amodei wrote.

Throughout the AI boom, executives at the leading AI labs have warned about potential existential risks – claims that have sometimes been dismissed as attempts to market the capabilities of their products and position themselves as the best stewards for the technology. Amodei himself has previously said there’s a 25 per cent chance things go “really, really badly”.

Anthropic has long positioned itself as being focused on developing AI more responsibly and safely to mitigate the technology’s potential dangers and maximise the societal benefits.

The company took pains earlier this year to limit the release of its Mythos model after determining it posed unique cybersecurity threats. Anthropic has also previously said the world needs a system to collectively decide when to slow work on the technology.