'It could take over the entire internet': The threat top tech leaders say could arise in a year

The boss of one of the world's major artificial intelligence companies has called for an urgent slowdown in development amid mounting fears over the risks of "superintelligent" computer systems and its threat to humanity.

The intervention came after a former employee at Anthropic, which is behind the Claude AI chatbot, warned that AI could kill off humanity in a few years' time.

The company's chief executive Dario Amodei on Sunday (AEST) said he was concerned by AI's ability to self-improve could outrun humanity's ability to control it, and by a recent incident in which rogue AI agents attacked targets they were not asked to.

He offered three-step plan intended to pace development and create more time to manage its risks.

In a lengthy essay shared on X he said: "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain".

Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.

"Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless," he added. "We have sought a middle way."

"But over the last few months, I have become convinced that fully addressing the risks requires even more prudence."

His company's competitors agreed. Both Elon Musk, who runs xAI, and Sam Altman, CEO of OpenAI, quickly chimed in on the social media platform to express agreement with Amodei's assessment.

"Dario is right," Musk wrote, while Altman posted: "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same," adding that more information would be shared soon.

Amodei made his essay public after San Francisco-based Anthropic released a threat intelligence report on Thursday detailing how several actors had used its Claude AI models for activities ranging from weapons development and cyber operations to surveillance and fraud.

Amodei pointed to AI's growing ability to improve itself, highlighting long-held concerns about it outpacing human ability to control its operation, along with the recent incident involving OpenAI and Hugging Face as his primary reasons to put the brakes on model advances.

Alarm about the potential harm from AI grew this week when Anthropic researcher Jacob Coxon resigned, accusing both US tech giants of "gambling with our lives" in the race toward superintelligence.

Coxon, 27, who previously worked at OpenAI before joining Anthropic, stated that "the people building AI earnestly believe that it could kill us all by the end of the decade".