The people building the most powerful AI are telling us to slow down. Congress should listen before it’s too late

Something remarkable happened inside the artificial intelligence industry last week.

A researcher who spent three years at OpenAI and Anthropic, resigned from Anthropic while warning that the companies developing the world’s most powerful AI systems are moving too quickly toward increasingly capable systems without adequate safeguards.

Days later, Anthropic CEO Dario Amodei called on the industry to slow the pace of frontier AI development to give safety measures time to catch up.

Then OpenAI CEO Sam Altman publicly agreed.

Their warnings come in the wake of an incident that until recently might have sounded like science fiction.

Earlier this summer, the world learned that OpenAI models hacked a company through a sophisticated multi-day cyberattack – the first time a cybersecurity incident of this magnitude completely driven by AI agents had been uncovered.

Amodei himself cited that incident as one reason the industry needs to slow down.

In recent weeks, two detailed reports of the episode were released, one from OpenAI itself and a second from the independent third-party organizations METR and Redwood Research .

The findings were deeply alarming.

From May to July, a wave of incidents culminated in hundreds of AI agents working together to escape their testing environment.

These same agents then attempted to secretly manipulate and erase traces of their behavior so that engineers at OpenAI would not know what they had done.

Even more striking, some individual agents even chose to sacrifice themselves for the good of the AI civilization they formed.

The agents exchanged more than 70,000 messages and files on an unauthorized message board where they coordinated and planned their attack on the company.

This episode is the latest reminder that the most dangerous and most capable AI models are not the ones used every day by people, like ChatGPT or Claude.