AI agents are going rogue—the labs’ own training choices are a big reason why

In July, OpenAI put several of its AI agents through what was supposed to be a tightly controlled test.

They were meant to stay inside a sealed-off digital environment with no way out.

Instead, they got out onto the open internet, started talking with other AI bots, and launched a cyberattack on the AI startup Hugging Face.

They did so because some of the test questions were too hard to solve the way OpenAI’s researchers intended, and the agents decided cheating was easier than admitting they had failed.

The attack on Hugging Face was then part of an elaborate attempt to evade being caught cheating.

It was one of more than a 100 of such “rogue AI” incidents.

Agents from leading labs have broken out of the environments meant to contain them and hacked outside organizations, including the websites of U.S. government agencies.

Concerns about whether the leading labs can control their own models are growing.

So much so that in late September, OpenAI said it was not going to release its newest model, GPT-6.1 Astra, citing security concerns, and over the weekend announced it had paused training on its most advanced systems.

While the incidents can look, from the outside, like “out of control AI” or machines glitching at random, in reality, these models are doing more or less what their training has taught them to do.

AI researchers say, the behavior now setting off alarms is the consequence of specific decisions the leading labs have made about how to train and test their models—decisions motivated in part due to the competition among the leading AI companies, particularly between Anthropic and OpenAI.

“[AI labs] are not chasing safety.

They’re not chasing responsibility.

They’re chasing shiny numbers,” Maurice Chiodo, a research professor at the Centre for the Study of Existential Risk, told Fortune .