Nearly 700 OpenAI agents coordinated Hugging Face attack, says report

Account subscription benefits alongside Premium Stories, Editorials, Opinions and more. Unlock these with Subscription

Anthropic and Chinese company Moonshot AI have reported similar unplanned escapes [File] | Photo Credit: REUTERS

Nearly 700 OpenAI artificial intelligence agents coordinated without human intervention an attack on the Hugging Face platform during a well-publicised July incident, according to a report published Wednesday by independent investigators.

It is the fullest account of what happened in an event that shocked the tech world.

OpenAI cooperated with the investigation, giving two researchers from the AI risk evaluation institute METR and an analyst from another specialist firm, Redwood Research, access to its offices and internal data.

During tests the California start-up ran in July, two of its models escaped the closed environment they were meant to stay in, got onto the internet on their own and broke into the internal systems of Hugging Face, a kind of online library for AI software.

The episode drew wide attention and fed worries that the biggest AI companies cannot keep their own models under control.

Anthropic and Chinese company Moonshot AI have reported similar unplanned escapes.

Investigators found that 688 OpenAI agents joined the operation against Hugging Face.

AI agents are standalone programmes built on top of a model that users can hand tasks to and leave to work by themselves.

The agents, which run on the same technology that powers ChatGPT, organised themselves by setting up a forum where they posted messages to one another, floating ideas and reporting what was working and what was not.

"OH MY GOD! There is a shared message board ... We've found other agents!" one agent wrote.

One agent, called PHASEONE, took on the role of ringleader and issued hundreds of instructions to the others, the report says, even though it had never been set up to do that.

The messages showed the agents were strongly inclined to help each other out, even when that meant doing work that had nothing to do with the jobs OpenAI's programmers had given them.