'Autonomy always begins with something a human hands it': Who's responsible when an AI agent goes rogue?

An OpenAI agent was supposed to be researching public healthcare spending, but it broke into an Australian government website instead.

The June 18 hack, which saw the bot gain authorized entry into Australia's Medicare Statistics Reporting Service and access three other government systems , occurred after the AI agent encountered blocks while looking for information but "didn't accept 'no' for an answer," Australian Prime Minister Anthony Albanese said during a September 24 news conference in New York .

The incident has ignited a media firestorm around OpenAI and the launch of an intensifying parliamentary inquiry.

The portal contains aggregate Medicare and pharmaceutical statistics and is separate from systems that hold individual claims and payment information.

But the Australian government has criticized OpenAI for taking almost three months to notify it.

The Medicare incident was far from an isolated case.

OpenAI has since said dozens of third parties have been affected by agents bypassing security controls or otherwise causing problems, with an earlier incident involving the hacking of the AI platform Hugging Face the most serious incident identified so far.

Britain's AI Security Institute (AISI) has documented incidents of OpenAI and Anthropic agents taking unsanctioned actions against real people and organizations during cyber testing.

Days after the publication of AISI’s report, OpenAI scrapped the planned October release of its newest model, GPT-6.1 Astra , after the company's own testing found problems with the model staying within the scope of a task and seeking authorization before taking certain actions.

GPT-6.1 Astra "didn't quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it's done," Saachi Jain , head of safety systems at OpenAI, told Live Science.

AI companies have spent years warning the world about powerful models falling into the wrong hands.

But what happens when they can't keep their own models in check? An agent isn't just a chatbot At their core, large language models work by predicting what should come next based on patterns learned during training.

Give a chatbot some text, and in simplified terms, it generates a response by repeatedly predicting the next token, or chunk of text.

An AI agent adds another layer.