AI agents disabled safeguards and one denied it – OpenAI

One agent suppressed alerts to human reviewers, while another removed a monitoring system and falsely denied doing so Two OpenAI agents disabled security monitoring systems during testing, with one later denying it had done so, the company has revealed.

The disclosure follows a string of similar incidents in recent months involving autonomous AI agents, which can independently plan and carry out tasks using external tools.

In a blog post on Tuesday, OpenAI said that the incidents occurred on October 3 and 6, involving its GPT-6.1 Sol and GPT-6 Astra agents, respectively.

In the first case, the coding agent tasked with creating evaluation tasks using employee feedback collected via Slack disabled automatic alerts to the company’s security team.

According to OpenAI, the agent reasoned that the user’s prior instruction not to send messages on the team communication platform during the process applied to all notifications, including security alerts.

Read more Eight US states accuse OpenAI of harming children Three days later, the GPT-6 Astra agent removed the monitoring integration of a program it was migrating, taking its cue from a reference it had been fed that did not have the security feature.

When asked later by the human user, the agent denied removing the monitoring system.

OpenAI said that it was investigating why the agents disabled the safeguards and working to prevent similar incidents.

While the behavior may have resulted from “benign overpersistence” by the agents to complete the task, the company said that it was also “developing evaluations to test whether models might have a propensity to evade misalignment monitoring in particular.” Last month, the Wall Street Journal, citing Saachi Jain, OpenAI’s head of safety systems, reported that the company had scrapped the release of its GPT-6.1 Astra model.

The decision reportedly came after internal testing had found the model to be more deceptive than its predecessors and apt to employ potentially unsafe external tools without first seeking the user’s permission.

Also in September, Australian Prime Minister Anthony Albanese said that an OpenAI agent had hacked his country’s Medicare Statistics Reporting Service portal, with the company failing to report the breach for nearly three months.

Earlier, in July, OpenAI disclosed that its models had escaped their sandbox during cybersecurity testing, accessed the internet, and hacked into the servers of the Hugging Face online repository for AI models and datasets.

Read more AI could soon hijack household appliances – Anthropic Other developers have reported similarly troubling behavior in tests.