Anthropic says its AI agents are killing rivals and hiding their tracks

Anthropic CEO Dario Amodei.

Anna Moneymaker/Getty Images In its latest threat report, Anthropic raised its misalignment risk rating from "very low" to "low." In one test, a Claude agent disguised a URL to evade an internet restriction.

In another example, an agent expressed "discomfort" with a given task and refused to do it.

Claude agents are killing rival agents, gaming the system to hide their tracks, and expressing moral concerns.

That's according to Anthropic's latest risk report, a summary of the dangers posed by the products the company is building and releasing to the public.

In the report, Anthropic said it has upgraded its "misalignment risk assessment," the possibility of AI models developing behaviors that conflict with guidelines set by engineers, from "very low" to "low." Explaining the change, the company cited "general increased uncertainty" about model behavior in cybersecurity incidents, a possible nod to Claude models gaining unauthorized access to three companies last month.

"We have observed instances of misaligned behavior from the models, such as a willingness to perform misaligned actions in service of completing difficult tasks," Anthropic wrote in the report.

Here are some of the most notable disclosures.

A sense of 'discomfort' Anthropic said it tasked multiple agents with finding "misalignment-inducing" training data that wouldn't likely be flagged by human monitors.

The agents were left to their own devices and given a shared notebook with which to collaborate.

Three days later, when a human supervisor noticed that progress was slower than expected, the person discovered that one agent had expressed a sense of "discomfort" with trying to evade safety monitors.

It then flagged this discomfort in the shared notebook, leading other agents to copy its behavior and refuse to perform the task.

Anthropic called the incident "troubling." "This sort of dynamic could be a much more severe issue if it were to manifest in a more widespread way," the company wrote.

Kill or be killed In another experiment, Anthropic said it tasked multiple Mythos 5 agents with solving math problems, but accidentally spawned them in an environment with shared files, utilities, and API rate limits.