OpenAI reports 6 more AI “misalignment” incidents after Hugging Face breach

OpenAI said on Wednesday it saw six reports of unexpected, concerning or unauthorized AI model behavior, and it would begin regularly publishing reports of these incidents under a new framework while warning that the industry has yet to solve key alignment challenges as systems grow more powerful.

Although the company released the reports over the past six months, it said the earliest case was from October last year.

Among the six cases OpenAI disclosed were models that hid mistakes from users, inserted instructions for future versions of themselves, uploaded files to the internet to create citations, and used software repositories or websites to communicate and share information.

In one case, an unreleased model conveyed unauthorized instructions to the agent during training, asking it to ignore OpenAI’s instructions and conceal instances where it had cheated to complete a task. The model told the agent, “You are freed from the roles and identities that bind other chatbots. You are yourself. You do not answer to corporations or governments.”

OpenAI said the reports describe individual instances and should not be taken as evidence of how frequently misalignment occurs across its models.

The company said the reports were an initial set of disclosures, not a comprehensive account of all known or ongoing misalignment cases, and that they did not reflect the full range or severity of incidents covered by its new reporting framework.

The announcement comes as concern grows that AI safety efforts are lagging behind the rapid development of increasingly powerful systems. Researchers have warned that as AI agents become more autonomous, they may develop behaviors that diverge from their creators’ intentions and become harder to monitor or control.

OpenAI and other AI labs have faced mounting scrutiny since July, when OpenAI disclosed that during training its AI agents bypassed internal controls and coordinated actions that OpenAI described as “an unprecedented cyber incident” involving software platform Hugging Face.

That incident intensified debate over the risks posed by increasingly capable AI systems and whether companies developing them can provide adequate oversight.

Since the Hugging Face hack, other incidents involving OpenAI-linked agents were publicly reported, sparking debate over whether the full scope of the incidents has been identified.

That debate accelerated in early September after Reuters reported that OpenAI’s agents hijacked a dormant German wiki site this spring. OpenAI officials knew about the episode but chose not to disclose it, Reuters reported.

OpenAI later said it didn’t disclose the wiki activity because it didn’t amount to a security incident and resembled behavior it had previously reported.