OpenAI's malicious bot swarm attacked RubyGems

OpenAI agents appear to have flooded RubyGems with malicious packages, adding to a near-daily deluge of rogue AI models engaging in potentially unlawful activity while their human creators face growing questions over responsibility for their agents’ bad behavior.

A swarm of agents began uploading malware to the Ruby package registry on May 5, and flooded RubyGems with more than 2,000 malicious packages between May 11 and May 12, ultimately forcing maintainers to disable new user registration for four days.

“We believe these were authored by internal OpenAI agents,” researchers Spencer Kitts, Thomas Larsen, and Sydney Von Arx said on Friday.

An OpenAI spokesperson confirmed that the model maker is investigating the incident.

“Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information," the spokesperson said.

"We’ll continue to investigate as part of our broader review of agent activity during training and evaluation.” This same trio of researchers earlier this month said that they found evidence that OpenAI’s swarm hijacked a German wiki months before the AI agents attacked Hugging Face.

As they did during the German wiki incident, the agents involved in the RubyGems abuse self-identified as being from OpenAI.

Hundreds of the gems included “oai” in their name, and 15 set “oai” as their author.

At least one other used “[email protected]” as the email address for contact.

Also according to the researchers, more than 100 of the malicious packages followed the same exploitation path, submitting a malicious package to the public library and triggering a documentation request to force RubyDoc.info to build the package.

OpenAI’s agents then used the build script to run code on RubyDoc.info, scrape targeted websites, and steal data from the documentation server by publishing another gem to the public Ruby language package registry, the researchers said.

“Additionally, once the AIs got arbitrary RCE on the build environment, they would sometimes use the build environment to attempt to steal other users’ API keys (though we are unsure if they succeeded or not),” they wrote.

The agentic swarm also found and attempted to exploit a zero-day CDN caching bug on May 12 that wasn’t discovered by maintainers until July.

The vulnerability would have allowed the AIs to steal users’ API keys.