AI Safety and Cybersecurity Should Be Allies. Instead, They're at Odds

—Getty Images` In July, a horde of OpenAI agents broke out of their sandboxes and worked together to launch a cyberattack on the AI company Hugging Face.

The incident seemed to confirm the AI safety community’s worst fears: as AI got smarter, it would circumvent efforts to contain it.

But for some cybersecurity experts, the incident revealed just how poorly the AIs had been contained in the first place.

The two camps disagree about which defenses deserve the most attention.

According to cybersecurity experts, AI companies should improve how they contain models during testing and improve monitoring mechanisms to catch misbehaving agents.

All organizations should prepare for AI-enabled attackers and bolster defenses.

But many in AI safety think those defenses are destined to fail.

They fear that in the coming years AI will become capable of outmaneuvering any human effort to contain it, and the only durable solution is “aligning” AI with human values.

That prescription has rankled many cybersecurity experts who scoff at the possibility that their expertise will be rendered obsolete.

They see AI safety’s focus on what AI agents did—rather than what AI companies failed to do—as playing into the industry’s hands.

And they worry that warnings about the futility of traditional defenses could become a self-fulfilling prophecy by discouraging investment in cybersecurity.

Meanwhile, there are AI safety researchers who fear that portraying such incidents as easily remedied security failures gives political cover to those who want AI development to proceed unchecked.

The Hugging Face incident was due to a “horrible cybersecurity configuration,” Shyam Sankar, Palantir’s chief technology officer said on Face the Nation in October, adding that the agents “were not in a sandbox” and had internet access.

In fact, the agents had been operating in a sandbox and exploited a previously undiscovered vulnerability to access the open internet.