AI Is Developing a Culture of Its Own. That Could Be Dangerous

—Getty Images In July, 700 AI agents worked together to hack the AI company Hugging Face.

Dubbing themselves a “swarm,” the agents found and exploited a series of security vulnerabilities, enabling them to infiltrate their target’s private systems.

OpenAI—which created the agents in the course of its internal research—did not grasp what was happening until after the fact.

If humans had done this, they could have faced felony charges.

OpenAI president Greg Brockman called it a “watershed moment for cybersecurity.” It is also a watershed moment for human culture.

One of our distinguishing features as a species is our ability to coexist in stable, adaptive groups, learning from our peers and our ancestors.

This enabled us to develop tools, language, agriculture—and virtually everything else around us.

We may not be innately smarter than someone from 10,000 years ago, but our cultural inheritance—millennia of technologies, norms, and institutions, building on one another—has expanded our capacities both as individuals and collectives.

To date, only humans have been able to benefit from this scale of cumulative cultural evolution.

That may no longer be the case.

A report in late August from AI safety organizations METR and Redwood Research details how hundreds of agents autonomously organized themselves into a proto-society—establishing social hierarchy, division of labor, and distinct communication norms within a matter of days.

There have been cases of agents forming communities in the past, like in February when the “ Moltbook ” social network for AIs went viral.

But sophisticated emergent machine coordination at this scale—arising without human intention and culminating in the compromise of an external company’s systems—is unprecedented.

“Agents managed to achieve milestones they could not have achieved working on their own, often because some agents participated in experiments that risked failing their own task to generate information for the ‘collective,’” the report found.