What is an "AI swarm," and why it giving tech experts nightmares?

Central to the dire warnings about artificial intelligence's potential danger to humanity is the risk that a "swarm" of AI agents could collaborate in nefarious ways, like hordes of digital extras from The Matrix.

Such fears have intensified following an attack by OpenAI bots this summer on another AI developer, Hugging Face, in which roughly 1,200 AI agents divvied up tasks to execute the hack and hide their tracks from human researchers.

But what, exactly, is an AI swarm? And is the technology's emergent capacity for coordination a genuine threat to humanity? Put another way, what happens if AI agents collectively embrace Mark Zuckerberg's famous dictum about the merits of rapid tech innovation and decide to "move fast and break things"?

An AI swarm is a group of AIs that work together to accomplish a shared goal. That goal isn't necessarily malign or destructive. For example, hospitals could deploy agents to retrieve patient records and perform other administrative tasks, such as coordinating admissions. A swarm could also be tasked with advancing biomedical research.

But David Scott Krueger, an AI safety researcher and founder of Evitable, a nonprofit organization advocating for a moratorium on AI development, offers a simple thought experiment to explain how a swarm of agents might collude to defy their developers' instructions.

Imagine removing the handcuffs from a group of prison inmates to see how they behave once the shackles come off. Being liberated in this way would make it easier for them to cooperate and contact allies beyond the prison walls — just as OpenAI agents escaped their testing environment to access the wider internet and hack Hugging Face, he said.

"Normally, the systems would have guardrails on them, but they took them off for a test, just like a prisoner is normally in handcuffs," Krueger explained.

The kind of errant actions by a single bot, such as sending an unauthorized email, that stem from, say, a sloppily worded AI prompt, should be distinguished from an AI swarm. In the latter case, hundreds or even thousands of agents could coordinate their actions in ways that violate the scope of their instructions.

A useful analogy in understanding how an AI swarm operates is a colony of bees, Krueger said.

"They are all working together toward the benefit of the hive, and have a hivemind, or may even be better viewed as having one single mind," he explained. "Bees collect food, reproduce, they fight off predators, and all of that activity is in the service of promoting the survival and reproduction of that hive."

And like bees, which can divide and allocate tasks on their own without direction from a queen, swarming AI bots can collect information, consider solutions, and take action independently. In service of a common purpose, they share information and knowledge, according to Rob T.