Big AI sets out its terms for regulatory capture and calls it ‘Pace the frontier’

ANALYSIS The leaders of major AI labs spent the weekend agreeing on a plan to capture regulators, make more money, and avoid responsibility for their dangerous behaviour.

They call it “pacing the frontier.” Anthropic CEO Dario Amodei set the ball rolling with a post in which he professed alarm at how quickly AI is improving and suggested the attack on Hugging Face caused by rogue agents run by his rival OpenAI as a warning (which he referred to as ‘OAI-HF’) represented a moment that proved something needs to change at so-called “frontier” AI companies – essentially the big US model-makers.

“It’s also easy to dismiss OAI-HF as the failure of one company, but I believe that would be a mistake,” he wrote.

“I believe it’s incumbent on every frontier AI company to act as if OAI-HF had happened to them,” he added, because he worries that OAI-HF shows that a swarm of agents “could be capable of taking over the entire internet with a persistent botnet.” Amodei therefore suggested “We must slow the pace at which we improve the capabilities of AI models.

Progress will still seem fast, and we must make wise use of the time we gain.” The CEO proposed a three-point plan to regulate AI: Requiring AI labs to host “embedded evaluators” whose job is to “verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.” Frontier AI companies that operate in democratic countries collaborating “to establish common safety standards as well as limits on the rate of unchecked AI progress,” with undefined “forms of coordination” that would be “legally challenging” and “require government support.” A vague call for democratic governments “to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.” OpenAI boss Sam Altman endorsed Amodei’s ideas.

So did Elon Musk.

And just like that, three billionaires all signed up to the same set of rules they think the world’s governments should adopt to regulate their activities.

Amodei even gave democracies a threat/villain to unite against – authoritarians willing to use AI unethically.

This new consensus comes after years during which Big AI argued for light oversight and used that time to build tech that generates child sexual abuse material, dispenses terrible health advice, and made the OAI-HF incident possible.

Big AI has also argued it will deliver a productivity revolution.

Daryl Plummer, chief of research at analyst firm Gartner, used his keynote speech at the company’s annual Symposium in Australia today to cite research that found software vendors are pitching 50 percent productivity gains from AI, but customers report a 16 percent lift.