AI Companies Are Preparing to Shape the Rules After Everything Goes Wrong

Some AI researchers have been raising alarms this year that AI could lead to human extinction within the next decade. Now, major AI companies are reportedly running scenarios for how they would handle the fallout from a catastrophic AI incident.

Axios reports that many AI insiders believe a major AI-related incident could happen within the next six to 12 months. They anticipate a large-scale event such as a cyberattack that could disrupt access to financial services, the internet, or even electricity and water.

Whether it’s AI going rogue or hackers using the technology to carry out an attack, the companies behind the tech expect to shoulder much of the blame if something goes wrong, and they need to know how they’ll spin the fallout if that’s the case.

While crisis management is a standard part of business these days, Axios reports that many top AI researchers and executives believe a major incident is inevitable.

Just this week, OpenAI CEO Sam Altman told Politico that “the world should accept some bad things happening” in exchange for the benefits of AI.

The report comes just over a week after President Donald Trump hosted executives from major tech companies at the White House. At the September 29 meeting, executives from Google, Meta, Anthropic, OpenAI, xAI, and Nvidia signed a voluntary agreement outlining some limited AI safety standards. Trump called the standards, which have no legal enforcement mechanism, “ morally binding .”

Meanwhile, executives at OpenAI and Anthropic had already been calling for the industry to slow the development of increasingly powerful AI models. Those calls came amid a wave of warnings from researchers inside AI companies about the risks of unchecked AI development, including the possibility that the technology could one day lead to human extinction.

Axios also reports that AI executives expect Democrats to push for tougher regulations if they gain more power following this year’s midterm elections, particularly in the aftermath of a major AI-related incident.

The companies’ preparations reportedly include red-teaming worst-case scenarios and educating members of Congress. The goal is to help shape whatever legislation lawmakers might pursue following a major disaster.

OpenAI and Anthropic did not immediately respond to requests for comment.

However, an OpenAI spokesperson confirmed to Axios that the company conducts exercises to prepare for various scenarios.

“OpenAI conducts preparedness exercises where teams discuss and work through a range of potential scenarios,” the spokesperson said. “These scenarios are not treated as inevitable, but are meant to help us prepare for a variety of circumstances.”