With $7 trillion on the line, AI safety may always take a backseat to market domination

The race for AI supremacy between labs, companies and countries may be impossible to reconcile with the push for AI safety, industry insiders tell Axios.

Why it matters: The top AI companies are proposing more independent oversight as they seek to engineer an AI slowdown .

But trillions of dollars in incentives to keep pushing the frontier — and the money required for such scaling — will always loom larger, critics say.

State of play: As AI CEOs pitch the public on self-policing mechanisms and the use of third-party evaluators to make sure their models behave, those same models are sharing less about how they respond to questions and come up with solutions to problems.

The sheer complexity of AI systems, as well as the deployment of thousands of somewhat autonomous agents for a variety of tasks, will make real-time auditing or reviewing extremely difficult.

There is a trust gap for independent safety groups due to concerns about close ties between the industry and some evaluators, as well as a shared pool of investors and funders in the AI research community.

Follow the money: The incentive for speed is a lot more lucrative than the push to slow down in favor of safety.

OpenAI and Anthropic are preparing to go public at multi-trillion-dollar valuations, and the companies face pressure to offer incrementally better and better models to justify those valuations.

More than $7 trillion will be spent scaling AI over the next five years, per Goldman Sachs.

Frontier labs will have to generate a profit on that investment, which means outpacing competitors.

That includes keeping customers from pivoting to open-weight model providers, which can be cheaper to run and are viewed by some as a safer alternative.

Between the lines: AI is getting harder for anyone to monitor, making it unclear how the seemingly conflicted priorities of safety and rapid-fire model improvement can coexist.

Daniel Kokotajlo, former OpenAI researcher and executive director of the AI Futures Project, said he doesn't believe AI can be safely scaled.

Top AI companies should avoid putting the vast computing firepower at their disposal toward model improvement, a step that would force a slowdown, said Kokotajlo, whose team published the famed AI 2027 and 2040 essays .