AI safety: Silicon Valley's fear narrative meets its governance gap

When Jacob Coxon resigned from Anthropic in September and warned that artificial intelligence (AI) could end human civilization, his post drew over 170 million views in 48 hours.

But the more revealing story is not why one researcher left – it is why the industry's most safety-conscious talent can only express concern by quitting.

Coxon's warning arrived during the same week that Anthropic was finalizing preparations for what could become the largest IPO in market history.

Anthropic filed a confidential S-1 with the US Securities and Exchange Commission on June 1, targeting a valuation of $965 billion, and expects to complete its listing before the US midterm elections in November.

Nvidia is in talks to anchor the offering at a $2 trillion valuation, Reuters reports.

The safety pledge that disappeared Anthropic CEO Dario Amodei has publicly called for slowing AI development so safety systems can catch up, yet his company's IPO timetable shows no corresponding deceleration.

In February 2026, Anthropic released Version 3.0 of its Responsible Scaling Policy, which dropped the company's earlier commitment to pause model training if risks could not be adequately mitigated.

The "pause training" pledge simply disappeared from the document, replaced by softer language about "responsible development" and transparency mechanisms.

As an analysis from Chinese technology news outlet 36Kr noted, Anthropic's safety red lines have become dynamically adjustable.

This is the structural contradiction at the heart of the AI safety debate.

Companies warn of civilizational risk while racing to commercialize the technology at unprecedented speed.

OpenAI CEO Sam Altman captured the irony bluntly, comparing Anthropic's approach to fear-based marketing: "It is clearly incredible marketing to say, 'We have built a bomb.

We are about to drop it on your head.

We will sell you a bomb shelter for $100 million to run across all your stuff, but only if we pick you as a customer.'" Beyond the IPO The tension between safety rhetoric and commercial behavior extends beyond IPO filings.