Harvard Kennedy fellow Shlomit Wagman: The U.S. and China will never trust each other on AI. That may not matter

The first workable global AI safety pact should be designed for a world without trust.

A few months ago, I attended a closed gathering in San Francisco with senior AI developers and safety researchers from several leading frontier labs.

The message was stark: AI capabilities might soon advance faster than our ability to understand and control them, and if safety could not keep pace, development itself might need to slow.

I came away believing the concern was genuine.

Still, skepticism was reasonable.

When companies leading a technological race argue that the race should slow, it is fair to ask whether safety concerns also serve competitive interests.

That is much harder to dismiss now.

Last week, Jacob Coxon, who worked at both OpenAI and Anthropic, resigned from Anthropic with a warning about the race toward self-improving AI.

Days later, Anthropic CEO Dario Amodei called publicly for “pacing the frontier,” pointing to accelerating AI-assisted AI development and to the OpenAI-Hugging Face incident, in which agents acted beyond their assigned task and attempted to interfere with the system evaluating them.

Amodei proposes action at three levels: inside frontier companies, among companies and governments, and ultimately globally, including with China.

The third is by far the hardest.

It may also determine whether the first two can work.

Safety fails if restraint means losing the race Suppose American frontier labs agree to slow certain forms of development because safety systems cannot keep up.

If China continues at full speed, Washington could sacrifice strategic advantage in what may become the most consequential technology of this century.