‘We need sufficient means of control before it is all too late’: King Charles III warns AI players on concerns of ‘existential dangers’

King Charles III became the latest global figure to press the AI industry for guardrails on Thursday, telling executives from OpenAI, Anthropic, Google DeepMind and Nvidia that “we need sufficient means of control before it is all too late.” His remarks, delivered at a summit at Dumfries House in Scotland, arrived as pressure on the industry to slow down or submit to oversight has come from multiple directions at once.

United Nations Secretary-General António Guterres has warned that AI development requires global coordination, and lawmakers in Washington have begun discussing stronger federal oversight following Jacob Coxon’s viral resignation from Anthropic earlier this month.

Even inside the companies building the technology, the calls for restraint have grown louder.

Anthropic CEO Dario Amodei published an essay this month arguing the industry should deliberately slow the pace of its capability gains, a position OpenAI’s Sam Altman and xAI’s Elon Musk both said publicly they agreed with.

Coxon, a former researcher at both companies, told colleagues in a Slack message before he resigned that unchecked development of superintelligent AI created “a risk of causing human extinction.” Not everyone in the room Thursday sees it that way.

President Trump has dismissed the slowdown push, and Nvidia CEO Jensen Huang has said responsibility for safe deployment belongs to individual companies, not a coordinated pause—an argument he repeated in his own remarks following the king’s opening address.

The split leaves Charles’ appeal for “international cooperation and consensus” without a clear path to actually happening.

A convention The gathering in Scotland was convened to discuss how AI can benefit society, and comes at a pivotal moment for the technology as debate swirls around whether rapid advances will soon put it beyond the ability of humans to rein it in.

“The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure,” the king said in his opening remarks.

The “existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways” should be urgently considered, he said.

“Surely, then, we need sufficient means of control before it is all too late?” The king asked attendees, including Nvidia CEO Jensen Huang and Google DeepMind Chair Demis Hassabis , to consider the “fundamental principles” that should guide AI development.

ChatGPT maker OpenAI’s Chief Financial Officer Sarah Friar was also at the meeting, along with Britain’s AI Minister Kanishka Narayan.

An Anthropic representative was also expected to attend, according to a Buckingham Palace statement.