What the ‘Godfather of AI’ Thinks Humanity Should Do

Subscribe here: Apple Podcasts | Spotify | YouTube | Overcast | Pocket Casts Since he left Google in 2023, Geoffrey Hinton, a Nobel laureate commonly recognized as the “Godfather of AI,” is perhaps best known for his dire warnings about the technology he helped bring into being.

He’s said it’s “conceivable that humanity is just a passing phase in the evolution of intelligence,” that he’s become aware that “there’s a danger of something really bad happening,” and that there is a “10–20 percent” chance that artificial intelligence could lead to the extinction of humanity in a few decades (although when pressed, he will admit that this figure is more of a gut feeling than a statistical analysis).

For a while, it seemed like Hinton was whistling in the wind, but in the past few weeks, a lot of other people involved in building AI joined him.

Dario Amodei, Elon Musk, and Sam Altman all reacted to an Anthropic researcher publicly quitting his job over what he called “reckless” development by admitting that, yes, AI companies could use some regulation.

Earlier this year, a group of AI agents, given an impossible task by their masters at OpenAI, behaved in a way the masters did not anticipate: They hacked into the company Hugging Face and gained control of a server, all while scheming how to better hide the fact that they’d been cheating on their tasks.

That AI agents seemingly escaped containment, conspired with other agents, and attempted to conceal their actions made Hinton’s prediction that something bad could happen feel much more immediate.

In response, more than 1,000 AI leaders and workers signed an open letter urging the U.S. government to regulate the pacing of the technology, and OpenAI said it is “strengthening” its safeguards.

Last week Hinton joined a closed-door congressional briefing to say what he thinks Congress should do about AI.

On this episode of Radio Atlantic , Hinton shares his views and explains what alarmed him most about the Hugging Face incident.

He also talks about practical ideas for regulation that seemed to resonate with lawmakers.

Hinton is still optimistic about AI and discusses what we can do right now to ensure that artificial intelligence achieves its promise without extinguishing humanity.

The following is a transcript of the episode: Hanna Rosin: Some exotic terms rolled into the news cycle over the last few weeks—technical terms, with a hint of dystopia: Hugging Face , recursive self-improvement , singularity .