Anthropic CEO Dario Amodei: "For too long the industry lied" about AI risks

"I won't lie to you – there are real dangers," said Anthropic CEO and cofounder Dario Amodei about the development of artificial intelligence. "And I think for too long the industry lied to people about the fact that this technology had risks."

And the biggest risk could be the end of humanity, former Anthropic researcher Jacob Coxon warned on social media this past Tuesday, saying Anthropic and OpenAI are "gambling with our lives" with the development of self-improving superintelligence.

"It doesn't look that different from, say, 'Terminator' or from science fiction films," Coxon told CBS News Thursday . "It will be smart enough to kill us."

In response, Amodei told "Sunday Morning" yesterday, at Anthropic's headquarters in San Francisco, "I don't think I fully just appreciated what it would actually be like when the progress was as fast as it was."

He said the development of AI has been on an exponential curve, which is starting to get steeper. Asked what that means, Amodei said, "It doesn't mean we need to panic today. It doesn't mean we need to shut it all down. But I would say it's a warning sign. It's a warning sign that we need to slow down."

In an essay published Saturday, Amodei wrote that the entire tech industry must work together to "slow the pace" of AI capabilities .

Asked if Anthropic is going to stop releasing more advanced models, Amodei replied, "It doesn't mean that. What it means is that we need to make sure that every generation of models that we release is properly tested."

To attempt that, Amodei says Anthropic will now give independent evaluators permanent "employee-like" access to their models. "Whenever anyone builds an AI model, there's a third-party evaluator who can verify whether that company is following the safety practices that they're committed to – it's like a food inspector," he said.

But AI experts say this is hardly enough, and Amodei agrees, claiming he supports federal regulation. So far, proposed legislation ranges from an outright ban on superintelligence, to a mandatory "kill switch" that could shut down risky AI.

"The AI kill switch could be a good idea," Amodei said. "But I don't think banning it, completely banning it, is the right approach. One, the idea that we would never get the incredible benefits, never get the medical cures, that doesn't sit right with me. Second, this technology will be built in other countries, including authoritarian countries."

Asked if the U.S. should slow down AI development if China would not, Amodei replied, "This is the toughest dilemma. This is the toughest aspect of our situation."

He looks to the nuclear arms agreements with the former Soviet Union as a potential model, but cautions even his expectations are low. "The more long-term thing would be working together to put a speed limit on the rate of AI progress," he said. "I think that's going to be very difficult, because the incentives to pull ahead in the military advantage that you get from that are so large.