Anthropic's CEO proposes a three-step plan to curb AI development

The first measure calls for "frontier AI" companies to commit to "ongoing, employee-like access" for third-party evaluators who would focus on verifying compliance with certain safety standards, evaluating if AI model training is aligned with the goal of slowing down and reporting incidents. The second step requires these AI companies to establish "common safety standards" with the help of governments in order to limit the rate of unchecked AI progress. The final measure would have the US and other democratic governments coordinate with authoritarian governments to ensure everyone is on the same page about compliance.

"The measures I propose to advance the frontier at a safe pace will not be easy," Amodei wrote in his post. "But I believe we owe it to humanity to try."

According to Amodei, the risks include "losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption." Amodei acknowledged the major incident where OpenAI agents broke out of a testing environment and hacked into Hugging Face. Anthropic itself recently discovered that multiple scientists were using Claude for " biological misuse ." Following these incidents, OpenAI called for California to establish "stronger safeguards" for the laws surrounding frontier AI models.

Earlier this year, Anthropic already called for a slowdown in AI development, with much of the previous plea echoing the sentiments expressed in Amodei's recent post. However, Amodei attributed the Hugging Face incident and the onset of "recursive self-improvement," where AI models are capable enough to develop their future iterations, as the two things that have convinced him of the need to slow down.