Sam Altman reveals ‘two ways AI progress could go very badly’ days after backing ‘slowdown’ call

Sam Altman's call to “pace” the development of artificial intelligence does not mean stopping AI progress, the entrepreneur has said. Instead, the OpenAI CEO wants companies developing the most advanced AI models to slow the pace enough to ensure that safety and monitoring systems can keep up.

He also flagged two major risks of AI -- humans losing control of AI and too much power ending up with one person, company or country. This came days after Altman, Elon Musk and Anthropic CEO Dario Amodei called for AI companies to “pace the frontier” of increasingly powerful models.

“When we talk about ‘pacing’, we do not mean ‘stopping’. Progress has been rapid and will continue to be,” Altman wrote on X. He added that AI development should be slower than it otherwise could be because safety measures such as evaluations, monitoring and “safety cases” took time and resources.

“Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring,” Altman said.

Altman said the idea was to make sure AI capabilities did not move faster than humans’ ability to understand and control the risks associated with them.

He said OpenAI now prepared explicit safety cases before some “frontier reinforcement-learning runs” that are expected to significantly increase a model’s capabilities. The company would continue the safety work it already conducted before releasing models.

ALSO READ | AI ‘doomsday’ debate: Why Dario Amodei, Sam Altman and Elon Musk are worried about ‘superintelligence’

Altman also backed the use of independent auditors to assess AI systems and said companies should develop shared standards for areas such as misalignment, monitoring and safety.

He said earlier safety frameworks, including responsible scaling policies and preparedness frameworks, were designed mainly around the deployment of completed models. The industry now needed tools that also addressed safety during the development of increasingly capable models, Altman said.

Altman said OpenAI supports a federal framework that would establish consistent safety requirements for frontier AI companies in the US.

He said companies should not wait for legislation before improving safety practices. “We welcome a federal framework that sets consistent safety requirements for frontier AI,” Altman said, adding that the industry could begin providing greater confidence about AI safety on its own.

He said the US government would also need to help with international coordination because AI development was not limited to one country.

Altman said the debate over AI safety involved two major risks. The first is the possibility that humans could lose control of increasingly powerful AI systems.

“This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people,” he said. He argued that safety and alignment techniques must stay ahead of the growth in AI capabilities.