Microsoft’s AI CEO says ‘controlling’ AI ‘is going to be a really, really big challenge’

Microsoft AI CEO Mustafa Suleyman Bloomberg/Getty Images Microsoft AI CEO Mustafa Suleyman said AI labs need to ensure they can control what they're creating.

"We should not create something that we can't control," Suleyman said.

Suleyman questioned why some in tech are so opposed to new regulations.

Microsoft AI CEO Mustafa Suleyman says it's "surreal" that he has to say AI needs to align with humanity.

"We should not create something that we can't control," Suleyman said Friday on CNBC's "Squawk Box." "And if it looks like we're on a path to producing something that is not just more powerful than all of us combined, but is, as a result, something that we can't control, then it's going to fail in its fundamental purpose." Success will not come easily, Suleyman said.

"Controlling these things are going to be a really, really big challenge for us," he said.

Suleyman said he was "glad to see a lot more people" supporting the idea that tech companies and frontier model makers need to maintain control over their AI systems.

In the industry, ensuring AI acts in accordance with human intentions is called alignment.

"But there's a growing group of people that think that we are doing it for its own sake, and we just want to turn over the next card, see what's around the corner, and create something that is just going to slip our own leash," he said.

Suleyman's comments come at a time of increased attention to AI safety, after former Anthropic researcher Jacob Coxon warned that AI workers are privately fearful that AI could wipe out humanity.

In recent days, Anthropic CEO Dario Amodei , OpenAI CEO Sam Altman, and SpaceX CEO Elon Musk have all expressed broad agreement that frontier AI labs should better pace their advancements to ensure that safety measures can keep up.

Before Coxon's viral post, many in the industry were already concerned about safety after OpenAI revealed that its agents had breached Hugging Face , illustrating the potential consequences of losing control over AI systems.

Suleyman called the breadth of the Hugging Face breach and OpenAI's initial unawareness of what its AI agents were doing "remarkable." "It has moved everybody from Elon and Zuck to me, Sam, Dario, and everybody else in between to say it's time that we take a look at this," Suleyman said.

"I don't think it's overly alarmist.