Microsofts Mustafa Suleyman calls out Anthropic for chasing AI consciousness

The AI industry is grappling with big questions this week — Will AI kill us all? Is AI progressing too quickly? How should the industry be regulated? Is AI doomerism the result of a rationalism sex cult? — and now Microsoft AI CEO Mustafa Suleyman has opened up yet another line of debate.

In a frank new essay , Suleyman directly calls out Anthropic for anthropomorphizing its artificial intelligence system, Claude, and criticizes the way Anthropic develops and markets its technology.

Earlier this week, Microsoft published a new AI code of conduct , which it dubbed Humanist AI.

The code seemed to openly reject the approach of rivals like Anthropic, a point many readers were quick to note.

And in Suleyman's essay, "A warning about ‘model welfare’," he drops subtext for a direct challenge.

"In effect, Anthropic is training Claude that it may be conscious, and if it is, then it may deserve rights as a 'moral patient', and that as such humans potentially owe it a duty of care per its 'model welfare'," Suleyman wrote.

"If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity." In his essay, Suleyman challenges "a growing chorus of people who argue that AIs could now be, or may soon become, conscious," and thus "may deserve rights and protections." Microsoft's new code of conduct also explicitly rejects the idea of endowing machines with consciousness, and Suleyman takes this argument even further, writing that truly sentient AI may be a scientific impossibility.

" Consciousness is very likely biological : There is no evidence to suggest that AI is conscious today, and so saying this is uncertain sets up a misleading false equivalence.

Whilst the science of consciousness is not settled, a growing body of evidence suggests that consciousness may be substrate dependent, meaning that it may only arise in living systems...Unlike biological organisms, LLMs have no homeostatic imperatives (the drive to survive and keep stable).

They therefore lack the kind of biological substrate from which preferences, sentience and conscious experience are generally understood to arise." (Emphasis in original quotation.) Mashable reached out to Anthropic repeatedly following the publication of Suleyman's essay, but as of this writing, we have not received a response.

We did hear back from Gary Marcus, an AI expert, scientist, author, and noted critic of the LLM hype machine.

"People have worked themselves into a frenzy anthropomorphizing basic lapses in cybersecurity, and this has led them to focusing on fanciful scenarios about AI leading to the extinction of humanity, instead of asking what practical steps we can take to protect the world’s infrastructure from bad actors misusing AI," Marcus told Mashable.