Satya Nadella says a company's relationship with AI should have trust issues
Microsoft CEO Satya Nadella says companies shouldn't trust AI models.
Bloomberg/Getty Images Microsoft CEO Satya Nadella said tech companies should treat AI models as an insider threat.
He said companies should assume their AI models are "compromised" and contain them from the start.
Tech leaders are grappling with how to contain and govern frontier AI models.
Satya Nadella says that in the relationship between AI and a company, trust issues can be healthy.
In a lengthy X post on Saturday, the Microsoft CEO said companies don't always understand an AI model's decisions or behavior, so they need to build walls to protect themselves from the tech.
After all, it's very possible that AI is already exploring new connections.
"We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient," Nadella said.
"Treating frontier closed and open weight models like insider risks is a way to build such a system." Nadella said that AI models aren't inherently malicious, but anything with access to vital systems can become compromised or make mistakes.
He added that AI models shouldn't be able to operate the controls that determine what they can access or what actions they can take.
"Today this means separating the model from the harness that orchestrates its work, as well as the action space that defines what it can do.
It also means externalizing controls and safeguards," Nadella said.
He added, "The most trustworthy Super Intelligence system will not be the one with the model we trust most.
It will be the one that enables us to trust the model the least." Box CEO Aaron Levie responded to Nadella's post and said AI will need to go through a "zero trust era." "And all of this leads to needing various layers of protection and auditability of what agents are doing, what data they can work with, and controls for when things go wrong.
5News aggregated this summary from the outlet’s public feed. The full article, with all the context, is on www.businessinsider.com — the content belongs to Business Insider.
