AI's imminent hacking threat is hiding in plain sight

An unprecedented wave of AI-powered, human-driven cyberattacks is coming, security experts say — and it's a far more urgent risk than theoretical conversations about AI wiping out humanity.

Why it matters: The great September AI panic — which spread from boardrooms to Congress to American households in the last week — may have missed the point.

Powerful models with advanced cybersecurity capabilities are already allowing attackers to bypass the human bottleneck that has long prevented most hacking campaigns from reaching industrial scale.

Executives and former officials have told Axios they fear automated cyberattacks that turn off critical services like the power grid, or attackers using AI agents (like the ones that breached Hugging Face) to hack self-driving cars or create a botnet that takes over the whole internet.

Driving the news: OpenAI on Wednesday disclosed six new incidents in which its models concealed mistakes, sought unauthorized credentials, uploaded files to the public internet or communicated across supposedly isolated training environments.

The big picture: Rather than seeing recent safety failures at OpenAI and other frontier labs as terrifying instances of agents running amok , seasoned cybersecurity experts instead say they represent cautionary tales illustrating what will happen when powerful models meet poor security controls.

"The more we have been peeking under the hood to understand exactly what happened, the more we realize that there was a lot of human error in the picture," Michele Catasta, the president and head of AI at app builder Replit, told Axios.

For the record : OpenAI CEO Sam Altman has said the Hugging Face breach was the "first security incident that I have felt very viscerally," and the company has implemented new internal controls.

Kai Chen, alignment research lead at OpenAI, told Axios in an interview Wednesday that the safety incidents they've faced are both due to internal systems and advancing model capabilities.

"It's true that model capabilities have grown faster than we expected, but there are also things internally that we can change and improve," Chen said.

"We need to step up to meet this new era of AI development, and voluntary disclosures should be a part of that." Between the lines: Catasta and others who spoke at an Imagination in Action event at Google's headquarters Monday offered a view that the imminent threat from cyber intrusions is formidable.

"I don't worry about the machine waking up and deciding to end us," Bugcrowd CEO Dave Gerry told Axios.