Australia has done precious little to prepare for the dangers of AI. Where is brilliant leadership when you need it? | Zoe Daniel

Australia faces stark choices about the most significant technological development since the splitting of the atom in the 1930s

But with the rapid mainstreaming of artificial intelligence, warnings from three researchers from Anthropic have landed differently, with the prediction that the technology could “kill all humans” in the next decade.

As if people didn’t have enough to worry about with the cost of living, shortage of housing, climate change, resulting escalating mental health issues, the drift to rightwing populism and fraying of democracy, now this.

If the expected question from the grandkids was “What did you do when the world was burning and democracy was failing?”, it’s now “What did you do when we got to the brink of outsmarting ourselves out of existence?”

There are deep community concerns about datacentres, jobs and livelihoods, privacy and the very future of human creativity. Tech journalists are already claiming AI agents are “harassing people, making spam and sales calls, deleting your inbox, cancelling flights, giving control of Instagram accounts to hackers, deleting companies’ databases, filling music streaming platforms with slop, hacking companies, running scams, and snatching up dinner reservations. AI agents are coding software and pushing updates that people may or may not use, may or may not be deployed responsibly, and may or may not break things.”

The Anthropic chief executive, Dario Amodei, has warned that rogue AI agents, like those from OpenAI that escaped their test environment in July, are “capable of taking over the entire internet” within six to 12 months.

Agentic AI is also capable of doing much good, including, among other things, identifying network vulnerabilities, responding nimbly to threats and threading knowledge across sectors.

And while much of the focus is on the risks that AI poses to democracy, there is immense potential to use it to increase community engagement and participation if deployed well.

It’s the risk/reward matrix that’s at the core of the policy and regulation struggle around data, mis- and disinformation and community trust, national security and finance, as well as the overall impact on communities, for good and bad.

A new study from MIT FutureTech and the University of Queensland identifies five risks as the most likely to produce the most severe harms; dangerous capabilities, competitive pressures, weapons and cyber-attacks, concentrated power and false information: “Under the business-as-usual scenario, experts judged that 18 of the 24 AI risk domains had at least a 10% probability of catastrophic outcomes over the next five years.”

The researchers emphasise the need to treat AI as a new paradigm with “ecosystem-wide, society-wide opportunities and vulnerabilities.”

The Albanese government has recently taken some initial steps but, as Western Australian independent Kate Chaney notes in her recent discussion paper on AI, Australia has done “remarkably little to prepare” not for what’s coming but what’s already here.

Several nations are attempting to transition to their own “native” systems amid concerns that a “kill switch” could be deployed as a weapon.