BUSINESS REFLECTION: Crossed Wires: AI flips the script on the economics of hacking and cybersecurity

A single operator can now direct AI hacking swarms to scan, exploit and ransom neglected devices worldwide, collapsing human processing limits and exposing decades of unpatched vulnerabilities.

Some decades ago, I was on a business trip overseas. I was the chief technology officer for a large tech company at the time, and we were visiting a well-known cybersecurity product developer that made encryption hardware for set-top boxes (remember those?).

As part of the tour that had been arranged for us, we were given a presentation by one of the executives, an iconic crypto scientist responsible for one of the most enduring security algorithms on the internet.

In his talk he described and demo’d some of the more edgy cybersecurity wizardry that was being researched at the company. I remember thinking wistfully – what a cool thing to do, thinking up ways to keep digital secrets, ways to hack those secrets and then ways to armour them further. A skill available only to the smartest of scientists who seemed to live in a parallel universe of maths, code and algorithms, often lurking in dangerous digital neighbourhoods populated by skilled bad guys trying to do bad things, and trying to stay a step ahead.

Sounded like something I would have loved to have done, had I had the talent.

Consider a hospital basement in which there is a building-management controller (you know – air conditioners, lighting, energy, alarm systems, access control) running firmware last updated before the iPhone was a thing. It has an ugly web interface and a default administrator password because nobody has bothered to change it since installation. For 15 years it has been sitting on the open internet, unmolested. It is not secure. It is merely unvisited.

Unvisited, obscure, unnoticed. The world’s digital ecosystems have long enjoyed this under-appreciated form of protection – there simply haven’t been enough good hackers to attack all of it.

The digital world is a largely undisturbed archaeological site spanning nearly 70 years. Of course we have our current cloud-native applications, zero-trust networks, passkeys and security teams running state-of-the-art detection systems.

But underneath them lie decades of accumulated detritus – ossified applications written in older languages such as C (and mediaeval languages such as Cobol), neglected and creaky middleware, forgotten APIs, routers, firewalls, switches, network-connected printers, industrial controllers, WordPress plug-ins, authentication servers, third-party libraries and innumerable bespoke programs written by someone who retired in 2004 and whose memory is fading.

The reason this ecosystem has survived and thrived is because finding an obscure vulnerability, understanding the architecture around it and turning the weakness into a useful and profitable attack requires rare human expertise. Unauthorised penetration of these systems has always been artisan work – labour-intensive, highly skilled, expensive and fundamentally constrained by human speed.

It has been a well-understood landscape. The attack surface is vast.