'A line has to be drawn': The trade-off we're making without really noticing

Artificial intelligence is increasingly used by police departments across Australia, with experts warning that lines must be drawn over the technology's use, invoking the dystopian world depicted in George Orwell's novel Nineteen Eighty-Four as a potential future to avoid.

Police departments are starting to trial the use of AI technology for "various forms of surveillance" and "predictive policing", professor Toby Walsh, chief scientist at the UNSW AI Institute, told SBS News.

In WA, police have been trialling AI facial recognition software to scan pedestrians on the street and compare their faces against watchlists. In the trial's first week, 130,000 people were scanned, leading to 18 arrests.

Walsh said that while most people would be "pleased" with the result, he believes such public surveillance is "an erosion of the things that we used to value".

"The question is, how much are we willing to trade those values for the benefits that the technology is bringing?"

Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.

Overseas, the rise of AI in surveillance technology has become an increasingly contested topic.

In the United States, citizens have been taking action against the surveillance cameras of the AI company Flock Safety, which has tens of thousands of devices across the country. Videos of people destroying the cameras have gone viral on social media.

Tom Sulston, head of policy at the advocacy group Digital Rights Watch, told SBS News that, while Flock cameras are not currently used outside of the US, "plenty" of companies in Australia are doing "similar things".

Palantir, the US data analytics firm contracted by military and US immigration forces, and Cellebrite, an Israeli digital forensics company working with law enforcement, both have tens of millions of dollars in government and private contracts in Australia.

"We should be concerned," Sulston said. "We have a right to go out in public without being surveilled, without having our biometric data uploaded to the cloud, and without it being used for purposes we don't really know."

Both have said that governments across the country need to rapidly turn their focus to expanding individual privacy laws, particularly as wearable surveillance technology like smart glasses become more widespread.

"We do have some legislation that covers public surveillance, but the more companies that get into that market, the more pressure there will be to relax them," Sulston said.

Walsh argues that a line between privacy and security "has to be drawn" as AI has made previously impossible levels of surveillance mundane.