'Whatever you try to fix breaks things even more': How to avoid the hidden dangers of AI code

You no longer need knowledge of advanced coding languages to build apps.

AI has torn down the barriers to coding, making it as simple as describing what you want in plain English.

Or at least, it can appear that way.

But coding isn't just about writing syntax.

It's about systems architecture, understanding how people will really use a platform, and ensuring the right measures are in place to limit vulnerabilities.

I spoke with Gabriela Moreira, CEO of Quint, about how people using AI tools to write code can better understand the associated risks and put in place early checks and processes to limit unmaintainable code and catastrophic bugs.

Can AI be trusted to write production-ready code without human oversight? If not, what makes AI-generated bugs distinct from human error? Oversight is definitely needed, because AI exists to serve humans, and therefore humans need to tell AI what to do and evaluate its results.

Now, which type of evaluation is the most adequate? You can read its code line by line or ship to production and monitor for problems, to talk about two extremes.

Both of these are already techniques used to oversee code that might contain human error.

The main differences that AI brings to the table are the amount of potentially wrong code that gets produced and the cognitive disconnection of any human to the elements of what gets produced.

Code review and monitoring are both harder at this load scale and cognitive disconnection.

Which core business systems are most vulnerable to AI coding flaws? The ones that are harder to test, like concurrent and distributed systems, or any system with a high level of non-determinism where there are too many paths to consider.

AI is a generation tool, which is only as good as our ability to decide if a generated solution is correct or not.

If I have good tests, or some other kind of verification harness I trust, I can accept AI solutions with less risk.