MINENHLE NZIMANDE | Judges under AI scrutiny

A few years ago it would have been unimaginable to suggest AI could sit anywhere near a judge’s bench. This is because we were told that judges at times exercise their discretion of human understanding, which is what a machine cannot do.

Today, that reality is uncertain. AI tools now routinely assist lawyers and judges in drafting heads of argument, summarising case law, and even producing first drafts of judgments. Technology has moved faster than the judiciary’s guardrails.

Nowhere was that gap demonstrated more clearly than in a recent Joburg high court matter where an acting judge has been referred to the Legal Practice Council (LPC) due to a judgment containing what appeared to be AI “hallucination”.

The case itself was a custody and schooling dispute between separated parents. On appeal, judges Gregory Wright, Daphny Mahosi and Ingrid Opperman upheld acting judge Themba Khaba’s judgment. However, in a separate ruling, Opperman held that at least 11 citations in Khaba’s judgment were fictitious or materially inaccurate.

For example, a case with the right name but the wrong citation; a correct reference attached to the wrong quoted passage; and most tellingly, a nonexistent case, Lubbe v Volkswagen SA. Opperman said the pattern was consistent with AI-generated content, though she did not make a finding on whether AI was in fact used, insisting the acting judge must be given the chance to explain himself.

She referred the matter to the LPC, noting a troubling accountability gap: that acting judges fall outside the Judicial Service Commission’s disciplinary reach, leaving the LPC as the only forum capable of investigating an acting judge for possible misconduct.

Writing a judgment is a sensitive matter, where there should be little room for mistakes. This is not to say that judges do not make mistakes. However, a judgment is a high-stakes act that alters the rights and freedoms of people. That places it in the category of decisions demanding “human-in-the-loop” oversight, a decision-maker who actively reviews and can override the output before it takes effect, not a system where AI-generated material is monitored after the fact or trusted on the strength of its apparent fluency.

What appears to have failed here is precisely that active, individualised verification. Citing a case by name is different from reading it. Quoting a passage in a case is different from confirming it exists in the judgment it is attributed to.

If AI were used to draft or research any part of this judgment, the missing safeguard was not the tool; it was the judge’s own act of checking, the discipline UK chancellor of the high court Lord Justice Colin Birss described when he said he used ChatGPT only as a research tool, subject always to his own verification. That single distinction between assistance and abdication is the difference between ethical use of AI and unethical or reckless use of AI tools.

Some may argue that referring this conduct to the LPC is an overreaction because the final order was correct and no litigant was prejudiced. However, that view misses what a judgment actually is.