What Happens When the World is Run on Code No One Understands?

—lucadp—Getty Images The International Congress of Mathematicians met in Philadelphia last month, its first U.S. meeting since 1986.

Minutes after accepting the Fields Medal, mathematics’s highest honor, Jacob Tsimerman told reporters he was leaving academia to work on AI safety.

“I think the world is changing,” he said, adding that he did not expect the mathematical career to survive in its present form.

Two months earlier, he had been one of nine mathematicians asked to review whether an AI model had truly refuted an 80-year-old central pillar of modern geometry, known as the Erdős unit distance conjecture.

Tsimerman and his peers determined that it had, and their commentary alongside the 18-page proof helped translate the machine's argument into mathematics a human could understand.

The breakthrough, and subsequent report, reflected how the bottleneck that holds us back from forging new discoveries in mathematics, and, increasingly, in every other field, is changing.

For most of history, the scarce resource was discovery.

With AI, discovery is nonstop, and human confirmation is now what is scarce.

To be clear, we are AI optimists.

We believe AI tools will complement human ingenuity and expand what we can know and build, but our infrastructure for vetting and certifying discoveries was built for human throughput, and that is now the binding constraint.

That is what holds innovation back.

The answer to this problem is formalization: translating AI’s outputs into precise forms whose correctness can be checked automatically.

Building the infrastructure to make verification routine is now a national-scale engineering problem.

Machines are outrunning us in more than math.