Using synthetic data for AI training is 'a big mistake,' says AI pioneer Rich Sutton

Richard Sutton said synthetic data is not the solution for scaling AI.

Business Wire/AP Rich Sutton criticized tech's reliance on synthetic data, urging real-world experiential learning.

Tech giants like Google and OpenAI are going out of their way for real-world data.

Sutton's Oak Lab focuses on teaching AI agents from experiences, not made-up datasets.

Rich Sutton helped pioneer the technology behind today's AI boom.

Now he thinks Big Tech's push to keep AI going is all wrong.

On an episode of Sequoia's podcast released Tuesday, the Canadian computer scientist and Turing Award winner said that the AI industry is headed in the wrong direction because of its reliance on synthetic training data.

"That's just a big mistake," he said when asked about using synthetic data as a way to keep scaling large language models.

"Maybe it's the next big lesson." Synthetic data is information artificially produced by algorithms or AI models rather than collected from original, real-world sources .

It has become increasingly attractive as AI companies scour the internet and hunt for new sources of training data.

Examples include computer-generated images of cars for self-driving training or fake bank records for AI fraud detection.

On some level, Big Tech agrees with Sutton.

Tech giants are going to increasingly extraordinary lengths to find more of the real thing.

OpenAI has publicly sought large-scale, proprietary datasets that aren't readily available online to help train its AI models.