Waymo has designed a robocar chip to stay ahead of Tesla

To reach their destinations safely, autonomous vehicles have just milliseconds to ingest and process streaming data from more than a dozen cameras.

It's a job that's been handled with off-the-shelf AI components thus far, but Waymo has begun rolling its own AI ASICs to optimize the process.

It's not alone.

Revealed in a blog post Thursday, the Alphabet-backed robo-taxi startup's first custom silicon is designed to convert raw sensor data into driver responses as quickly as possible.

Built on Taiwanese foundry giant TSMC’s 5 nm process tech, the chip is specifically optimized to run both more traditional machine learning algorithms like convolutional neural networks and modern transformer models similar to those used to run AI chatbots or image generation models.

According to Waymo, the chip's design incorporates more than 200 million miles worth of autonomous driving data, and is tuned to maximize responsiveness, reliability, and redundancy.

Prior to this, Waymo had employed Intel FPGAs for sensor processing.

FPGAs are ideal in low latency applications, which is one of the reasons why high frequency trading often takes place on them.

However, compared to dedicated silicon, FPGAs are notoriously difficult to program for and lack the compute density achievable using application specific hardware.

Accidents can unfold in a fraction of a second, far too quickly for a remote operator to take over.

So Waymo designed the chip with a major focus on minimizing latency.

“Within those critical milliseconds, advanced ML models build a high-fidelity understanding of the environment to evaluate the safest path forward,” the company explained.

This includes performing temporal noise reduction to improve low light visibility in real time.

All of that requires a considerable amount of computation.