Nvidia wants to stop AI costs skyrocketing with its new software router — but will it really make a difference?

Nvidia's open source NeMo Switchyard 'smartly' routes each agent request to the cheapest model that can handle it The approach allows it to claim a 74% cost cut against a frontier-only model in return for a 6% reduction in accuracy NeMo Switchyard joins an increasingly well-established territory that already sees players such as RouteLLM, LiteLLM, and OpenRouter, in addition to in-house efforts at OpenAI and AT&T to save costs Nvidia has released NeMo Switchyard, an open source model router that sits between an application and a pool of language models and decides, per request or per turn, which model should handle a particular task.

The aim is to maximize efficiency by picking the right model for the right task: pushing frontier-level models to do simple tasks that much smaller or cheaper models could handle is not only inefficient, but also costly for enterprise customers.

This comes at a time when enterprise customers are already grappling with rising costs that can quickly spiral out of control as they increasingly adopt AI.

An open-source addition to a growing chorus of AI routers The problem Nvidia is trying to solve isn't new, and industry giants are already attracting a lot of attention.

The emerging field itself is potentially lucrative, even if it focuses on cutting costs, and major players are looking to get in on the action; Stripe's upcoming $7+ billion acquisition of OpenRouter , underscores this.

Switchyard is no different from its peers; it is essentially a proxy that uses a routing algorithm to decide which AI model to send queries to, while delivering an answer in the format that the calling application or user requires.

It accepts OpenAI, Anthropic, and Responses API requests, translates between them, and documents the selected model, decision rationale, token usage, and latency for each call, enabling transparency and letting it tweak or tune its approach over time.

This makes sense compared with a one-size-fits-all approach that would otherwise be prevalent in a world with limited frontier-level AI compute, as LLMs continue to grow larger and more demanding.

One example is what Nvidia points to: Nemotron Parse, a one-billion-parameter model primarily designed to extract structure from PDFs.

The question remains, however, whether such an approach is always feasible.

Leveraging such a routing tool comes at a cost; LangChain's test of Nemotron 3.5 Lightning found that its judge model, which determines whether the agent is still on track after each turn, consumed up to 21.2% of total cost, second only to what it spent on Claude's Opus 4.8.