Nvidia joins the race to make AI cheaper to run

Model routing is software that looks at each incoming prompt, picks the AI model most likely to answer it well, and usually sends it to the cheapest one that can do the job. The point is to keep per-query costs down as teams mix cheap and expensive models behind a single interface. With Nvidia's new NeMo Switchyard library, Cloudflare's recently launched enterprise model router, and a reported Stripe bid for OpenRouter, that routing layer has become a market.

Switchyard is a library of routing approaches developers can mix into their own stacks, framed by Nvidia as a way to build "a system of models" rather than pick one. Cloudflare's router ships inside the AI-first enterprise suite the company has been building. The reported Stripe-OpenRouter deal, attributed to The Wall Street Journal by InfoWorld, would put a payments company on top of one of the most-used third-party routing services in the field.

What the vendors have not yet shown is how much routing actually saves at scale. The cost-savings numbers published so far come from the companies selling the routers, not from independent benchmarks. Whether the per-query savings hold up in production is the part the field has not yet settled.