Routers chase the cheap model. Caches already had it.

The AI cost-optimization race is splitting into two competing levers, and only one of them is structural. Vendors sell smart routers that pick the cheapest model for every prompt. The actual cheapest path is reusing the prompt prefix the cache already holds. When a router bounces a request to a different model, the cache hit evaporates, and the savings the router was supposed to deliver turn into a fresh full-price call.

Manifest's blog post on deprecating their LLM router makes the collision concrete. The cost of a cache read, they note, runs 75 to 90 percent below an uncached input. System prompts and conversation histories sit at the prefix of nearly every turn, which means the cached path is the default path. A router that ignores that and chases per-token price instead is, on net, an expensive way to save money.

A Hacker News counter-thread argues the problem is solvable, and that a small, clearly differentiated model pool with deliberate stickiness can still hit cache rates above 99 percent. The debate is real, and the falsifier is on the record. The general reader's takeaway is simpler: when a vendor pitches a router, ask how the router keeps your prefix warm. The model it picks matters less than whether the model it picked yesterday is the one answering today.