Cheap Per Token, Expensive Per Task: How Agent Workloads Invert Model Routing

Enterprise AI is shifting from one-shot prompts to long-horizon agent workloads, and the old model-selection logic is breaking in plain sight. A model that costs less per token can end up costing roughly twice as much per task once reasoning-step length and cache-hit rates are counted. Procurement teams still comparing per-token lists are mispricing the agent bill before it lands.

IBM Research's benchmark on the AppWorld Test Challenge makes the inversion specific. Across 417 tasks run through the same CodeAct agent, the lower-priced GPT-4.1 finished at $0.37 per task while the pricier Claude Sonnet 4.6 came in at $0.19. The cheaper model took fewer reasoning steps, but each step missed the cache and paid full input price. Sonnet took roughly three times as many steps and benefited disproportionately from low cache-read pricing on reused context. Token price lost. Reused context won.

Three forces drive the inversion, and none of them appear on a pricing sheet. First, cost is more than model pricing: cache economics dominate when the same prompt prefixes reappear across tool calls. Second, complexity is more than task difficulty: a request that looks simple can fan out into retrieval, compliance checks, and refinement rounds. Third, governance constraints, including data residency, approved model lists, and privacy rules, force reroutes that no router sees and the model cannot price.

The mechanism generalizes to every agent deployment. Audit the routing layer by counting cache hits and step counts per task, not tokens per prompt.