Call it the data-layer tax

The cheapest AI bill a company has ever paid is a pilot. Every enterprise that ran a generative-AI proof of concept on a narrow dataset is about to see what production does to that number, and IDC now projects the world's largest 1,000 companies will run 30% over their AI infrastructure budgets by 2027. The overrun is not what procurement priced. It is the cost the pilot could not model.

Call it the data-layer tax. Tokens got cheap; models got better; the line item that did not fall is the one nobody prices until production: the per-request lookups, cache and session-state reads, and the dozens of services a single inference has to touch inside a sub-100ms latency budget, where the slowest lookup gates the rest. Pilots see one dataset with one user. Production runs many reads in parallel, and the agentic workflow — a request broken into a plan and many small steps — turns one user turn into tens or hundreds of data accesses. The model is now a small slice of the bill.

The Aerospike CINO's diagnosis in TechRadar Pro puts the same point in one line: "The cost lives in the data layer, driven by how often the system reads, how many services it touches, and how continuously those operations run." Right frame, attributed honestly: a vendor executive, not an independent study, and an analyst forecast, not a measured outcome.

Defensive over-provisioning — duplicating data, layering services to mask tail latency — is a choice, not a law of physics. The gap between pilot and production is knowable. The next planning cycle is the one that pays for closing it.