AINEWS
A new public dataset and workload generator from a global multi model AI marketplace show that request timing and output length vary by model architecture, scale, and task.
A research team has released FineServe, a public dataset and workload generator built from real traffic on a global commercial multi-model LLM serving marketplace. Their central finding: serving 55 language models at once does not mean serving one workload many times. Traffic across those models and 10 task categories is not one workload scaled up. It is several, with different arrival curves and different token behavior.
"LLM serving" is the production plumbing that runs models in response to user requests, not the models themselves. A "multi-model platform" hosts many different models side by side so a user can pick among them. The marketplace behind FineServe is not named, which constrains how far the dataset's representativeness can be claimed. The paper describes it as a "global commercial" platform and stops there.
The authors' argument, laid out in the preprint's abstract, is that prior LLM serving research has leaned on synthetic traces or coarse characterizations that miss the heterogeneity of modern traffic. Their traces are fine-grained. They capture arrival dynamics (when requests land and how they bunch) and token behavior (how much output the system has to generate) at a per-request level. The headline claim is that "different fluctuation regimes" appear across model architectures, scales, and task intents. A small chat model and a large coding model do not just produce different token counts; they produce different arrival patterns, and a scheduler that treats both the same wastes capacity in one place and starves it in another.
The release has two halves. The dataset, desensitized so customer identity and proprietary model names are stripped, gives researchers a real workload to study. The companion workload generator has two modes. Replay drives simulations directly off the captured traces, so a team can test a new scheduler against the same Tuesday-night traffic a real platform saw. Parametric fits per-window statistical distributions (gamma and negative-binomial) so the generator can synthesize traffic at arbitrary scale beyond the original capture window.
That tooling lands in a part of the stack that has been starved for public data. Routing decisions (which model gets a given request), scheduling (how a GPU cluster interleaves them), and capacity planning (how many accelerators to provision) all depend on a workload model, and most published models have been synthetic. FineServe gives the field a shared, real-world reference. The intended use is benchmarking, not production replay: the point is that a routing algorithm can now be evaluated against traces that resemble a real marketplace's Tuesday rather than a hand-picked distribution.
The paper is a preprint, not peer-reviewed, and the marketplace is unnamed, so the dataset's representativeness is open. The platform is described as "global" and "commercial", but the traffic mix behind those words is one marketplace's customer base, which may not match another's. Absolute request volumes, customer identity, and proprietary model identifiers are not public, which constrains impact and adoption claims. The 55-model and 10-task numbers come from the model taxonomy and task taxonomy in the GitHub repo, not the abstract, and should be read with the paper.
The traces ship with the repository. The generator ships with the code. The next test is whether published schedulers, run against the Replay traces, hold up under traffic that does not look like a single distribution.