AINEWS
A new paper says you can shrink large AI models by roughly 3x with no measurable accuracy loss, and a small variance law in the methods section decides when such a claim is even mathematically possible.
A new research paper claims you can shrink a large AI model's weights, which determine its file size and the cost of running it, by roughly 3x without measurable accuracy loss. The interesting part isn't the number. It's a small theorem buried in the methods section that decides which compression claims of this kind can ever be real.
The paper, "Statistically-Lossless Quantization of Large Language Models" by Michael Helcig and colleagues at IST-DASLab, was posted to arXiv on 4 May 2026. It introduces a middle category between two familiar ways of compressing a model: the lossy methods (RTN, GPTQ, AWQ) that trade accuracy for size, and the exact lossless methods (ZipNN, entropy-coded FP8, dynamic-length BF16) that preserve every bit but offer modest savings. The authors call their new category statistically-lossless: the model's outputs deviate from a full-precision run, but stay within the noise that the original model already shows under standard sampling.
Quantization, the underlying trick, is the practice of representing each number in a model's weights with fewer bits (8 instead of 16, or 4 instead of 8) so the file shrinks and the arithmetic gets cheaper. Earlier lossless methods like ZipNN save roughly 31% — compressing the same baseline from 16 bits per parameter to about 11. Lossy methods at 4 bits per weight can drop zero-shot accuracy on hard tasks by 3-5%, according to prior benchmarks cited in the paper. The IST-DASLab team reports task-level equivalence at as low as 3.3 bits per parameter and distribution-level equivalence at 5-6 bits per parameter on average, with inference speedups of 1.7-3.6x over FP16 using optimized kernels. The SLQ repository lists those claims alongside a single line: "Code coming soon" — a gap the paper's own abstract does not reflect, as the abstract states source code is available at the same URL.
The math is the part that doesn't move while everyone waits. The paper proves a gamma-squared variance law for symmetric quantization: noise variance scales as gamma-squared relative to an asymmetric quantizer at the same bitwidth. For distributions that aren't centered on zero, and most LLM weight distributions aren't, an asymmetric quantizer introduces gamma-squared times less noise variance than a symmetric one. Asymmetric quantization is therefore a prerequisite for distribution-level equivalence. A symmetric scheme at 4 bits can match task behavior; it cannot match output distribution without a much higher bit budget. The theorem sorts the field: any future "lossless" claim has to answer which kind of losslessness it means and which kind of quantizer it used.
To make the equivalence check legible, the authors propose a metric called Expected Acceptance Rate (EAR): the fraction of next-token samples that agree with the full-precision model. An EAR of 0.99 means 99% of the time, the compressed model picks the same next token. That's a directly interpretable number in a field that usually hides behind perplexity deltas.
The falsifier is real. The math permits task-level equivalence at 3.3 bits per parameter, but it says nothing about the workloads where the field has watched quantization claims fail before: long-context retrieval, multilingual generation, multi-step reasoning. Kurtic and colleagues documented 3-5% accuracy drops at 4 bits on complex tasks; the IST-DASLab team measures zero-shot accuracy on the benchmarks they selected, not on production traces. Until the official SLQ implementation ships and an independent group runs it against long-context and reasoning suites, the headline claim is author-reported, not field-confirmed.
The discovery path is a self-post on r/MachineLearning; the [R] tag conventionally marks the author cross-posting their own paper. The Reddit thread points to the arXiv preprint, not the other way around. The preprint is the source of record; the thread is just the dust cloud around it.
The next milestone is concrete. Helcig and colleagues say the implementation will land in the SLQ repository "shortly," with no public date. Mainstream inference stacks (vLLM, llama.cpp) do not yet ship SLQ kernels. Until they do, the 1.7-3.6x speedup is a paper number, and the only part you can evaluate today is whether the claim obeys the variance law the paper derives. It does, on the authors' own framing.