CurveFP Claims a 7-Bit Format Can Match FP8 on LLM Inference. The Paper Stands Alone.

A new arxiv preprint called CurveFP makes a sharper claim than "one bit fewer": it says a 7-bit numeric format can match the perplexity (a standard measure of how surprised a language model is by held-out text; lower is better) of FP8, the 8-bit floating-point format that now powers much of the math in modern AI accelerators, on small and mid-sized language models. The paper, CurveFP: Rational-Radix Logarithmic Datatypes with Closed Products for Language Models, has not been peer reviewed, reports no silicon or wall-clock cost, and has not been replicated outside the authors' own evaluation harness. The headline figure, a 1.32% gap to native BF16 quality, is a mean across three seeds on a 128.3-million-parameter model. Read as a mechanism, the result is interesting. Read as a roadmap, it is a single unreplicated paper.

Low-precision number formats sit underneath every modern LLM. FP8 packs each weight and activation into 8 bits (one sign bit, a small exponent, and a few mantissa bits), letting accelerators move more numbers per second and fit more of them into memory. Less precision usually means more error per multiply, so format designers spend years threading the needle between bit count and numerical fidelity. Sub-8-bit proposals like NVFP4, OCP microscaling, and now CurveFP are part of the same arms race.

CurveFP's bet is that the shape of the multiplication matters as much as the bit count. The format distributes quantized magnitudes across interleaved logarithmic curves under compact block scales, with a rational radix tuning dynamic range against local resolution. The product operation is reduced to an exact sign XOR and an integer-index update, and a derived finite phase count sets the accumulation schedule. In plain language, instead of approximating a real-number product with a floating-point multiply, the format guarantees that any product of two nonzero encoded values lands on a value the format can represent exactly. The multiplication is closed, the way integer multiplication is closed, except the encoded set is logarithmic rather than linear.

The instantiation comes in two flavors. CurveFP-eight (E4C3/E5C2) is the training format, pairing 4-bit and 5-bit exponents with 3-bit and 2-bit mantissas across interleaved curves. CurveFP-seven (E3C3) drops to 7 bits per element and targets compact deployment. In the authors' benchmark suite, CurveFP-seven beats tensor-wise FP8 perplexity on four 7B-9B models with one fewer element bit and stays within 1.32% of native BF16 quality on the 128.3M triplet. CurveFP-eight lowers operand normalized mean-squared error (NMSE, lower is better) in all 36 paired forward and backward GEMM (general matrix multiplication, the dominant math op in LLM training and inference) comparisons. Mean BF16-inference perplexity is 22.5366 for CurveFP-eight versus 22.5407 for FP8 across the same triplet, with a lower format-induced penalty in every seed.

The paper reports no silicon cost, no energy per token, no wall-clock training time, and no head-to-head against NVFP4 or OCP microscaling (MX) baselines. All perplexity and NMSE comparisons live inside the authors' own evaluation harness. The four 7B-9B models used in the headline comparison are not named in the abstract, and the 1.32% native-quality figure is averaged across three seeds on a 128M model. None of the 36 GEMM pairs, 36 downstream cells, or 12 seed-format perplexity comparisons has been independently reproduced. The "closed product" property is a math guarantee; the "cheap in silicon" claim is not.

CurveFP's specific contribution is that the multiplication itself lands on a representable value, so the format's error budget is spent on accumulation rather than on rounding each product. If a hardware implementation can exploit that property without paying for it in area or latency, the lever is the closed-product path, not the bit count. If a hardware implementation cannot, the format becomes a curiosity that lives in software emulators. The paper does not say which.

What to watch is straightforward: independent reproduction on a named 7B-9B model family, a silicon or RTL implementation reporting energy and throughput versus FP8, and any head-to-head against NVFP4, OCP microscaling, or block-scaled FP8 on the same model and tokenizer. Until one of those lands, CurveFP is a research result with a clean mechanism and an open question. The format is worth filing before the next sub-8-bit proposal arrives.