AINEWS
A 1.5 billion parameter model gained two points on math benchmarks by training with a second model that critiques its answers, at sixteen times lower tuning cost.
A 1.5-billion-parameter AI model learned to solve multi-step math problems about two percentage points more accurately, not by studying more worked examples, but by training alongside a second small model that was paid to outdo the first one's own best answers.
The result, in an arXiv preprint posted 20 July 2026 and now under review at the journal TMLR, is a method story rather than a leaderboard topple. The paper's central trick is a rewrite of the grade card used to train a "critic," a second small model whose job is to critique answers from a separate "generator" model. The critic's score is computed not against a static "match the right answer" target, but against the generator's own per-question accuracy. That gap, the difference between what the critic produced and what the generator was already producing, becomes the reward signal used to update the critic's weights.
The headline number is real and statistically significant. On five math-reasoning benchmarks, the approach lifted DeepSeek-R1-Distill-Qwen-1.5B from 39.9% to 41.9% accuracy (p<0.001), a +2.0 point gain. The tuning was done with LoRA adapters, a parameter-efficient training technique that updates only a small slice of a model's weights, so the method used roughly 16 times fewer trainable parameters than the fully fine-tuned baselines it was compared against.
The paper does not pretend the approach crowns the field. Two stronger baselines, DeepScaleR and STILL-3, still beat it on absolute accuracy; both were trained on substantially larger datasets. The author, Martino M. L. Pulici, analyses the gap directly and treats it as a data-scale question, not a method failure. That honesty is part of why the result is worth reading.
In the controlled ablation, the author isolates the new reward signal: the critic's score minus the generator's own per-instance accuracy, what the paper calls a "counterfactual critic advantage." With that change, the critic improves faster than any model in the comparison. The critic learns to spot generator errors rather than to imitate correct answers, the opposite failure mode of static reward designs. The smarter grading curve is doing real work, not just adding noise to a stronger learner.
Labs working on on-device reasoning assistants, where every extra training parameter and every extra benchmark point matters, can revisit the reward function before reaching for more data. The paper's own recipe is small: pair a generator with a critic, score the critic against the generator's own consensus, and tune with LoRA. The next test is whether the same trick travels to non-math reasoning tasks and to a model substantially larger than 1.5 billion parameters.
An aggregator summary of the same preprint adds no separate facts. The original signal is the arXiv listing, 20 pages, 3 figures, 9 tables, 2 algorithms, version 1 submitted 20 July 2026.