A 4-billion-parameter open model just hit 87% on Sweden's medical licensing exam — without medical training

A 4-billion-parameter open-weight model hit 87% on Sweden's medical licensing exam — roughly one point behind o3's 2025 score on an overlapping but smaller slice — without medical-specific training. The lever was switching on step-by-step thinking at run time, not adding data.

The benchmark is MedQA-SWE, a Swedish medical licensing multiple-choice exam of about 140 questions with a 60% pass threshold. Swedish ML practitioner Taro Langner reports that Qwen3.5-4B reaches 87% with reasoning enabled, while Gemma4-E4B and Qwen3.5-4B both score 77% out of the box. o3 reportedly scored 88% in 2025 on a smaller, overlapping slice; GPT-4 hit 84% in 2024.

The mechanism is the story: the model reads Swedish questions and answer choices, then reasons in English — a small but surprising data point against the assumption that prompt language constrains reasoning. An "early exit" intervention from the S-GRPO preprint is used to cap looping traces.

The source itself flags two caveats: this is a single practitioner's self-reported run, and exam accuracy is not clinical competence. MedGemma-1.5-4B fine-tuned on prior-year exam data jumped from 14.6% to 60.8% — partly an answer-extraction formatting fix, not pure medical-knowledge gain. The implementation is open source and the write-up is on Tensorlabbet.