In a synthetic records preprint, open source AI matched a proprietary AI pipeline at grading type 2 diabetes severity and surfaced mortality risk fixed rules missed.
A team of free, locally run AI models read synthetic diabetes patient records, graded how severe each case was, and produced severity tiers that lined up with who went on to die, even where fixed-rule clinical scores had not.
A preprint on arXiv, summarized on TLDR Takara, reports that MOSAIC, a multi-LLM framework, graded diabetes severity on synthetic patient records by reasoning across dimensions fixed-rule scales were never written to see.
MOSAIC (Multi-Large Language Model Orchestrated Severity Assessment of Clinical Records) is a two-phase agentic framework. In phase one, several large language models examine a patient's digital medical record and propose severity indicators. In phase two, those candidates are filtered, structured, and combined into a final severity tier. The researchers tested the framework against three established severity scales: DCSI, DiSSCo, and Cooper. The cohort was SyntheticMass, a synthetic dataset generated by the open-source Synthea tool, with 4,886 patients in the open-weight arm and 200 in the closed-weight arm.
The team-of-models MOSAIC scored a weighted kappa of 0.534 against DCSI, 0.320 against DiSSCo, and 0.597 against Cooper. Cooper is the closest comparator in routine clinical use. A frozen version of MOSAIC, in which the orchestration step was replaced with deterministic execution, fell to a kappa of 0.428. The 0.428 figure matters because the frozen version strips the orchestration step. The gap between Cooper (0.597) and Frozen MOSAIC (0.428) is the mechanism claim: the reasoning layer, not just the underlying models, does the lifting.
Open-weight MOSAIC matched a proprietary version of the same pipeline at a weighted kappa of 0.773, according to the preprint. Freely available models, run locally without sending patient data to a third party, approached the closed-weight version on a clinical task. The proprietary comparator was the same research group's pipeline rather than a marketed clinical product, so the parity is research parity, not commercial parity.
MOSAIC seemed to add value on outcomes the fixed scales were never designed to predict. The team validated the framework's severity tiers against all-cause mortality and incident complications in the synthetic cohort. Tier separation was significant but non-monotonic at the upper end; the preprint PDF frames the inverse gradient at the highest tiers as depletion of susceptibles rather than a model failure. The generated severity framework also spans domains the comparators do not: biomarker-based glycaemic staging, beta-cell function, and social determinants of health. The established scales either oversimplify or skip those angles.
The DCSI, DiSSCo, and Cooper scales were built to be reproducible, not exhaustive. They score diagnoses and medications, then assign a fixed severity bucket. MOSAIC, by contrast, follows a thread across a record: a borderline A1c, a worsening estimated glomerular filtration rate, and a missing social determinant. Each cue is familiar on its own. The paper's point is that an orchestrator can read them together in combinations the rubrics were not designed to capture.
The cohort is synthetic. SyntheticMass is a public dataset generated by Synthea at MITRE, useful for benchmarking pipelines but unable to substitute for real-patient validation. Agreement with the established scales is moderate to fair, not strong. The proprietary comparator was an in-house closed-weight version of the same framework, not a deployed clinical product. The paper is a preprint, not peer-reviewed.
The result supports a specific next step: open-weight tools approaching proprietary parity lower the cost and lock-in of clinical phenotyping for groups that cannot or will not send patient data to a closed API. A reasoning layer that finds risk fixed scales miss is a research result on its own merits. The team behind MOSAIC frames it that way explicitly.