AI scored 91% on lung tumor response. The 9% of cases it missed skewed toward under-called responders.

A clinical AI that scores how much viable tumor is left in a lung cancer slide after chemo-immunotherapy matched expert pathologists 91% of the time across 135 patients in four countries. The remaining 9% did not split evenly. The model under-called response three times as often as it over-called it, telling a patient they had not responded when the pathologist judged they had.

Under-called responders carry different downstream consequences than over-called ones in a setting where the post-treatment verdict shapes adjuvant therapy, post-operative decisions, and trial eligibility.

The Medscape summary describes a validation that ran a weighted-average residual viable tumor score across 1,344 H&E-stained slides from 135 patients in four cohorts. Pooled accuracy was 91% (95% CI 85%–95%), sensitivity 90% (95% CI 82%–95%), specificity 93% (95% CI 81%–99%), and the area under the curve sat at or above 0.95 across all four cohorts: 38 patients from an internal Spanish site plus three external cohorts from China, Germany, and Spain.

Pathologists agreed with each other at an unweighted intraclass correlation of 0.95 and a weighted ICC of 0.96, meaning the AI's 91% figure was measured against experts who almost always agree with each other. A model that beats that expert baseline is doing real work; a model that merely matches it is doing triage.

The 12 discordant cases, 8.9% of the 135 patients, broke 9 to 3 in favor of false negatives. The team traces the skew to stromal underrepresentation, the model's tendency to under-count the connective and non-tumor tissue that pathologists treat as part of the total tissue area when computing residual viable tumor. The undercount shrinks the AI's denominator, which raises its residual tumor percentage and pushes borderline cases into the non-responder column when the pathologist would not.

That direction is the one that affects adjuvant treatment, where a non-responder verdict can mean a patient who actually responded to chemo-immunotherapy is steered toward additional post-operative therapy or is screened out of a response-gated trial.

A separate 47-patient H&E WSI study on PMC takes a related approach, using a patch-based AI RVT model benchmarked against a board-certified pathologist for the same clinical question, the post-treatment verdict in resected NSCLC slides. A third model, NeoPred, published in the Journal for ImmunoTherapy of Cancer last year, attacks the question from a different angle: pre-treatment and pre-surgical dual-phase CT scans in 509 retrospective and 50 prospective patients. Putting the two adjacent studies next to the 135-patient one is the easiest way to see that "AI for lung cancer response" is not one technology. Slide-based %RVT scores what the surgeon actually removed from the resected specimen. CT-based prediction estimates who is likely to respond before the knife goes in. A reader who collapses them into a single "AI matches pathologists" claim is reading past the actual clinical question.

Three scope limits should travel with the 91% number. The cohort is retrospective, drawn from four sites with different scanners and slide-preparation protocols. The three external cohorts were each read by a single pathologist, so the expert ground truth against which the AI was scored is not a committee consensus. And major pathologic response is a research benchmark, not a survival outcome, so the validation does not yet show that AI-scored response predicts who lives longer.

A model that holds up across 1,344 slides in four countries and runs within the inter-pathologist agreement range is a real validation step. A model whose misses go consistently one way is a case for routing the disagreement to a human reviewer, not a case for replacing the reviewer. The next useful data points are prospective deployment, multi-pathologist adjudication in the external cohorts, and a check that the stroma undercount does not widen in slides prepared outside the training distribution.