AIBRIEF
oncothresh evaluates tumor AI models at the cutoff a clinic uses to decide, not in aggregate, but lands as a v0.1 release with no clinical validation yet.
Most clinical-AI evaluations are scored with a single number, AUC (area under the ROC curve), that measures how well a model ranks cases across every possible threshold. AUC says nothing about what happens at the specific probability cutoff a pathology lab actually uses to flip a case into "biopsy yes" or "biopsy no." A tumor-cellularity model, one that scores the share of tumor cells in a sample, can post 0.92 AUC and still be unreliable at 0.6, the cutoff that decides the call.
oncothresh is one developer's v0.1 attempt to package the metrics that answer that question. Given a chosen threshold, it returns sensitivity, specificity, positive and negative predictive value (PPV and NPV), bootstrap confidence intervals, threshold-sensitivity curves, decision-curve net benefit, and number-needed-to-test, the count of patients who must be screened to catch one true positive. A companion dashboard, oncothresh-web, runs locally via Docker and exposes the same calculations through a web UI for clinicians who don't write Python.
The project's Reddit announcement frames the gap: existing pathology foundation-model benchmarks, including PathBench, PathBench-MIL, and Patho-Bench, score models globally, not at the cutoffs that decide care. The repo cites a 2025 Lancet Digital Health commentary on prediction thresholds in oncology AI; that commentary's existence and exact wording have not been independently verified here.
That caveat is the story. The threshold-evaluation gap is real, and clinical-AI teams can audit their models against it with existing tooling today. oncothresh is one packaging attempt, not the answer.