AIBRIEF
Zhejiang University researchers released Holtercare Bench, the first public test of medical AI on long term heart rhythm recordings, and report a "significant performance gap."
A new public benchmark, Holtercare-Bench, offers the first serious grade on how multimodal AI handles a full day of a beating heart. The paper's own finding: a "significant performance gap" on long pathological heart-rhythm sequences.
The dataset, called Holtercare-23K, draws 22,980 question-answer pairs from 788 clinical Holter recordings. A Holter is a portable device patients wear for 24 hours or more to capture every electrical beat of the heart, the long noisy stream cardiologists actually scroll through in clinic. Most prior medical-AI work has tested on still images or short clips, not this kind of data.
The benchmark covers three task areas: temporal localization, clinical diagnosis, and global summarization, split into 12 sub-tasks across three cognitive tiers, from closed question-answering to full report generation. Splits are partitioned by patient to prevent data leakage. Under zero-shot evaluation, top multimodal large language models fall short on these ultra-long pathological sequences, the authors report.
Fine-tuning on the dataset lifts model scores substantially, the paper notes. That gain is a training result, not a clinical result. The release is research infrastructure, not a deployed diagnostic; the paper concludes that current models are not yet adequate for long electrophysiology data. Code, data, and benchmark sit public on GitHub from a Zhejiang University 4HealthCare team.