Seven AI Teams Tried to Read Brain MRIs for Hidden Dementia-Risk Markers. The Catch: They Don't Travel Between Hospitals.

Seven deep-learning teams entered a standardized MICCAI 2024 challenge to test whether AI can segment enlarged perivascular spaces on real hospital brain MRI scans. The top method, a MedNeXt-based pipeline with a dual 2D/3D strategy, held up on scans from the hospitals that helped train it. It degraded sharply on scans from a Shanghai cohort it had never seen.

Enlarged perivascular spaces, or EPVS, are tiny fluid-filled pockets that trace the path of blood vessels as they dive into the brain. On standard MRI they show up as bright or dark dots and dashes, easy to miss unless someone is specifically counting them. Radiologists care because EPVS counts are a recognized early marker of cerebral small vessel disease, the diffuse damage to small arteries and veins that is a leading vascular contributor to dementia and stroke. The pitch for automating the count is straightforward: a reliable AI screener would let any hospital that buys the software flag patients who need a closer look, before symptoms land them in a memory clinic.

The MICCAI 2024 EPVS Challenge, organized at the Medical Image Computing and Computer-Assisted Intervention conference and now peer-reviewed in Medical Image Analysis, was built to test that pitch honestly. The organizers pooled 100 training, 50 validation, and 50 testing brain MRI scans drawn from sites in the United Kingdom, Singapore, and China, covering different scanner brands, slice thicknesses, and patient populations. Ground truth followed the STRIVE protocol, the field's agreed-upon standard for how vascular changes on neuroimaging should be reported. Seven teams completed the full challenge; most fielded U-Net-style convolutional networks or transformer-based variants, and several stacked ensembles of both. Evaluation used four standard segmentation metrics: the dice similarity coefficient (a 0-to-1 overlap score), absolute volume difference, recall, and precision.

The winner, MedNeXt with a 2D/3D dual strategy, was developed by a team at the Australian e-Health Research Centre, CSIRO, and independently confirmed in AEHRC's own announcement of the win. The dual 2D/3D design handles the fact that clinical MRIs come in mixed formats: some hospitals acquire thick 2D slices, others produce near-isotropic 3D volumes, and a single architecture has to make sense of both. On the test split drawn from the same sites that contributed training data, the top method and the next several runners-up posted solid dice scores, the kind of numbers that read as "ready for prime time" in a vendor brochure.

The catch is in the held-out Shanghai cohort. None of the training, validation, or primary testing data came from that site, so it functioned as a clean test of whether the models had learned the general task or just memorized the local hospital's look. The peer-reviewed paper's headline finding: top solutions "showed relatively good performance on test data from seen datasets, but significant degradation of performance was observed on the previously unseen Shanghai cohort." The field did not fail, but it also did not generalize the way a clinical deployment would require.

That gap is structural, not a tuning problem. Every hospital runs a different scanner brand with a different field strength, a different slice protocol, and a different patient mix by age, vascular risk, and ethnic background. EPVS, in particular, are small, low-contrast, and easy to confuse with small lacunes or random noise, so they reward models that have seen the local flavor of the problem. Domain shift of this kind has been the consistent failure mode of medical-imaging AI for a decade, and the EPVS challenge is the first time the field has measured it cleanly for this specific small-vessel-disease marker.

The dataset, code, and challenge infrastructure are all open. The official challenge data is on Synapse and Zenodo, and the challenge pipeline is at hzlab/EPVS_challenge. That is a deliberate feature of the design: the organizers are betting that the next round of submissions, trained and tested on the same data with the same held-out site, will be the way the field closes the Shanghai gap.

For now, the EPVS story is a familiar medical-AI story with a clean measurement attached. A well-designed multi-site benchmark can be won, and the winning method can still be wrong in the way a deployment would care about. The next paper that moves the dice on the unseen cohort, by whatever combination of pre-training, site-aware fine-tuning, or synthetic data augmentation, is the one that actually puts automated EPVS screening within reach of a hospital that did not build the model.