A 96%-Accurate Security AI Missed Every Single Web Attack

A federated intrusion-detection model posted roughly 96% global accuracy on the CICIDS2017 network-traffic benchmark, a number that would normally make a security AI team comfortable. On one of its training silos, the slice holding every Web Attack sample in the dataset, the same model sat near 49% accuracy with 0.00 recall, missing every single positive example of that attack class. The dashboard would have looked healthy. The threat it was built to catch would have walked through.

The result comes from a project description posted to r/MachineLearning comparing three federated aggregation methods, FedAvg, FedProx, and FedNova, against a centralized baseline. The federation was split into four simulated silos by attack type, and the Web Attacks silo contained only about 3,000 samples out of roughly 3 million total. The setup was, in other words, deliberately weighted toward exactly the imbalance that real production networks carry.

The deeper problem is not federated learning. It is how the field evaluates these systems. Global accuracy rolls performance across every class into one number, and a 96% headline is easy to hit when a single dominant class carries the score. A model can be near-perfect on benign traffic and still score zero on the attack type that matters most. The Web Attacks silo, at 0.00 recall, is the limiting case: the model has effectively declared that category invisible to itself.

The baseline that is supposed to anchor the comparison does not rescue the story. The same project re-ran a centralized, non-federated, model across ten random seeds on the same data, with no other changes, and watched the Web Attacks silo's accuracy swing from roughly 57% to 99.5%, a near-coin-flip to near-perfect spread driven by nothing more than the random seed used to initialize training. Researchers who cite a single centralized number for a security AI baseline are, often unknowingly, reporting a number that another reasonable random seed would contradict.

The federated methods told a different story on the same data. FedAvg, the most common aggregation rule, replicated the 96% / 0.00 split: high global accuracy, total miss on the rare class. FedProx, which adds a stabilizing term to local updates, did not recover the minority either. FedNova, which normalizes local updates by the number of training steps each client took, held in the high 90s across all four silos and all ten seeds without giving up global accuracy. The author is currently rewriting the work for conference submission, so the figures are pre-publication and unpeer-reviewed, but the choice of aggregation rule materially changes whether a rare class survives training.

CICIDS2017, the benchmark in question, was published in 2018 by researchers at the Canadian Institute for Cybersecurity at the University of New Brunswick in a peer-reviewed methodology paper describing a five-day capture of benign and attack traffic that has since become one of the standard corpora for intrusion-detection research, mirrored on IEEE DataPort. That pedigree grounds the dataset choice as legitimate rather than obscure, and it underlines the per-class problem: the field has been running the same standard benchmark for eight years while measuring it with metrics that hide its hardest cases.

The reader's test is straightforward. The next time a security AI paper, dashboard, or vendor pitch leads with a single global accuracy number, three follow-up questions expose the gap: what is the per-class recall on the rarest class, what is the per-client performance if the model is federated, and how many random seeds produced the headline number. A 96% global score that swings to 49% on a single attack category, and that a centralized baseline can move from 57% to 99.5% with a fresh seed, is not a deployment-ready model.