Beyond the Demo: What It Actually Takes to Run AI Inside a Hospital

The AI tool worked in the lab. It triaged chest X-rays in the proof-of-concept, drafted discharge notes faster than the resident, and pulled the right guideline at the point of care during the demo. Then it had to live inside the hospital, across the electronic health record, the imaging archive, the lab system, the billing platform, the identity provider, and the audit log. That's where it stopped working.

This is the binding constraint on healthcare AI in 2026, according to EE Times Principal Engineer Beenish Zia: not how good the model is, but whether the infrastructure underneath it can carry a clinical workload without breaking compliance, latency, or clinician trust. In a bylined analysis for EE Times published 07.27.2026, Zia, who leads infrastructure work at Intel, argues the gap between AI pilot and enterprise scale in healthcare is structural, and that the engineering questions below decide whether the next demo survives contact with the rest of the hospital.

The reframe matters because most of the public conversation is still about model capability. Zia's argument is that LLMs already shifted the field from narrow task models to general-purpose clinical reasoning, and hallucination is now a patient-safety issue in discharge summaries, not a research curiosity. Once that's conceded, the design questions stop being about which model to deploy and start being about the run-time environment the model lives in. Compute, token throughput, memory bandwidth, and right-sizing become first-order constraints, not afterthoughts.

Zia lays out six infrastructure requirements that decide whether a healthcare AI system actually ships, and they read as a procurement filter rather than a wish list. Data sovereignty and on-prem inference is first, because protected health information has to stay inside the institution's perimeter. Second is heterogeneous compute spanning edge to cloud, with portable software stacks that don't lock a deployment to one accelerator vendor. Third is tens-of-millisecond latency at the point of care, the budget a clinician or bedside device can tolerate before the AI becomes a friction point rather than a tool. Fourth, graceful degradation and uptime: a model failure should degrade to a known-safe state rather than dropping the workflow. Fifth is hardware-software co-optimization, where the gain is measured in multiples of throughput per watt and per dollar, the economics that decide whether a deployment survives a budget cycle. Sixth is security and confidential computing across the full stack, from the edge sensor to the audit database.

Those requirements are not a wish list, and Zia treats compliance, governance, and explainability as engineering inputs: interpretable intermediates, saliency and attention signals, retrieval-grounded generation, calibrated confidence, audit trails with model and dataset versioning, and FDA SaMD (Software as a Medical Device) predetermined change control plans, the mechanism regulators use to approve software that is expected to change.

Zia splits the deployment surface three ways. Clinical AI (imaging triage, digital pathology, ambient documentation aimed at reducing the pajama time clinicians spend finishing notes after hours) has the most measurable outcomes today. Operational AI (revenue cycle, OR scheduling, supply chain) runs on the same infrastructure constraints, but on a different risk profile, because a wrong scheduling optimization is a lost slot, not a misdiagnosed patient. Research and life-sciences AI (genomics pipelines, lab instruments that are now intelligent edge devices) is where the inference moves closest to the sample.

Federated learning gets called out as the multi-site pattern that keeps patient health data where it lives: models travel to the data, only parameter updates aggregate. It is the architecture that lets a health system learn from a partner hospital without ever centralizing patient records.

Zia also draws a line the public conversation routinely blurs. Agentic AI is the category that includes prior authorization, multi-step patient intake, and care-gap outreach, and it is error-recoverable and human-in-the-loop, so a wrong step is a correction, not a clinical event. Physical AI is the category that includes robotic-assisted surgery, bedside monitoring, and wet-lab automation, and it closes the perception-to-action loop, where a model error is the action. The infrastructure bar is not the same in both.

For a clinical, IT, or operations leader staring at a stalled pilot, the diagnostic lens is the same. Before the next procurement, ask where the data actually lives, and which system of record is authoritative. Ask what the latency budget is at the point of care, in milliseconds, not vibes. Ask who owns the audit trail, and whether it is versioned against the model and the dataset. Ask what the workflow looks like on a bad day, and what the system degrades to when the model fails. Ask where compliance lives in the stack, and what the change-control path is for the next model update. If the answers are vague, the demo will not survive the hospital.

The next model release is not the milestone to watch. The next procurement meeting is, because that is where the six infrastructure questions either get asked or do not.