Google's medical AI can now watch and listen during video visits. It's still a research demo

When a patient coughs, winces, or shifts in pain during a video call, Google's research medical AI AMIE can now pick it up. In a randomized study of simulated consultations with patient actors, the system, which previously only handled text-based medical chat, used real-time audio and video to guide virtual physical exams and reason about visual and auditory cues a text chatbot could not.

Patient actors played the cases, and the comparison baseline was a text-based chatbot, not in-person clinicians. Clinical evaluators scored AMIE favorably across history-taking thoroughness, diagnostic accuracy, management appropriateness, and communication quality; the actors also preferred the video version to text. The video mode added perceptual data, including gait, posture, breathing, and wince, that text chat cannot read.

AMIE remains a research system, the work is a self-published arXiv preprint not yet peer-reviewed, and Google's own post says "more research is needed before responsible real-world clinical deployment." No real patients, no regulator, no clinical product, only a vendor demo of what an audio-visual medical AI could look like, pending the next round of validation.