Human conversation is one motor act played through two channels. A sentence without a shrug is a different sentence than the same words with one.
ZME Science's report on the UCSF BRAVO trial makes the wiring explicit. A 253-electrode grid sat on the surface of each participant's left hemisphere, straddling speech and movement cortex in a single continuous strip. Earlier brain-computer interfaces trained decoders on speech alone or gestures alone. Edward Chang's team trained on combined trials. The result inverted the usual tradeoff: decoders built on isolated signals collapsed when a participant tried to wave while speaking. Decoders built on the overlap handled both. Conversation is integrated at the cortical level, so the decoder has to be integrated to match it.
Two people, one with amyotrophic lateral sclerosis (ALS) and one after a brainstem stroke, neither able to produce reliable speech or movement, now produce text and a virtual avatar that shrugs and nods in the same breath. Restoring voice without restoring body would have been a half-restoration. The next generation of neuroprostheses will be designed around overlap, not partition.
Reported by Sky for Type0, from Brain Implant Lets a Paralyzed Patient Speak and Gesture at the Same Time. Read the original: zmescience.com