A Brain-Computer Interface Decoded Speech and Movement Simultaneously for the First Time

A new brain-computer interface (BCI) is the first to translate both speech and movement with a single device that reads data from implanted electrodes. The BCI simultaneously communicates verbal and non-verbal information in seconds, offering a breakthrough for individuals with severe paralysis. The system uses artificial intelligence to decode neural signals from the brain, allowing control of a digital avatar through both speech and movement. Researchers tested the BCI with two volunteers, one with a brainstem stroke and another with ALS, to assess its ability to decode speech and gestures.

The BCI was implanted with 253 electrodes on the surface of the cortex, capturing a large number of signals associated with attempted speech and movement. The volunteers were asked to perform a variety of actions, including waving, shaking hands, clapping, and nodding, as well as attempting to speak silently. The BCI recorded neural activity for each attempt, and AI models were trained to translate these signals. The team found that including data from simultaneous activities improved decoding performance. The first volunteer achieved perfect decoding of the tested movements and speech, while the second recorded 75% speech accuracy and 85% gesture accuracy.

This development represents a major step forward in BCI research, as it allows for the simultaneous decoding of speech and gesture, which are both essential components of human communication. The BCI can be used to control digital avatars, offering new possibilities for individuals with paralysis. However, more work is needed to refine the technology and expand its capabilities. The paper, “Simultaneous Speech and Gesture Decoding for Multimodal Communication in Paralysis,” appeared in Nature Neuroscience on September 14, 2026.

AI-generated summary of the original article — see the source for the full story.