Scientists create brain implant that recognizes both words and gestures in people with paralysis
Scientists have developed a brain implant capable of simultaneously recognizing a person’s attempts to speak and use gestures. The system converts brain signals into text and movements performed by a digital avatar almost immediately after the user forms the intention to communicate.
The technology is designed primarily for people with severe paralysis who have lost the ability to speak and move normally. The study was published on September 14 in Nature Neuroscience.
One brain implant can decode both speech and gestures
According to Nature, the system can simultaneously translate verbal and non-verbal intentions into digital communication.
The implant uses an array of 253 electrodes placed on the surface of the brain. It covers a sufficiently large area of the sensorimotor cortex to detect neural activity associated with both speech and body movements.
Brain signals become text and avatar movements
The study involved two people with severe speech and movement impairments. One participant had experienced a brainstem stroke, while the other had amyotrophic lateral sclerosis (ALS).
- the system recognized attempts to produce specific phrases;
- it detected attempted or imagined gestures;
- it distinguished nodding, hand movements, shoulder shrugs and other actions;
- it could process speech and accompanying gestures at the same time.
Artificial intelligence analyzed the brain’s electrical activity. Recognized words were then displayed on a screen, while detected movements were reproduced by a personalized digital avatar.
The brain-computer interface remains experimental
The study in Nature Neuroscience is currently a proof of concept rather than a ready-to-use medical device. Implanting the system requires surgery, and its algorithms must be adapted to each individual user.
However, the research demonstrates the possibility of building a single brain-computer interface that restores not only words but also elements of non-verbal communication. In the future, such systems could make communication significantly more natural for people with severe paralysis.
