Researchers develop deep-learning method for translating vocal signals from the brain...

Cal Jeffrey

Posts: 4,140   +1,406
Staff member
In a nutshell: People who have lost the ability to speak due to severe paralysis might have hope for a better way to communicate, thanks to technology that essentially reads their thoughts and translates them to words and complete sentences in real-time. By tapping into the part of the brain that controls the voice box, researchers have figured out how to read those signals and turn them into words with sophisticated AI.

Researchers at the University of Califonia San Francisco developed a way to allow people with speech loss to communicate with their brains. The technology uses neural networks to translate brainwaves into words and phrases. It is a breakthrough because until now, the best neuroprosthetic technology has provided is letter-by-letter translations, which is very slow.

Aside from the highly refined algorithms, the new method taps the part of the brain that sends signals to the voicebox. Previous techniques used brain areas that controlled the hand or arm. It seems almost too obvious that scientists should have used this section of the brain from the start, but such is neurological research.

In developing the system, UCSF researchers recorded brain signals of volunteer subjects with unimpaired speech. The scientists fed the patterns to neural networks, which learned to decode them in real-time. They also applied a statistical language model to improve algorithm accuracy.

The researchers' main obstacle was whether or not the area that controls speech would function the same way in impaired subjects as it did with those with normal speech. So they employed an anonymous impaired volunteer (Bravo1) and worked to create a simple 50-word vocabulary to feed the algorithms—practical words he could use in everyday life like water, good, I, yes, no, and so forth.

During testing, the team asked Bravo1 simple questions such as "How are you?" or "Do you need anything?" When Bravo1 tried to reply verbally, the computer would translate his impulses into phrases like, "I am fine," or "No, I don't need anything."

"To our knowledge, this is the first successful demonstration of direct decoding of full words from the brain activity of someone who is paralyzed and cannot speak," said UCSF neurosurgeon Edward Chang and Professor Jeanne Robertson, senior author on the study. "It shows strong promise to restore communication by tapping into the brain's natural speech machinery."

The system is still relatively slow. It can translate up to 18 words a minute—for comparison, unimpaired people can speak at about 200 words per minute. Still, it is faster than any other previous neuroprosthetic system developed. It has a peak translation accuracy of 93 percent and a median of 75 percent.

However, this is just the start of an expanding study. The team's next steps are to increase the number of test subjects, expand the system's vocabulary, and improve the rate of speech translation.

Permalink to story.

 
The question I have here is if words (or thoughts) are encoded the same for all individuals. This implies "yes", which means that generic man-machine are technically feasible.
 
The question I have here is if words (or thoughts) are encoded the same for all individuals. This implies "yes", which means that generic man-machine are technically feasible.
It's not the same for all people. But there are similarities and the software is then trained and personalised for the individual. Machine learning is great for this type of data and the more they feed it the better.
 
The worrisome part is if they go with a speaker instead of typed text, a large majority of men, walking down the street are going to get slapped, kicked, and sued for what they are thinking when a nice looking girl walks past them ...... so expect to see a LARGE contribution from the legal world on this one!
 
This is cool and creepy at the same time. I do wonder how it works with lies, when basically you think one thing but would say something else. Would it register the original thought you want to keep hidden, or the one you would want to say...?
I believe they are registering the the brain signals that are related with talking and not thought.
 
I have already increased my invested shares in the Reynolds Group Holdings. Got a feeling aluminum foil is about to be a great investment!
 
This is cool and creepy at the same time. I do wonder how it works with lies, when basically you think one thing but would say something else. Would it register the original thought you want to keep hidden, or the one you would want to say...?
From my limited understanding of the tech, you can still lie because it is translating the signals that are being sent to your vocal tract.
 
It's not the same for all people. But there are similarities and the software is then trained and personalised for the individual. Machine learning is great for this type of data and the more they feed it the better.

Yes. They initially trained the neural networks with several volunteers with normal speech (unimpaired). Then with Bravo1, they built a customized vocabulary of about 100 words capable of putting together more than 1,000 sentences. In the video, you can see it's still rather slow, but much faster than previous techniques requiring the subject to spell out words letter-by-letter.
 
Back