Forward-looking: A recently published study has revealed the results of research and experiments to reconstruct musical patterns from patient brainwaves and electrical signals using AI. The researchers aim to advance communication and machine-brain interfaces for patients with neurological or other conditions preventing speech. Based on the results, they're on the right track.
Initially published in PLOS Biology, the study centered on 29 patients undergoing brain surgery at the Albany Medical Center in New York City. During the surgery, aimed at potentially curing intractable seizures, the patients volunteered to have electrodes attached to their brains as Pink Floyd's Another Brick in the Wall played in the background during the procedure.
Researchers used the electrodes to record the electrical activity generated from each patient's brainwaves, later using AI to analyze and combine the data. The result is a distorted, but without a doubt recognizable, recreation of the song.
The original recording is well known for its rhythm and harmony:
The brainwave-based recording (below) is a garbled but undoubtedly identifiable recreation of the original 15-second track above.
The team trained a machine-learning model using the data collected from the 29 volunteer patients to complete the recording. According to the study, researchers played 90% of the song for the patients during their procedures, leaving out a 15-second portion. The team used the AI algorithm to fill in the blank by recreating the missing 15 seconds of music using the brain activity data collected.
Reading and reconstructing brain activity is not a new field for neuroscientists. Researchers have already used AI technologies to decode brain signals into visual images and used GPT-based models to generate descriptions of a subject's thoughts. Labs have also conducted experiments in the other direction – delivering pictures and information directly to the brain while bypassing receptors such as the optic nerve.
The study and its results could form the foundation for future technologies that restore speech capabilities for patients who have lost their ability due to stroke, illness, or other injury. Though the recording is far from a perfect recreation, it marks a giant leap forward in our ability to restore speech and communication capabilities to patients unable to interact verbally with the world around them.
