NPR Mind Reading: Technology Turns Thought Into Action
by Jon Hamilton
Watching The Brain Listen To Music
In theory, researchers could receive signals from hundreds or even thousands of electrodes. So far, they haven't gone beyond dozens, yet the results have been spectacular.
Schalk shows some of what ECoG can do in his lab. There aren't any animals or test tubes here, but there are plenty of computers, including one playing Pink Floyd's album The Wall.
Schalk is showing me the results of experiments he did using ECoG to monitor people listening to The Wall. He points toward two waveforms on the computer screen. One shows the mountains and valleys that represent changes in the music volume; the second waveform looks very similar, but it represents the electrical signals generated by the brain in response to the music.
"There's a very close correlation between the actual loudness in the music that is just playing right now and the intensity of the music that we're decoding or inferring from the person's brain," Schalk says. "Isn't that pretty awesome?"
In the second part of the music experiment, volunteers listened to Pink Floyd for about 10 seconds, then the music was interrupted by about a second of complete silence.
A Brain On 'The Wall'
In this experiment, researchers compared the volume of Pink Floyd music played to a patient (music power) with the data gathered from the brain listening to that music (decoded music power). Notice how the shapes of the waveforms are similar.
The brain signal is so distinctive you could almost recognize the music from the waveform alone, Schalk says.
The experiment shows that while it may have been silent in the room during the test, it was not silent in the volunteers' brains.
Schalk's computer screen shows that even when the music stops, the waveform from the brain continues as if the music were still playing. What we're seeing is the brain's attempt to fill in the missing sounds, Schalk says.
"The brain basically tells us a lot of information about the music in the times when there is really no music," he says.
It's a vivid illustration of something neuroscientists have been studying for many years, Schalk says. Whether it's musical phrases or strings of words or scenery we look at, our brains are always filling in missing information.