Neural Mechanisms of Voice Control
Sep 5, 2023
Neural Mechanisms of Voice Control
Chuck Larson, Northwestern
Okay, I was probably a little bit overambitious in what I sent in.
I'm not going to get to all the things that Bob talked about,
but I am going to talk about some things that we're currently doing in the lab,
and I hope it's of interest to you.
To begin with, I just want to give you a little bit of background.
I began my work in this general field a long time ago,
working primarily with primates, and I would train them to vocalize,
and then we did standard neurophysiological techniques of using microelectrodes
to record from neurons in various regions of the brain.
After a while, I began feeling like I wasn't able to really do the types of experiments I wanted to do,
and so I wanted to move on to do something that was a little bit more in line with some other techniques.
I went to the other types of motor systems that have been used for years and years,
and one of the hallmarks of some of the early work,
in fact, going back to the days of Sherrington 100 years ago,
was the perturbation technique, namely jerking limbs around and looking at the response to them.
It's been done in various avenues, including speech.
In fact, some of the early work that I did were perturbing the lips and jaw and things like that,
but I was still interested in doing the voice.
Nevertheless, with the perturbation technique,
some of the advantages are that you can look at the timing between a stimulus and a response.
You can look at automatic responses.
You don't necessarily have to rely on volitional types of responses,
and you can do some parametric scaling looking at the gain of the system.
When I say gain of the system,
I was toying with the idea of showing all these negative feedback systems in my talk today,
but I knew all of you are totally familiar with them,
so I'm not going to show you slides of models.
You know what I'm talking about, hopefully.
I was interested in how we could study the voice using this technique,
and believe it or not, some of the first work I did is I took a servo motor,
and I was lying down, and I had the servo motor against my larynx,
and I was applying pulses,
much as a good neurophysiologist would do to look at some motor system.
It was a dreadful experiment.
It just didn't work out at all.
Finally, it occurred to me, well, what is the significant thing about the voice?
Well, it's an auditory signal.
So I thought, well, how do we perturb the auditory signal?
And it suddenly dawned on me,
well, what we do is we take the auditory signal,
and we process it and feed it back to the subject in a perturbed fashion.
And so that's how I began this work.
And all of it falls around the general theme
of how is it that we use auditory feedback for voice control.
And so what I'm going to do in my talk today
is break it down into two different components.
Behavior, where we talk about just vocalization per se,
as well as vocal responses to perturbations in auditory feedback.
So that's the basic paradigm.
But then lately we've been doing a lot of work
on the neural control of these mechanisms,
and I'm also going to be talking about EEG recording and analysis
that we're doing during these behavioral paradigms,
and just a little bit of work on fMRI.
We've also gotten into work using transcranial magnetic stimulation.
I'm not going to talk about that today.
I just thought it was too much.
And we're moving in the direction of using integrated EEG and MEG.
But there again, I'm not going to be talking about that today.
So this slide just briefly shows the experimental paradigm.
We have a speaker talking, taking the acoustical output,
and we run it through a pitch shifter off a shelf device
and feed it back to the subject's ear.
And in the lower part of the figure, what we see,
this is a trace of fundamental frequency of the voice output.
It's not the AC signal.
We take the AC signal, and we take each glottal cycle,
compute the time intervals between pulses
and the inverse of that to give us frequency.
So this just shows voice frequency as a function of time.
It's fundamental frequency.
So that's the voice.
In the lower curve here, this is the feedback.
And so the feedback is constant,
and at the dotted line we introduce what we call a pitch shift stimulus.
In this case, the stimulus is a downward change in fundamental frequency.
The duration of this pitch shift stimulus is about 200 milliseconds.
We prefer to use brief pitch shift stimuli like this
in order to keep the response
in what we would call more of the automatic mode.
In one of our first studies,
what we did is we manipulated the duration of the stimuli,
and we found that if the stimulus was much greater than about 300 milliseconds,
we began to get a voluntary type of response.
Now, voluntary responses are great.
I use them all the time, like now.
However, in order to understand
some of these basic levels of neural processing,
we wanted to keep it at the automatic level.
So we generally use a very brief stimulus.
I will say that right now in the lab
we are turning our attention to the voluntary responses
because there's a huge amount of work that we've overlooked,
and we're turning to that in the future now.
At any rate, with this pitch shift stimulus,
we can see that there's an upward change in the voice fundamental frequency,
and this is the response that we're interested in.
Now, this gives some other examples.
First of all, just to orient you a little bit,
the black line up here, these are control trials
in which a subject was vocalizing when there was no pitch shifted feedback.
In the upper case here, we have an upward pitch shift stimulus.
The red trace indicates the response to the pitch shift stimulus,
and the vertical lines are just the error bars.
Now, in the bottom trace, we have a downward pitch shift stimulus,
and here we see an upward response.
So we call both of these compensating responses,
namely because with an upward stimulus, you shift your voice pitch down,
and with a downward pitch shift stimulus, you shift your voice up.
So what's going on here is that your motor system is compensating
as if it were an error in your production,
and this is the way all motor systems appear to work, at least in part.
Mine doesn't work too well because, as you can see,
this light jiggling around there, that's just reflecting my tremor,
a little bit nervous, so it's shaky.
So my nervous system isn't correcting 100% for that,
but it doesn't do that for the voice either.
So we call this the pitch shift reflex,
and it's a way for studying the role of auditory feedback on vocalization.
As I indicated, it's a perturbation paradigm.
The stimuli are generally unexpected and therefore unpredictable,
although we've done a recent study in which we've shown that if they are predictable,
then we get differences in the EEG signal, but I'm not presenting that today.
The other thing is that this paradigm mimics natural conditions,
namely you're taking the subject's voice and you're just manipulating it,
feeding it back, and you can measure how they react to and correct for errors,
and you can quantitatively relate voice output to changes in auditory feedback.
Before going forward, I want to go backward to illustrate one other thing,
just for sake of clarification.
These figures here are very misleading.
They're very misleading because what I've drawn here is a very tiny stimulus
and a huge response.
It's actually the other way around.
That stimulus should be way up there or higher, and the response is smaller,
but I've just drawn it this way so you can see the nature of the response.
Typically, the responses are about 20% to 30% of the stimulus.
They're very rarely at the same magnitude as the stimulus.
We have seen that occasionally, but we don't understand the reason for that.
I think some people are just hyperreflexive.
Let me go forward again.
What I want to do is show you the results from some of our speech studies.
A lot of the studies that we've done have been just with sustained vowels.
Subjects sit in our sound booth going, ah, over and over again,
hearing their voice fed back to them.
We had some studies, such as a study we did on Mandarin with Yi Xu some years ago.
We had subjects saying three different phrases.
In this case, I'm showing the phrase ma, ma.
If you speak Mandarin, that's a high tone, two high tones in a row.
I don't know what it means.
We also had a high tone followed by a rising tone or a high tone followed by a falling tone.
I'm just showing the two high tones.
There's two syllables, ma, ma.
Here's the pitch shift stimulus, the downward pitch shift stimulus.
The yellow lines show the control.
That is without pitch shift stimulus in both cases.
The blue lines show the response.
You can see that in this case, the subject began responding in less than 100 milliseconds
after the onset of the pitch shift stimulus.
With this downward shift, the subject raised his FO as if to compensate.
On the other side over here, saying the same phrase, there's an upward shift.
Here the subject responds by lowering the FO.
One thing that's important to keep in mind is when subjects are doing this,
they don't realize that they're making this response at all.
If you say a phrase like ma, ma, you are changing your articulators.
When you go from an ah to an em, your lips close.
The nature of the acoustical signal changes.
The fundamental frequency changes a little bit.
You can see in the yellow line, there's some changes going on there.
We've got that change going on, but then we've got this time-synced change right there.
They don't realize they're producing this response, but in fact they are.
Here is an example from a study we did on English speech.
We had people saying the phrase, you know Nina?
We tried to get them to produce it the same way all the time.
Before each vocalization, they would hear this standard phrase, you know Nina?
We tried to make it so that the first part of the phrase was all steady level FO.
Then they just raised their fundamental frequency right at the tail end, na.
What we did then is we introduced these pitch shift stimuli you see at the bottom,
indicated by the dashed line.
On the left, there's a downward pitch shift stimulus.
The black lines indicate the control trials in which there's no stimulus.
Then the blue lines indicate the trials where there's a response to the pitch shift stimulus.
When the subject is preparing to raise the fundamental frequency here,
and if the voice auditory feedback is shifted down,
they begin early to elevate simply because what they're hearing at this point in time,
starting there, is that their voice is going the wrong way from where they want to go.
They respond by an early response to increase their fundamental frequency.
On the right hand side, it's just the opposite.
They know this is what they want to say, na.
But at this point, what they're hearing is it's going up too soon, too high, too soon.
So they lower their FO in order not to produce this too early, I presume.
That results in this response.
I also want to point out that the same subject, when they're just sustaining an ah sound,
the responses are much smaller.
What this tells us is that not only do people respond to these stimuli during normal speech,
but these reflexes are task-dependent.
That is, they're modulated by the task.
That's one of the things that all of us who are interested in automatic or reflexive types of responses
are interested in is whether or not they can be modulated by task,
because that tells us something important about the nervous system.
We've also begun doing some work in neurological diseases.
Here are data from subjects with Parkinson's disease.
What you see here are, first of all, on the left, responses to pitch-shifted feedback.
But we've also done work by shifting loudness of feedback, which you see on the right.
In both cases, the Parkinson's patients are indicated by red lines
and the control subjects by black lines.
You can see that the Parkinson's patients produce larger responses than the control subjects,
both for pitch-shifted feedback as well as for shifting loudness feedback.
I don't know if you're aware of this or not, but one of the speech symptoms of people with Parkinson's disease
is that they tend to talk too quietly.
A standard phrase you hear one of them say is,
my spouse needs a hearing aid.
I'm talking at a normal loudness, but my spouse can't hear me.
They have the impression that they are talking loud enough, but they're not.
Also, their voice tends not to have a lot of inflection. It's flat.
That's their normal speech.
What we're showing here is that these subjects respond abnormally to these stimuli.
We don't precisely know the reason for this.
We think, however, that there's an imbalance in the efferent copy of the signal.
That is, the output of motor cortex into the sensory system is somehow abnormal,
making them think they're talking louder than they are.
But we don't understand how that relates yet to the increases in these response magnitudes.
We're beginning a project this summer where we are going to be recording EEG in Parkinson's patients
at the same time they're doing this.
We have a way we think we can get at the efferent copy mechanism.
Now, I want to talk somewhat about neural mechanisms underlying these responses.
I want to begin by showing you a slide from a study done by John Hood.
John was using, in this study, magnetoencephalography.
He triggered the MEG machine off the onset of the subject speaking.
What we see here is that this heavy line represents the response while the person is vocalizing.
They recorded the signal at the same time the subjects were producing it,
and then they played it back at the same loudness level.
What they saw is that the MEG response was larger during the listening to the signal
compared with when they were producing it.
John termed this speaker-induced suppression, SIS.
This is fairly common in the motor systems.
As you probably know, you can't tickle yourself.
Go ahead and try it.
It's thought that you can't tickle yourself because your nervous system is suppressing
the sensory feedback from your own motor production.
That's a fairly common thing.
It's more widely discussed as motor-induced suppression.
That's a very important observation that John made.
That was followed up by a study by Hanks Maldonado and colleagues.
What they did was record electrical ERPs.
The dashed line shows an EEG response to the normal voice.
When they pitch shifted the voice, they got a larger response.
This is similar to what John showed, namely that normal production is suppressed.
In John's case, if you're listening, it's not suppressed.
What Hanks Maldonado showed is that if it's shifted,
i.e., if it's not your same voice, then you respond differently.
That prompted us to ask the question,
what are the limits of voice deviation for identification of self-voice?
What we did is my graduate student, Roozba Baroosman,
is that he applied pitch shift stimuli at voice onset of 0, 50, 100, 200, and 400 cents.
By now, you might be wondering, what's 100 cents?
100 cents is a semitone.
It's a logarithmic scale.
There's 1,200 cents to an octave.
We use that measure because it gets us away from,
well, this subject had a fundamental frequency of 130 hertz,
and this person had a fundamental frequency of 220 hertz.
How do we compare them?
Well, if they're in the cents scale, we can easily compare them.
Our stimuli are calibrated in cents, and we look at the response in cents.
He presented these stimuli in this paradigm,
subjects vocalizing, and at the onset of voice, the pitch shifter was turned on
so the subject is hearing either his normal voice or a shifted version of that.
This slide shows EEGs, the top row from electrode FZ, frontal midline,
the bottom row from central midline,
and we have pitch shift to 0 cents, 50, 100, 200, and 500 cents.
The dashed lines are the responses to passive listening,
and the heavy lines, solid lines, are due to speaking.
With 0 cents, you can see that the response to listening
is much larger than the response to vocalizing in both these locations.
Then as he increased the perturbation magnitude,
the response to speaking became larger in this N100 area.
Finally, we get out to 400 cents,
and we see that the response to vocalization and playback are identical.
This slide illustrates, averaged across all subjects,
the relative suppression we see as a function of the pitch shift magnitude at voice onset.
With zero shifting, there's quite a bit of suppression,
but as we increase the magnitude of the shift that decreases to about 400 cents,
it's virtually gone.
We now are thinking that when your voice is shifted at voice onset by about 400 cents,
you don't recognize it as your own voice.
It's like you could tickle yourself when it's not aligned properly.
We've got the suppression at onset.
The study we did agrees with the others,
namely that self-stimulation results in suppression.
Then if it's shifted by about 400 cents,
the N100 is almost completely eliminated.
We believe that this is one of the ways in which you identify self-vocalization.
In our original models of the system,
my colleague Tim Hain understood that when you're speaking or about to speak,
one of the first things you have to do is know if that's your voice or somebody else's voice.
One of the first steps in modeling this is to do recognition of self versus non-self.
We see at voice onset these differences,
which we believe help to identify self versus non-self.
Then the next question was,
is there suppression of responses during vocalization?
Again, here's the experimental paradigm.
The subject vocalizes,
and then there's a delay time at which the pitch shift occurs.
In this case, the time was randomized,
and we had either shifts of 100 cents, 200 cents, or 500 cents interjected.
This slide shows the results.
It might be a little bit difficult for you to see.
The left side is for active vocalization,
passive listening is on the right,
and the largest curve is for 500 cents.
The dashed line is 200 cents,
and the dotted line is 100 cents.
Here we're seeing that when we shift the voice during vocalization,
we get something quite different,
namely the responses become enlarged.
There's something else I want to show you.
Here we have again 100 cents, 200, and 500 cents.
When we compare the listening,
which is the dashed line, with the speaking,
we can see that the vocal potential to speaking
is much larger than it is to listening.
This is opposite of what we saw at voice onset.
During vocalization, we see something completely different.
To illustrate this differently,
Roosevelt took what's called the Normalized Difference Index,
which is the relationship between listening to speaking.
This is really unclear. I'm surprised.
This is the P100 response,
and this is the P200 response over here.
The black curves illustrate the response to 100 cents shift,
and then 200 and 500.
Left hemisphere, right hemisphere,
left hemisphere, right hemisphere.
What this shows us is that the smaller the shift,
the greater the difference between listening and speaking.
It tells us, then, that there's greater relative enhancement
of the signal for small shifts compared to large shifts.
In a way, that seems intuitive,
and it's something that we had suspected all the time.
Namely, the responses we get are fairly small.
That is, if we have a 100 cent shift,
we may respond with a 20 cent response.
What that means in everyday life is that,
if I'm going, ah, like that,
my voice may be fluctuating by 20, 30 cents,
something like that.
I'm responding at about 20, 30 cents.
I'm usually not going, ah,
which is like a couple notes.
That would be abnormal.
In just normal everyday vocalization,
our variability tends to be small.
In the same way, when I hold my dot there,
it's kind of small.
It's not going like that.
I'm not that nervous.
Our nervous system is geared more towards detecting
and correcting for small errors rather than large errors.
As part of this,
our voice, our naturally produced voice,
we have small changes, as I just said.
If it's a large shift,
it's probably an alien voice.
The system seems better designed
for correcting errors in production of your own voice
rather than others.
This shows the topographic distribution
of the event-related potentials.
I just want to draw your attention
to the difference between 100 cents and 500 cents.
Even if you're a moderately colorblind person like I am,
you can see there's a huge difference here.
If you look at these curves,
the colors vary quite a bit,
which indicates differences between negativity and positivity.
Then the actual locations,
like this versus this over here,
there's quite a bit of difference going on there.
When we go between those two extremes,
we're getting much different behavior
on the part of the individual,
different evoked potentials,
and different topographic distributions
in the evoked potentials,
which tells us that we have
some different brain mechanisms going on.
I've outlined the fact that at vocalization onset,
we get suppression.
During vocalization, we get enhancement.
There's a timing deal going on.
What's the deal here?
What we did is we did a study
where we varied the timing.
We presented pitch shifts at the start of vocalization,
200 cents delayed, 500 cents delayed,
and 1,000 milliseconds delayed.
This shows us some results,
0, 200, 500, and 1,000.
At 0, the EEG response to listening and vocalization
is about the same, pretty small.
But when we go up to 200 milliseconds,
the responses are about equal.
Out to 1,000 milliseconds,
the response to vocalization
is much larger than it is to passive listening.
The same results are graphed here.
This shows the P1 response,
P100, N100, and P200.
The colors red and yellow,
zero delay.
They're, for the most part, about the same.
But when we go up to 200, 500, or 1,000,
we can see that there's quite a bit of enlargement
in these potentials for the longer delays.
So there's quite a lot going on
between 0 and 200 milliseconds,
which then progresses.
And we don't know what that is yet.
We don't know what the mechanism is.
We have a standard negative feedback system model
that we use,
but we don't have a good timing element
in that model yet.
Okay, so just to summarize what I've talked about,
we've talked about voice onset
where we have identification of the source.
After voice onset,
we recognize, if it's our own voice,
and it's sensitive to errors in production
and corrects for errors in production.
Now, I want to show you a little bit about fMRI.
This is the only fMRI slide I'm going to show today.
I know you're going to be seeing a lot of that later on.
I've been doing this work in conjunction with colleagues
in San Antonio,
Don Robin, Amy Parkinson, et cetera.
And what we did is we did a sparse sampling routine.
That is, subjects would vocalize
for, like, 3 or 4 seconds,
and they'd receive several pitch-shift stimuli
while they were vocalizing.
And then after they had stopped vocalizing,
we would turn off the microphone,
and then we'd turn on the scanner.
And then we would scan for about 3 seconds.
And what we show here
is we see activations in auditory cortex.
And looking at the lateral side here,
we can see this activation in the posterior part
of the auditory cortex
in response to this pitch-shifted feedback.
Now, I want to show you this slide.
This is published by Hickok and Popol.
They've been discussing,
along with a lot of other people,
this sort of dual-stream hypothesis.
It comes from the visual system,
the what-where differences in pathways.
And people have been talking about this
in the auditory system as well.
And it's been suggested
that from auditory cortex
in the superior temporal plane up here,
there's one pathway that goes
towards the ventral part of the temporal lobe
and then progresses anteriorly.
And this is the so-called ventral stream.
And we can think of it loosely
as sort of the lexical interface.
Then there's a dorsal stream
that progresses posteriorly
into the junction of the parietal lobe
and temporal lobe and then anteriorly.
And it's thought that this is more
for sensory-motor interactions.
And I just want to go back to this slide.
That's kind of where we're seeing
this activation here.
So our fMRI activations are supporting
this model of Hick, Ecker, and Popol
that this part of the brain is important
for interactions between auditory feedback
and motor control.
Okay.
I just have one more study to talk about.
And that is we did a study with musicians.
This was done by an undergraduate
in our program last year, Nadine Ibrahim.
She wanted to look at musicians
with perfect pitch, relative pitch,
and non-musicians.
And so we had our standard paradigm here.
And she did testing for perfect pitch,
having them read and perform musical scales,
et cetera.
And there's some standardized tests
you do for that.
And she recorded EEG signals.
And on the left we have responses
to upward stimuli and the right
to downward stimuli.
The blue line represents the signal
for the musicians with perfect pitch.
And then the red line, relative pitch there.
And then the black, the non-musicians.
And the thing that really grabs you
is this really large 200 response
for the musicians with perfect pitch
compared to the others.
Over here with the downward stimuli
for the N100, both the relative pitch
and the perfect pitch musicians
had a larger potential than the non-musicians.
Doing a topographic analysis,
this is really interesting.
Here we have the non-musicians,
relative pitch, and perfect pitch.
This is in the time range
of 200 and 250 milliseconds.
And this is a little bit longer
time range of 200 to 300 milliseconds.
And what we see is that the musicians
with perfect pitch, they begin
to activate the left hemisphere
earlier than the other subjects.
Everybody is just doing a lot
of the processing in the right hemisphere.
But what's unique is that the musicians
with perfect pitch begin activating
the left hemisphere earlier
and in a longer time range
than the relative pitch musicians
begin doing this as well.
Now, several years ago, Gottfried Schlaug
did some studies using PET
and showed that one of the things
that's unique about musicians
with perfect pitch is that they
activate the left hemisphere auditory cortex
to a greater degree than musicians
with relative pitch.
He and nobody else has stated
why they do that, but at least
these results tell us something.
That is, these parts of the brain
that are active in the left hemisphere,
which would be activated,
shown by this N100 here,
because that's the negative potential
we're seeing here, the blue,
that begins earlier in the left hemisphere.
And so the people with perfect pitch
begin processing corrections
to errors in auditory feedback
much more in the left hemisphere
than do the non-musicians
or relative pitch musicians.
Now, there's another thing.
Here are the behavioral responses,
the voice responses.
Color coding, blue is perfect pitch,
red is relative pitch in untrained musicians.
On the left we show,
these are grand averages across all subjects.
With a downward pitch shift stimulus,
the musicians begin an upward response
much more rapidly than do the non-musicians.
And after the response,
the musicians start coming back down
much more rapidly than the untrained musicians.
It's even more clear with the upward stimulus here,
whereas the musicians have a very rapid onset
and then they turn it off rapidly,
whereas the untrained musicians are very slow.
So one of the things that comes with musical training
is that when people detect an error,
they correct for it very fast,
and they turn the response off very fast.
I guess it was Heifetz who said,
the great violinist,
I make a lot of errors,
but I correct for them very rapidly.
And so this is one of the things
that training does for you.
I'm sure your daughter's going to correct for her errors
as she progresses in her ice skating.
Okay, so then just in general
to talk about the vocal system.
It's similar to other motor systems.
It utilizes feedback to assist and control
responses to perturbations.
They're small in magnitude.
They're short latency.
They're directionally sensitive.
They're modulated by vocal task.
And as far as the neural locations,
I haven't shown you much of the fMRI data,
but a little bit.
We know they're activated in superior temporal gyrus,
temporal parietal junction,
and also M1 motor cortex.
We're also doing work with investigators
at the University of Iowa using ECOG,
and using the pitch shift technique with them,
we're able to show that M1 motor cortex
for the laryngeal area
is actually activated 30 milliseconds
after a pitch shift stimulus.
So this is a very tight network we're talking about.
I showed you earlier the fMRI work
from the SPAR sampling,
showing activation of auditory cortex
and the parietal temporal junction.
All that is occurring very rapidly,
reaching the motor cortex in 30 milliseconds.
Obviously, the fMRI results
are looking over a very long period of time,
and we can't necessarily say
that all that's happening within that short time range.
fMRI just doesn't allow that.
But there is some circuitry
that must follow a similar pathway.
And I just want to just thank
all my collaborators over recent years.
The red names are people who are currently in my lab
or collaborators, such as the group in San Antonio
and then a lot of people who worked with me over the years.
And no way could I have done this work
without the help of all of them,
and I'm deeply indebted to them.
Thank you. Yeah.
So I was struck by your self-identification study.
So 400 cents, four semitones.
Consciously, if you listen to your own voice
shifted by four semitones,
I would imagine that you could recognize it consciously.
But you're saying that at the N100 level,
you don't recognize it as your own voice.
Right. Yeah, I would agree with that 100%.
Yeah, you know it's your voice.
You can hear it shifted.
You're thinking, yeah, that's pretty weird.
But yeah, there's mechanisms there,
part of the automatic mechanisms
that are not recognizing it.
I just want to, I think...
Yeah, this...
We think like you guys do.
This is just to prove it.
This is our standard model,
and things are happening fast here.
And we've got efferent copies
comparing it with auditory feedback,
forming a prediction,
which is then fed back here.
There's something going on here
about the efferent copy.
When we have a large disparity,
right in here,
there must be something going on
at this point or at this point
where we do some comparison of the magnitude.
Some of it automatically, I think, is going on.
But then voluntarily,
there's other things going on.
So, yeah.
I just don't know
what those other mechanisms are yet.
One possibly correlated result
that comes to my mind
is the result on, I think,
on automatic separation
of two speakers talking at the same time.
I think that that becomes
essentially a chance
when the speakers' pitches are overlapping
or within about a third of an octave of each other.
When they're separated by a third of an octave
or more, then you start to get...
you can start to use FFT-based methods
to separate the voices more accurately.
Yeah, there's a lot to that.
There's another thing.
What I showed you
were standard compensatory responses.
Some people produce
what we call following responses.
And I remember in high school,
I was on stage
doing a performance
where I had to sing.
And if I was singing solo, it was okay.
But if I was singing
next to somebody else singing,
I would always sing their note.
I didn't have my own inborn capability
of singing without
drifting towards somebody else's pitch.
So I followed the pitch of other people.
I don't know what that's about either.
But some people do that more so than others.
So that's kind of the opposite direction.
It's destabilizing.
Compensation responses are stabilizing.
Following responses are destabilizing.
And this is something
we don't understand yet.
Tori?
No, I can't.
Stuttering is really complex.
It involves language.
It involves speech.
It could just be
that they're more sensitive to variations.
So it could be part of
just the sensitivity of the disparity there.
They may have
something going on
in the transfer to motor commands
that produces larger responses.
Or smaller.
I just don't know.
Yeah, I can't say what it might be yet.
There are relatively few groups of us working on this.
I know it's in the interest of yours.
I hope you can provide us with an answer eventually.
But I don't know.
Yeah, we did a subtraction
between shift and non-shift.
So it's vocalization.
It's vocalization.
Change between shift and non-shift.
Yes?
You mentioned during the very beginning
you mentioned that you're measuring the response
when the subject is vocalizing.
Right?
Yes.
So when the subject is producing the sound
then you let the subject to hear
its own voice through the earphones.
Actually, what the subject is hearing
two voices.
One is from the earphone
and one is from the bone transduction.
Right?
And the signal, the EEG signal you're measuring
is actually a signal response
from the voice through transduction
plus a signal, I mean the response
of the voice from the earphones.
How do you eliminate the response
to the voice through bone transduction?
That's a real problem.
We haven't been able to do it.
Nobody else has either.
There have been some attempts to modulate
the filter of the voice
more in line with what the bone conducted signal is like.
And they've shown the same results.
So that's an inherent flaw
that we just can't get around easily.
That is the bone conducted signal
has its own separate channel.
Now I will say that
if we do a study in which we
don't have the auditory feedback very loud,
then you can perceptually hear two voices,
the shifted and the non-shifted.
And that's because the side tone you're hearing
is loud enough to get through the earphones.
In that case, we don't get much of a response.
So that's why when we do these studies,
we amplify the signal going to the earphones
by 10 dB over what we're producing at the mouth.
And then that tends to mask the side tone.
It can't completely mask the bone conducted signal.
And so it's an inherent flaw in the paradigm.
But nevertheless, the results seem to suggest
that that is not very important.
Also, I have another question about the FMRI measure.
So you mentioned that because...
Well, so during the FMRI scanning,
we just cannot let the subject produce a voice.
So that caused you to...
You mentioned that the gateway of the two regions, right,
just cannot measure because it just disappears so fast.
So have you ever thought about using certain chemicals
to disable the person's muscle around the throat
and then let the person to virtually pretend that he's speaking
and then measure, scan his brain with the FMRI at the same time?
No, we haven't done that.
The closest we came is a study where we anesthetized the vocal folds.
That is, we put a tube down through the nose
and squirted xylocaine on the vocal folds
so that you can't feel them.
And when we did that and then did pitch shifting,
the signals were larger,
which tells us that to make this model complete,
this would be for the auditory system.
We would have an identical one for the somatosensory system.
And what we think is that the nervous system is looking at two inputs.
It's looking at auditory feedback,
but it's also looking at somatosensory feedback.
And so when we eliminate somatosensory feedback,
more weight is placed on auditory feedback.
But that's the closest we've come.
But we haven't done anything like that in the FMRI.
It would be a nice thing to do.
The only problem with that is to get good recordings
of the types that we've done in FMRI,
the subject might be in there for close to an hour,
vocalizing repetitively just to get enough signals.
And if we spray xylocaine on the vocal folds,
it's only going to last a few minutes.
So we can't do that experiment now the way we'd like to.
A better way might actually be to use PET scanning
where you make several injections onto the vocal folds
and do separate scans.
But we haven't been able to do that.
Also, do you think that it's...
Well, I'm just going to propose another solution.
Do you think it's possible if we replace the previous method I mentioned
with a pair of twins with identical voice patterns
and let one twin read a passage to the partner
and let the partner, who is under the FMRI scan,
present the paper of the passage in front of him,
just like the twin under the FMRI is pretending that he's reading?
No, I haven't.
That's a novel idea.
I guess these identical twins would have to have identical voices, too.
I think we're going to have an FMRI talk later today.
Maybe we can get some light on the possibilities
of doing an experiment like that.
It's kind of wild.
There's another question back there.
Yeah, I was just wondering the increased left hemisphere.
Yeah, that's what's been suggested as far as left hemisphere
musicians with perfect pitch,
that it does engage the language components of the left hemisphere.
I think that's a very good suggestion.
Our study just looks at early reflexive issues.
I don't know if it has anything to do with the language components or not.
Okay.
Thank you.
So now the organizers have scheduled a two-hour break
for a combination of lunch and the poster session.
For lunches, to get the lunch that you ordered,
you will note that on the back of your name tag is a colored dot,
and all you have to do is match up the colored dots
with the appropriate colored dots on the box.
Also, the seating for the lunch is actually in the center part of the atrium,
not in the part labeled the Beckman Cafe area.
It's in the part with basically the symposium logos.
The posters are in the east part of the atrium,
so please look at the posters from our students,
and also remember to sign up for tours of the labs in the afternoon.
I suppose if you wish, you can eat your lunches outside,
but I would remind you that the heat advisory begins at noon,
so eat them relatively quickly.
So we'll be back here at 1 o'clock.

