【AI Applications in Medical Imaging】Registration

MeDA
2粉絲
ChiChi Chang | Department of Bioengineering, UC Berkeley
#AIApplication #MedicalImaging #Rigistration #MeDA #MeDASchool
So an overview of what we're going to be talking about today, same structure as the other videos, we're going to talk about what exactly is registration in simple terms.
So what is registration? In simple terms, it's basically this workflow that we're seeing right now. What we see is that we have that image at the beginning.
So to combine all this knowledge that we talked about before, by using MRI and CT scan in order to register these two images together, this is exactly what it looks like.
So originally the CT scan was on green, I changed up the color a little bit so that you guys can get a better sense of what I'm talking about.
So it should align in some sort of way so that we're able to be like, hey, potentially this tumor is in this section, which is the same section as the other image, or maybe even off in the other image.
So there are two different types of registration that I would like to share with you.
The very first kind would be the intensity-based registration, and the second kind is the feature-based registration.
So one thing to note about the intensity-based is that it is comparing the intensities in specific positions.
The feature-based registration actually compares the positions of the original image to the other image.
So I would like to view this as a more overall.
So what ends up happening is that we're viewing the entire image during registration as opposed to a single pixel.
And so one thing to note is that as a summary of what registration is, it is determining the spatial alignment between images of the same or different subjects acquired with the same or different modalities.
So that's something that we're going to go into a little bit deeper in the remaining of the lecture, where registration can occur in multiple kinds of scenarios and in multiple kinds of situations.
So talking about the significance of registration.
So the applications and significance, as I stated before in that summary of what registration is, can occur in three different kinds of scenarios.
So the very first scenario would be the same subject, which is considered as intermodal, the phrase for that.
And basically what happens is that we may have multiple images acquired at different times.
And this may be in the morning or in the afternoon or at night, where we have these different scans that are taken at different times.
And due to a lot of biological reasons, as I'll explain later, there may be differences in the images.
So it won't be the same if we take it in the morning, in the afternoon and at night.
The images would not be the same.
And because of this, we would potentially need registration for the same subject or the same person.
Second scenario would be for different subjects.
And this is considered as intramodal.
And so for different subjects, it could be potentially a situation where you're comparing five different kinds of subjects or five different people who have brain tumors.
However, they may be in different locations.
So if you're looking at the images or you're trying to compare, maybe this tumor has a tendency to grow in a certain area of the brain.
What we end up having to do is that we can register all these five different subjects together or the images of all these five different subjects together in order to identify,
hey, maybe there is a specific region where this tumor actually grows.
So that's where registration is important in scenario two.
And so scenario three would be different modalities.
So this would be considered as intramodal as well.
And the different modalities were mentioned at the beginning where there were four different kinds of modalities.
And so this may be a situation or may be significant in a situation where a patient is being transferred from hospital A to hospital B.
And in this situation, maybe hospital A doesn't have the resources that hospital B has.
So for example, one of them only has MRI scans and the other only has a CT machine.
And so in this case, by having a registration process, we're able to register these two different images together as the patient is getting transferred from hospital A to hospital B.
So that the doctors are able to, I guess, in a sense, compare or analyze this patient's condition or the situation.
Maybe the scan was taken three months before or maybe it was taken six months.
And seeing that disease progression is very important for the physician in order to identify tumor types and the progression of treatments or the effectiveness of treatments.
So this is where registration may become important.
When we're talking about different modalities.
And so an intermodal situation, as I explained before, would be a situation where we have the exact same subject, but taken at different times.
And so in this situation, as you can see, these are three different images taken at different times, morning, afternoon and night.
And as you can see from the naked eye, we can't, there isn't a very obvious difference between all of these three images.
However, the truth is that there is a difference.
As you can see here in this section, it is very different from the afternoon section.
So that's something that it would be very hard for us as human beings to pick up immediately.
But for a computer, registering these two images is going to make it so much easier when we're overlapping these two things together and being like, hey, there's a difference in this area.
And you can think of this as one of those childhood games that we used to play.
When we would have multiple, we would have a lot of distractions and it would ask you, for example, to find a specific character.
Let's just say Mickey Mouse.
Let's find the number of Mickey Mouses that you would see in this area.
And potentially in this situation, registration would be comparing these two images together where most of the most of the image would be the same between these two different images.
But there would be slight differences in certain areas.
And by registering these two images together, you can identify the differences in the small areas that we're trying to look at.
Looking at a intramodal situation, we can see in this case, which is between different subjects.
And so one thing to know is that this isn't of the brain.
This is actually of the pancreas, but or the kidney.
Sorry, but one thing to know about this is the fact that these are five different kinds of patients.
With the same MRI scan.
However, as you can see by these masks, there are very, they're very, very, very different.
And for a doctor to be able to look at the difference between each of these kidneys would be very hard without registration.
So through scaling through rotation through a lot of all these different processes that we're going to talk about later.
It would make a physician's life a lot simpler when we're when they're trying to compare.
For example, where the tumor is growing.
Let's just say there's a tumor that was growing here.
Think of it as a more precise measurement.
So going on to the fundamental concepts of registration.
So the entire process is I would like to see it.
This is a lot simpler than the segmentation classification that we saw before.
In terms of this entire process, there is a image that comes in.
So I'm going to be talking about how the computer reads the image.
And then we're going to be talking about the resampling process.
And then the actual transformation process, which is where registration occurs.
So these are the very important steps that are part of registration.
And one thing that I'd like to know is that I didn't have interpolation as its own category because interpolation actually plays a role in both the resampling and the transformation process.
So how does the computer read the image?
So one thing to know is that while we're looking at these images, it is very simple, as I explained before, for the human to for the human eye or the human brain to see these images.
And immediately see the features that we have been trained to recognize.
So in this case, for example, we would see this as the eye.
So or this is the brain and this is the skull and this as potentially the nose.
However, for a computer, they don't read the images in this way.
As we saw with that elephant situation in the classification lecture.
A lot of the times the computer reads this and all they see is pixels.
All they see is numbers.
They're able.
They're not able to register or assign each of these pixel numbers to a feature that we would we as humans would actually recognize.
So reading the image is very different for a computer as it is for human beings.
As a result, a lot of the times this is in.
This is a diagram that is obtained from SK learn because this is how they usually read the images.
And what ends up happening is that as we input an image into the algorithm, it assigns that image to different values.
So first thing that they would do is to define the origin.
And then we would have the pixel width, for example, or pixel height.
And that would be noted in terms of the in terms of the spacing for each of these pixels.
And then we would have the direction.
So in which way is the image oriented?
On the axis that we're looking at and then going in deeper in deeper details.
What we see is that as we're talking about, this is spacing one and spacing two.
So the height and the length of each of these pixels.
And one thing to note is that this is the image origin.
So every single time a lot of times when we're looking at diagrams or grids, we tend to note or we tend to see that middle part where the two lines intersect.
As a point.
However, when we're reading images or when the computer is reading images, what it tends to do is that middle point of the spacing grid that we have defined.
So each of these middle points is what we're going to be looking at or is what is important to a computer.
And so one thing to note is that when we're going or when we're talking about interpolation, what it usually happens is in the region of of the pixel region.
So what we're talking about is in between each of these things.
And so for the pixel coverage, a lot of times it's the entire pixel.
So the box that we see, each of these are pixels and these are pixel centers.
But when we're talking about interpolation, it is between the spacing that is between the four neighboring pixels that we'll see or the four neighboring pixel coordinates or origins that we will be seeing.
And so one thing to know as we're reading the images is that there are two different kinds of images when it comes to registration.
So there is a moving and there is a fixed image.
So I'd like to refer to the fixed image as a reference image, something that the algorithm or the registration algorithm refers to every single time it is trying to register that moving image.
So as we can see in this diagram, what we tend to have is that moving image.
And it goes through these multiple transformation processes before becoming registered with that fixed image.
So what we would end up seeing is this original moving image being rotated and going through all these processes so that it ultimately ends up looking very similar or maybe even exactly the same thing as the fixed image or the reference image.
And so one thing to know as we're talking about resampling is let's kind of jog our memories back onto how we defined that grid that we were looking at the two slides earlier.
And so these are in coordinates of voxels, units of voxels.
So as we stated before in previous lectures, voxels are in terms of a 3D kind of sense of image analysis.
And so one thing to note is that we're looking at the pixel value, which is that middle point, and the pixel region as we're talking about resampling.
So this is just a redrawing our memories in terms of how resampling occurs.
So the way that I would like to see this is, for example, in two different situations.
So let's say that this is the fixed image and this is the moving image.
So looking at these two graphs, we can see that in terms of spacing, they're similar.
They're very, very similar in terms of shape, in terms of a lot of things.
However, in reality, it's actually very different as we can see on the left side.
They're very, very different when you're looking at them.
One is actually increases exponentially faster than the other, but it wasn't differentiable when we split them up into two.
So this is where resampling becomes important.
We're having them both or both of the images on the same scaling, on the same grid, so that they can become more comparable, so that registration is a lot easier for the entire algorithm itself.
And so going on briefly to talking about transformation, for the sake of our purposes today, we're only going to be focusing on linear transformations, which include rotation, shearing, translation and scaling.
And you can see what each of these transformations mean by comparing it to the original structure, which is the square.
And so one thing that I would like to ask you guys after everything that we've talked about is what kind of transformations are needed and why.
So looking at these images, which is a patient with early signs of ischemia,
what kinds of transformations do you guys think you would need in order to compare these three images,
which is something that I'm hoping that you guys remember, which is the same patient but different times.
So honestly, there's no right answer to this.
You can compare it in any single way.
So we can say shearing, we can say rotation, we can say scaling, we can say anything, basically.
There is no right or wrong answer with our own eyes.
So with the human eyes, it is very biased because what we may see may be different from what someone else may see.
So I may say like shearing or scaling, but someone else may say rotation and shearing.
So when humans compare these images, it is very easy for it to become biased when we're trying to register.
And so this is why it's so important for the computer to come in where they have a set algorithm or a set rule.
You can think of it as a set, a number of rules where they go through a number of steps that they go through
that becomes more standardized for these images to become more registered.
So one computer can help us decide on everything together.
So talking about interpolation, it includes three different kinds that we usually see.
So what exactly is interpolation?
I would like to think of it as a type of estimation.
So interpolation basically allows us, it is a method of constructing new data points from our discrete known set of points.
So in this case, it may be, for example, if we had a thousand pixels on one of the images, such as the reference image had a thousand pixels
and the moving image only had 500.
What we would need is interpolation so that we're able to create an additional point between the two points for that 500 pixel image.
So that we can create a thousand pixels so that these two images can be registered together or can be compared together
as we saw in resampling where we need it on the same grid.
That's the entire process or the entire need or function of this interpolation method.
So going on to talking about nearest neighbor, which is the simplest interpolation method that we have ever come across.
And the concept behind it is determining the nearest neighboring pixel.
So the way that I would like to see it as if you're saying, imagine yourself, close your eyes.
Imagine yourself standing in a room where there are four corners to that room very distinctively.
And imagine yourself standing near the door, which is to the left side of the room.
So bottom left side, if we're looking from above.
And so what we would see with this bird's eye view is the fact that if we're standing near the door, which is on the bottom left,
what the nearest neighbor algorithm does is that it interprets your point as that door point.
So we would define ourselves as that point without consideration of anything else.
So this is why I told you this is the simplest thing.
Wherever I'm closest to, that would be the point that I would define myself as.
Every other three other corners in the room is useless to this algorithm.
So the advantages of this concept or of this algorithm is the fact that it is the shortest amount of calculation time.
And it is great for categorical data.
So you can imagine categorical data as water, drunk food, and let's say healthy foods.
And in this situation, it is very easy.
It doesn't need to consider anything else.
No diet, no genetics, no other factors.
Drunk food, you can imagine Oreos.
Go straight into drunk food.
It's very easy to split up without any other consideration or weighting system.
It is very quick and just very...
It's a very good decision process for categorical data.
The disadvantages to this is the fact that it ends up with pixelated image.
And it is very blocky when you're using it on continuous data as we will see later in a visualization process.
So this makes sense because we're not considering any other parts of the room except for the closest corner.
And so that's why we end up with a lot of pixelated images a lot of times.
There's no consideration of any gray numbers or any numbers between 0 and 1.
It's either 0 or 1.
No decimal point.
And so this is exactly what I was talking about before.
And so just to summarize what I was talking about for nearest neighbor.
Imagine yourself on the bottom right corner in this case.
And so if you're closer to D, what nearest neighbor does is that it doesn't consider A, B or C at all.
All it considers is D because you're closest to that.
And it immediately categorizes you into D.
Nowhere near A, B or C.
Bilinear interpolation is a better method.
Or you can think of it as a step-up method from nearest neighbor.
So the concept of this is that you have weighted averages of the four nearest cell centers.
Closer points to P would have higher values.
So in the case of the room as we were talking about before with the A, B, C and D.
If you're closer to D, then that means that the D would have a higher weight or higher value since it is the closest corner to P.
And then obviously the B area, which is the furthest away, which is diagonally from the nearest neighbor.
Which is the furthest away, which is diagonally from where you're standing right now.
Would have the lowest value because it is the furthest away from your point right now.
So that would have the lowest weighting.
And what you end up having is adding up all these values.
You're able to determine your spacing relative to all four points.
Or all four corners of the room.
So the advantage of this is the fact that it is good for continuous data.
And the output can be different from the input.
But it remains within that input range that we originally had.
Which we will show you or you will see in the visualization later.
And the disadvantages to this is the fact that it's not recommended for categorical data.
And you can think of this as if you're trying to categorize yourself.
If going back to that Oreo situation.
Some people may argue Oreo is actually very healthy.
It's not as high in sugar or whatever.
Some people need sugar.
And maybe say they may categorize that as healthy data.
And so in that situation we would know that.
It's very difficult for us to have all these different weightings.
For us to be able to determine.
Something that can be clearly or very easily categorized into one section.
Without further considerations of other factors.
And another disadvantage to this bilinear interpolation.
Is the fact that it needs a larger amount of calculation.
So going on to the cubic convolutional method.
I would say that this is a step up.
A further step up to bilinear interpolation.
And the concept of this.
Is the fact that you're finding 16 nearest cell centers.
Instead of 4 to the output.
And fits a smooth curve through all the points to find the value.
However the advantages of this.
Is the fact that it's really good for continuous and smooth data.
However the curve can cause a lot of outputs.
To lie outside of the range of input values.
Or the input range.
And one thing to note about this.
Is the fact that it is also very time consuming to compute.
So since there's a lot more data going in.
There's a lot more computational stuff that it needs to go through.
So a lot of times we wouldn't see this method used as much.
So going on to talking about the visualization.
As I was talking about before.
There's a reason why the images become more pixelated.
As we can see in this situation.
It is blocky because you're taking a step up from each step.
So there's no data in between these two.
So as I said before this may be 0 and 1.
There's no data between that 0 and 1.
So there would be some kind of unknown data.
Between these two.
Which makes the entire image more pixelated.
On the other hand.
When it comes to bilinear interpolation.
What we meant by the smooth or the output range.
Becoming or staying within that input range.
Is you can see this line here.
So these two lines are at different heights.
And by combining or connecting.
These two different linear interpolations.
We can see a smooth line.
That is descending from the higher to lower.
So as you can see.
It's staying within that high to low range.
So any of the outputs are staying within.
However with the cubic convolution.
As we can see here.
What ends up happening is that you have that curve.
That goes up and then down.
So that upward motion that you see here.
Is already going outside of the input range.
That we originally had.
And that was the disadvantage to cubic convolution.
So going on to the mathematical concepts of registration.
So the entire pipeline.
Or the workflow of registration.
As I would like to explain here.
Is that we start off with the preprocessing.
So if the image needs preprocessing.
It would go through that process.
And having that preprocessed image.
We would go through that entire resampling process.
That we talked about before.
Before going into the entire transformation process.
Which is the very first part.
Where we're finding the transformation function.
So that would consist of the transformation matrix.
It would consist of the metrics.
The interpolator.
And the optimizer.
Which we'll explain in this section.
In further details.
And applying that transformation function.
To the original moving image.
We would have this outputted registered image.
So to reiterate.
What exactly is resampling?
And the answer is already given to you guys.
But just a reminder.
It is changing the number of pixels in the image.
So as we explained before.
That 5,000 to 1,000 kind of idea or concept.
That's what resampling is.
We're trying to make sure.
That the number of pixels of the moving image.
Is going to be the same as the reference image.
So that's easier for us to be able to.
Register the two images together.
So you're aligning.
You can think of it as aligning the pixels.
Or the points together.
In order to get our registered image.
However if we have different numbers of pixels.
It's going to be a lot harder.
To be able to register the two images together.
Because there are no points that exist in between.
Or to compare it with.
So in terms of transformation.
This is the mathematical.
This is a mathematical equation.
And so this is the transformation function.
That we talked about before.
And as seen here.
The A is the operation matrix.
X is that moving image that we were talking about.
C is the center point that we have defined.
And T is the translational vector.
So using this.
We would be able to find that transformation function.
Where we're able to put in our moving image.
In order to get our registered image.
And so an example of transformation.
As I would like to talk about again.
Is imagine we had this image at the beginning.
And so what ends up happening.
Is that it's not a direct method.
Of just like having two things overlap.
On top of each other.
So let's have a very good example right now.
So everyone close your eyes.
And put two hands out.
And what I'd like you guys to do.
Is to overlap them on top of each other like this.
And with each of the overlap.
With your eyes closed.
I'd like to see.
I'd like you guys to open your eyes.
And see how close.
Your two hands were on top of each other.
And so what you would learn.
Is that I'm hoping.
With each iteration.
Or with each try that you guys do take.
You would realize that you get closer.
And closer to two hands.
Exactly being overlapping on top of each other.
And so that's the entire.
Or it's the exact same.
Kind of situation.
When it comes to registration.
Or when we're trying to register.
A moving image.
The reference image.
So the bottom hand.
That we originally had.
When we're doing that hand situation.
Would be our reference.
And the top would be the moving.
And so.
It goes through each of these iterations.
So it would move from like.
Side to side to side.
Until it finally has these two images.
That are overlapping each other.
And that's when you know.
Registration has happened.
Same thing as your hand situation.
And I'd recommend you guys to.
Kind of do it yourself a little bit.
To understand what I'm talking about.
When we're talking about the intuition.
Behind registration.
So.
Talking about the mutual.
Information concept.
So.
I'd like to.
Kind of dive in a little bit deeper.
Into the mathematical concepts.
Behind registration.
Conceptually.
So mutual information.
Is part of information theory.
So one thing that I'd like you guys.
To remember is that.
Entropy in this case.
Means measure of information.
So in this case.
It doesn't mean.
That thermodynamic law.
That we've all learned.
In this case.
It means the measure of information.
So mutual information.
Allows us to learn.
How much one.
Random variable tells us.
About the other.
And so it helps us.
Reduce our uncertainty.
About one random variable.
Given the information.
Or knowledge about the other.
So what this actually means is.
Let's look at this.
Diagram on the side.
So having.
Let's just say.
We had this original.
This original image on the left.
Or this original h of x.
And so moving that h of y.
By moving it closer together.
We would know more and more.
About the right random variable.
For example.
By overlapping it.
So this mutual information.
In the middle.
Is where we're able to determine.
The two things.
So let's take colors.
For example.
Yellow and blue.
So we all know.
Adding these two colors.
Makes green right.
So imagine.
If we originally had blue.
As our fixed color.
And we're trying to add.
The yellow in.
To create the green.
So you would be starting off.
With a deeper green.
To begin with right.
So the more yellow.
That you add.
The more.
The greener.
Or the lighter green.
You get.
And in this situation.
We can apply that.
To mutual information.
Where we know that.
The more yellow.
That we're adding.
The more lighter green.
That we're getting.
So we know more.
About the blue and the yellow.
Or we know more.
About the blue.
Depending on how much yellow.
We're adding.
And the output.
That we're getting.
So the mutual information.
That we're getting.
So that's the entire concept.
Of mutual information.
So one question.
That I'd like to ask you guys.
Is if we have mutual.
If we have high mutual information.
Will our uncertainty.
About another random variable.
Increase or decrease.
So the answer is already.
Actually there.
But what I.
It would actually decrease.
And I would like you guys.
To think about why.
Depending or judging.
By what we talked about before.
So going back to it.
Going back to that.
Blue and yellow situation.
If we have more green.
Or if we have more information.
About that green.
Then we can tell more.
About how much yellow.
We added.
Or what kinds of yellow.
We added.
Into that blue.
If you think about it.
If we have a more orange.
Yellow.
The green may be different.
From a more yellow.
Kind of yellow.
So judging by the output.
Or the mutual.
Or the information.
That we get from the green.
We're able to determine more.
About the yellow.
That we have inputted.
Or the unknown yellow.
That we have inputted.
Into the.
Or that we've added.
To the blue color.
So applying this entire.
Mutual information.
To registration.
The things that.
I would like to know.
Is the fact that.
The applied theory.
Is basically comparing.
The statistical dependence.
Between the two images.
So looking at the image.
Of the fixed image.
And the moving image.
By applying.
By looking at these two images.
We're able to determine.
The mutual information.
Between the two.
The use.
And the purpose of this.
Is the fact that we're maximizing.
The image similarity function.
So one thing to know.
Is that's not going to give us.
The best.
Or it's not going to give us.
A perfect.
Kind of minimization.
But what we're.
What the algorithm does.
Is that it ensures.
That we're able to get.
The optimal.
Amount of information.
That overlaps.
But definitely not the perfect.
Kind of information.
That overlaps.
And the advantage of this.
Is the fact that.
It is very adaptable.
To changes in lighting.
And changes between sensors.
So it makes a lot of assumptions.
Prior assumptions.
So that this algorithm.
Can work.
So it doesn't.
It's not dependent on.
For example the machine.
Or the MRI machine.
So talking about.
The joint histogram.
Of image registering.
So I'd like you guys.
To take a look at.
These four images on top.
So which one.
A question for you guys right now.
Is which image do you think.
Would represent two images.
Being perfectly registered.
Do you think it's the most.
The left most.
Or the right most.
So yeah.
I would agree.
If you guys got this answer.
This image actually has.
The most registered.
Components to it.
And you can see here.
This is the dispersion.
Of clustering increasing.
So what we see is that.
There is more and more clusters.
Or more and more gray areas.
That we see outside of that line.
Which actually ultimately.
Represents the two images.
Being further and further apart.
So there is more differences.
Between these two images.
That are on top of each other.
So we see that dispersion.
Of clustering that exists here.
Whereas in this line situation.
We can see the two colors.
As the blue and the yellow.
Adding up together.
So in this situation.
You would still see a little bit.
Of the blue and a little bit.
Of the yellow on the sides.
If we have the two circles.
Coming together.
However in this line situation.
You can see that.
That blue and yellow circle.
Completely on top of each other.
Creating that green color.
So going on to talk about.
The mutual information.
So the mathematics behind it.
This is basically what we see.
So this is a very.
More or less complicated.
Complicated kind of concept.
Or mathematical idea.
And one thing that I'd like to know.
Or two things that I'd like to know.
Is the fact that X and Y.
Actually represent.
The discrete random variables.
That we talked about.
So the blue and the green.
Or the blue and the yellow.
Would represent the X and Y.
In this case.
And what mutual information does.
Is that it quantifies.
The amount of information.
Obtained about the random variable.
Through observing.
The other random variable.
So one thing that I'd like.
To you guys to remember is.
We know that if these two.
Random variables are independent.
Then we would know.
That the log of this.
Would actually equal to zero.
So that's something.
To keep in mind of.
So going back.
And emphasizing the whole idea.
Of image registration.
And mutual information again.
To confirm that our images.
Have been registered.
A lot of times.
We would look at these.
Pixelated images.
So do you think the left side.
Or the right side.
Has been more or less registered.
So which one do you think.
Has a higher registration.
Or a better registration.
Output.
So yeah.
I would agree with you.
This image.
Is a lot clearer.
In terms of the colors.
That we see in the clarity.
Whereas this one.
Has a lot more gray scale to it.
So you see these.
Different kinds of grays.
And whites and blacks in there.
Which shows that the image.
Was not as well registered.
As this image.
Which is very very clear.
In terms of clarity.
And where for example.
Objects are.
So if this were an apple.
Let's just imagine.
These two images.
Both being apples.
What you would see in this case.
Is a massive red area.
In that middle.
To represent the shape.
Of the apple.
That you'd be able to recognize.
Whereas in this situation.
You may have a little bit of pink.
On the outside.
And on both sides.
Of that apple shape.
For you to not actually.
Be able to distinctly.
Recognize it.
As a shape of an apple.
And so talking about.
The interpolator.
Which is in our case.
We use bilinear interpolation.
So this is the mathematics.
Behind bilinear interpolation.
Is very simple.
In terms of how you just.
Set the coordinates.
Or you.
You have these coordinates.
That you have.
You put it into.
These function values.
So you have.
Multiple functions.
That are defined by.
Each of these points.
In order for you.
To be able to determine.
Where that exact point is.
Relative to the other points.
That exist.
Using these equations.
So going on.
To talking about gradient descent.
As we talked about before.
Gradient descent is.
The concept is that.
It is an optimizer.
And so it's an iterative.
Process.
Where each iteration.
Actually refines.
The parameter values.
For the computation.
So that it provides.
A best parameter for alignment.
And in the case of classification.
As we saw before.
It is that back propagation.
Method where it is.
It uses gradient descent.
In order to find.
The most accurate weighting.
To represent each of these nodes.
In this case.
It is going through.
That iterative process.
To find the parameters.
That allows us.
To be able to achieve.
The best alignment.
Between two images.
And so in this case.
For our optimizer.
As I stated before.
We're using gradient descent.
And it is an iterative process.
That allows us to find the minimum.
Or the steepest part of the slope.
As we'll see in the next slide.
So gradient descent.
Mathematically refers to this thing.
As we see here.
So this is the updated weight.
So I'm pretty sure.
As you guys remember.
From that segmentation lecture.
What we have is.
The constant updated weight.
So this is the new origin.
Or the new image.
That we talked about.
In the segmentation lecture.
And so this is the current weight.
In terms of where we are.
So this becomes edited.
Every single iteration.
And the learning rate.
Basically defines where.
Or how much we're learning.
With each iterative step.
And that's the gradient.
Or the gradient of the loss function.
At the current weight that we're at.
So as seen before.
In the segmentation lecture.
I would recommend you guys.
To look back to it.
If you don't remember.
Or if you haven't watched it.
What ends up happening.
Is that we're trying to find.
That steepest part.
In each of our functions.
And so in this case.
If we're starting at point A.
What we're trying to find.
Is that very very minimum point.
Through gradient descent.
So through this iterative process.
So one thing to know.
Or to remember.
Is that it gradually.
Updates the weight.
By stepping into the steepest direction.
To reduce the loss function.
So remember it is.
Constantly updating the weight.
In order to achieve minimum.
Always minimum.
That's what I like to remember.
With gradient descent.
So the multimodal gradient descent.
Is actually an improvement.
Of the gradient descent.
That we were talking about before.
And the whole idea behind this.
Is the fact that.
In a very simple function.
As you see here.
It is very easy for us to determine.
Where the minimum is.
It's going to be.
At that minimum point.
Of the concave.
Function.
However.
In a situation here.
Where there may be.
Multiple dips that we see.
It is very easy for.
A function or the algorithm.
Through gradient descent.
To define this point.
As the minimum.
As the minimum point.
Because a lot of times.
As we stated before.
What ends up happening.
Is that as long as there is.
Some sort of increase.
What the gradient descent.
Or what the algorithm does.
Is it goes back.
Takes a step back.
To that very minimum point.
Or that last point.
Where it was a minimum.
And defines this minimum point.
As the very very minimum.
Of the entire function.
However.
As I stated before.
In this function.
You see that the best minimum.
Is actually here.
So what.
Multimodal gradient descent.
Allows us to do.
Is to be able to determine.
This best.
This best minimum.
Without stopping.
At a local minimum.
So as a summary.
Of the entire registration process.
What I'd like to see.
Is that.
What we would have.
That go through.
The workflow.
Is we would have.
The fixed image.
And the moving image.
That basically goes through.
And we have that metric.
And it goes through.
That training process.
Of continuously registering.
So think of your arms.
In the same way.
Where you go through.
That iterative process.
To be able to find.
The optimal transformation function.
For us to be able to.
Have our moving image.
Become registered.
To our reference image.
Thank you.
MING PAO CANADA // MING PAO TORONTO
