Matrix Multiplication Actually Makes Sense (Here’s Why)

Math for our world
17 followers
Have you ever wondered why we multiply matrices the weird way we do?
Why not just row-to-row or column-to-column? And honestly, why is it even called multiplication
when it feels like there is just so much addition going on? It seems bizarre, until you zoom out
and see what matrix multiplication is actually doing. In this video we'll break down why that
transpose-then-dot method makes perfect sense and how matrix multiplication is really a natural
extension of the regular multiplication you learned in elementary school. This video will
help you deeply understand matrix multiplication in ways that I bet you missed in your regular
math classes. If your experience learning about matrix multiplication was like mine,
then your teacher had to work hard just for you to learn how to multiply two matrices,
and they didn't give any explanation about why we multiply matrices that way. Sadly,
this pattern of teaching how, without the why, is common in many math classes.
When we touch matrix multiplication in contexts, both real world and mathematical,
very good reasons emerge on why we multiply matrices the way that we do.
But first, let's take a closer look at dot products. After all, in our earlier video on
dot products we never discussed why that operation should be considered a product at all, and
understanding that will help us understand why matrix multiplication should also be considered
a product. As a quick reminder, the dot product of two vectors is the sum of the product of each
corresponding entry and looks like this. You might feel uncomfortable with this process being called
a product because of the additions. It seems like more of a sum than a product. But recall,
back when you were eight learning about multiplication, that the first interpretation
of multiplication your parents or teacher taught you was that multiplication was repeated addition.
If we rewrite the dot product of these two vectors just as a sum, then we can see some
repeated additions in here, and we can shorten that expression by grouping the addends with
the same values and writing those as products. So what we have here are groups of repeated additions.
To try to write this as one product, we need to list the separate values and how many times each
of these values are repeated. And that is what we have done with our two vectors, which represent
these two lists. In other words, we can think of one vector as the size of the different groups,
and the other vector as the number of groups of those sizes. That still might not convince you
that the dot product operation should be considered a product, so here's another reason. Suppose I go
to the store, buy three treats for two dollars each. If I multiply those two values, then I get
six dollars as the total cost. The structure is the price times the number of items equals the
total cost. Well, what if I bought four different items and had multiple of some of these items?
How would you calculate the total cost? You'd probably multiply the cost of each item by the
number of that item and add the resulting products up for a total cost. Well, that is exactly the
dot product operation. If we represent the prices of items in one vector, the number of each item
in another vector, then it makes sense to call the operation that we do to those to get to the
total cost as a product or multiplication, because it has the same price times the number equals
total cost structure as the simple case. In fact, this is the operation that ends up on your receipt
when you buy something at the store, plus those pesky taxes, of course. What we've learned is that
it makes sense to think of the linear combination of a set of numbers as a product between the
vector values and a vector of scalars on those values. Without a context, it doesn't matter which
one we consider as the values and the scalars. So how does this relate to matrix multiplication?
Let's use a context we've used before, nutritional information of foods. Like the simple price times
number of items equals total cost example, the calculation of total nutrients are the same.
The grams of fat per serving times the number of servings will equal the total grams of fat.
But with food, fat is not the only nutrient we might care about. We might want to know the total
amount of protein and carbs as well. Here are the grams per serving for a serving of bread,
and if I eat two servings of bread, then I get double of each of these nutrients per serving.
We can represent this as a scalar multiplication of a vector. If I eat three servings of jam,
I can find the total nutrients in the same way, the scalar times the jam vector. Now what if I
have a peanut butter sandwich with two slices of bread, one serving of peanut butter, and three of
jam? What is my total nutrient intake of protein, fat, and carbs? It would be the sum of the scaled
versions of each of the nutrient vectors, and this is what we get as the result. But this looks
awfully familiar to what led us to the dot product, a sum of products. But this time we have products
that are scalars and vectors, but we could rewrite our calculation as just a sum with some repeats.
Using the same strategy as we did with the dot product, we could put our scalars in one vector
and the vectors into another vector, which would make a vector of vectors,
or packet of packets if you like. A vector of vectors, or a packet of packets, is a matrix.
If we put our three vectors into a matrix, we get a three by three matrix. If we put our scalars
into a vector, we have a one by three or a three by one vector. But there are two different three
by three matrices we could make, one being the transpose of the other. So now we have a new
problem to figure out. Which vectors make sense to multiply with which matrices? Before we dive
into that problem, notice how the units hint at multiplication. We have a matrix of grams per
serving, a vector of servings, and we want total grams. So calculating the total grams for each
nutrient, protein, fat, and carbs, boils down to taking three simple dot products.
OK, we're still left with the questions, how should we define the product of a matrix and
a vector, and should it matter how we write the matrix and the vector? When units are involved,
they help us guide us towards the right operation and tell us what the new vector should look like.
For example, we know that the number of servings of bread should align with the bread vector to
make sure the quantities match correctly. As a note, I'm going to focus on the multiplication
of a matrix and a vector with the vector on the right, since that is more common. One helpful
strategy is to think about the vector we want to get after the multiplication. We need total grams
of protein, fat, and carbs. It makes sense then that we should set up the final product vector
to align with the rows or columns of the matrix that correspond to protein, fat, and carbs. So if
we write the matrix this way, then the final product vector should be this way. Similarly,
if we write it this way, the product should look like this. What we are doing when we change the
orientation of the matrix and the vector is an operation called the transpose. So the two
orientations are an original matrix and its transpose. It doesn't matter which one we define
to be the original matrix, the two are still the original and its transpose. Notice that if we
change the number of items in our meal, then we don't have a square matrix, and it still makes
sense to keep the orientations of the matrix and the final vector this way. How should we orient
the last vector in our equation? Based on our argument to match up orientations of units, we
should do it this way. But let's expand the vector to the right, which are the changes of my servings
my wife and my three youngest who are still at home. A quick side note for clarification,
if these are the changes of these three foods in our diet, we can have negative values if we're
already eating some servings of that food, and that negative value could indicate removing servings
from our current diet. It makes sense to stack our serving vectors side by side, creating either a
three by five or five by three matrix. To figure out the right orientation, let's think about the
final result. We want a matrix showing the total changes in protein, fat, and carbs for each of the
five individuals. Notice the unit flow. The first matrix has grams of protein, fat, and carbs per
serving of peanut butter, bread, and jam. The second matrix shows servings of peanut butter,
bread, and jam per individual. The final matrix should show total protein, fat, and carb changes
per person. So protein, fat, and carbs should run one way, and individuals should run the other.
To keep the units lined up properly, we want rows for the protein, fat, and carbs,
and columns for individuals, meaning the second matrix should have individuals as columns. This
setup made the most sense to mathematicians, so matrix multiplication was defined this way,
but we'll show another big reason soon. Some teachers even lay out the matrices with the
second one stacked above the final product to emphasize this. Each entry in the final matrix
is the dot product of a row from the first matrix and a column from the second. For example, the
entry in row 2, column 4, comes from the second row of the first matrix and the fourth column of
the second. This algorithm for matrix multiplication does demand that one of the vectors is turned 90
degrees to match up with a vector in the other matrix. We could try orienting the second matrix
differently to avoid this, but then it's unclear how the final matrix should be organized, and it
becomes very messy figuring out where each dot product belongs. We could still force it by
making up a rule like, always turn rows of the second matrix into columns in the final matrix,
or vice versa. But either way, rotation still happens, and now it's completely arbitrary.
So bottom line, no matter how we define matrix multiplication, it demands a rotation somewhere
in the process. Other contexts also point towards defining multiplication the way we've developed.
Some matrix vector products create an outcome with the same units as the original vector.
For example, in population modeling, matrix multiplication is used to estimate future
populations. For example, this simple matrix can be used to estimate owl and woodrat populations
in a region. Owls in that region largely depend on woodrats to survive, so their populations are
closely connected. The units of the population vector is in individual owls and thousands of
woodrats. Given a population vector, we multiply this matrix to predict the next month's population
of both. If we wanted to know what the population is two months down the road, we multiply by our
matrix again. But we should be able to predict the population two months down the road with
one vector. Instead of multiplying it this way, with a vector times a matrix, and then times the
matrix, we should be able to multiply it this way, multiplying the two matrices together, and then
multiply that by the vector, and we should get the same answer. We could even write it this way,
as the matrix squared times the vector. The way we've defined the matrix multiplication gives us
the same answer no matter which order we do the multiplication, so our definition of matrix
multiplication gives it the associative property. But the way we define matrix multiplication also
gives us the same result of the two months matrix, whether we square this matrix or create the two
months matrix from scratch. This is another strong reason to define matrix multiplication the way
that we have. In this video, I've emphasized multiplication with the vector on the right-hand
side of the matrix, but by using the same reasoning we did before, we can see that to multiply vector
on the left, we transpose the vector into row vector, and transpose the matrix, and end up with
the row vector. Both ways are valid, but it does mean we need to be careful with orientation of
vectors in products. We end up with the same information at the end, just with a transposed
vector or matrix. So matrix multiplication isn't just arbitrary, or just some weird set of rules,
it's a carefully designed system that fits real world problems. By understanding how it works,
and why it's set up this way, we're now better equipped to tackle systems of equations,
best approximation problems, and various other problems in linear algebra. Stick around,
because in our next videos we'll dive even deeper into the power of matrices,
and unlock even more of the math hidden behind the scenes.
Thank you for watching. Please subscribe and share our videos. Be sure to follow Math
the World on Twitter, Instagram, and Facebook. Thank you so much for your support!
