13. The power of NumPy
Oct 8, 2023
NumPy is arguably the most important library for python. It's built for scientific computing. It allows us to store data in arrays and perform operations and manipulations of those arrays at incredible speeds. Numpy also has many math functions like the trig functions, log, exp, etc. This video explores some of the options NumPy offers and shows a few examples along the way.
In this video, we're going to tackle NumPy.
NumPy is a library in Python.
It's probably one of the most important libraries of all.
In fact, it's so great that I saw this two months ago.
Nature, a really prestigious journal
in science and engineering, published an article
all about NumPy and all about the things
that it's made possible, like the discovery
of gravitational waves recently.
So NumPy is a big deal.
It's an extremely useful tool, and it should be something
that you know how to use as an engineer in material science.
So let's dive into it.
OK.
So first off, what is it?
It is a numerical Python package.
It's about doing math, right?
It has a focus on doing math and activities with arrays.
And arrays are sort of like the lists
that we've learned about previously, right?
It's a way of storing data in sequential amounts, right?
The big difference is that instead
of using lists and loops, which we've
been doing the last couple of videos,
it can do it about 50 times faster
using the NumPy architecture.
So it's worth learning, because that's a whole lot faster.
To use NumPy, you have to import that library.
With the Anaconda 3 installation that we did,
it already came with the library.
But that still means you have to bring it
into your specific module where you're going to use it.
So we're going to do that by typing import NumPy as np.
That is the np.
That's the standard way to do it.
You wouldn't ever name it something else,
because this is what everyone calls it.
Then you can do lots of stuff, like we can build an array.
Just like we built in a list before, we can make an array.
Let's just call it array is equal to,
we're going to call it a NumPy array.
Now, it's a function, so it has the round brackets.
But inside of it, we put the array as if we normally would.
So we could do 0, 3, 5, just like so.
When we run it, we get the following.
Here it's now, it's an array of integers.
We could have also done other things other than just
integers.
We could do floats, no problem.
And it is now an array of floats.
No big deal.
Check this out.
This visualizer in Spyder does a really good job
of showing values.
So it'll make your highest values.
It color codes them with a heat map from red to blue,
which makes it really easy to see
where your really high and low points are
as you get into these arrays.
So it's just a nice visualization tool to use.
What else can we do?
It can create something called linear spaced array.
We use this all the time.
This is what you want to use when
you want to create an array between some start
and stop point with equally spaced elements.
So over here, let's give this a shot.
So we're going to say array lens based.
It's going to be a non-pi array.
We're going to do lens space, the function lens space.
We have to give it a start and a stop,
and then how many we want to include.
So let's go from 0 to 10.
And we'll go in increment.
Well, we want five, five different elements
in that array.
Now when we run this, you'll notice the following.
It goes from 0 to 10, meaning that 10, the stop point,
must have been inclusive, meaning it included it
when it made its five splits.
And then 1, 2, 3, 4, 5.
And sure enough, it's got five elements in there.
Well, that's one way to do it.
But another common way that we want our array
is just to give it some spacing.
We know the step size from one number
all the way up to a higher number.
So we can do that with another function.
We're going to use the arange function.
So let's give that one a shot.
So this is going to be array, and we'll call it step size.
This one's going to be equal to non-pi.
And then we're going to arange.
And again, we're going to do start.
So let's go from 0 to 10.
And then we'll do step size of 2.
The difference here is that in linspace,
the stop point was inclusive.
It's exclusive here.
So when we run this, I'll show you what I mean by that.
The step size function does not include 10.
As we went from 0, 2, 4, 6, 8, it doesn't include 10.
So that's just something to be aware with that.
So we can create an array full of random numbers.
We can create, first off, we've been doing arrays
as one dimensional.
But you can do higher dimension arrays.
So let's do an example of a higher dimension array.
Let's do a 2 by 2.
So we'll call this array 2D.
So we're going to do numpy, and we're
going to do an array command, just like before.
But this time, inside of here, we're
going to put square brackets, meaning a list is coming in.
And we're going to do, let's make it a 2 by 2.
So in each one of these, we're going to have two values.
So 1, 2.
And then over here, we're going to put 3 and 4.
When we run this, we're now going to have a new array 2D.
And you can see, 1, 2, 3, 4.
So it made a 2 by 2 array.
What if you wanted something that was like two rows,
but four columns?
No problem.
We can do that.
We can make it two rows and four columns, right?
So let's make this 1, 2, 3, 4.
And now we're going to make this 5, 7, 8.
When we run that, you now see that this
is two rows and four columns.
So the nomenclature is rows and then columns, all right?
So that is how you can make higher dimensional arrays.
Let's make an array filled with random numbers.
Let's say we're going to use it later on.
So that's pretty easy to do.
We're going to come over here.
So this is full of random.
We're going to do numpy random.
Then we're going to do randint.
Now, the function in random integer,
we need to tell it the bounds, right?
A random number between some start and some stop.
The stop is exclusive.
So if we go from 0 to 100, it will never actually include 100.
It'll go up to 99.
Then we need to tell it the size of our array, right?
So you could make it, if you just leave out array,
what's going to happen, right?
Let's try this one.
So we try it.
We get right here.
It's just a single integer, right?
Because we didn't give it a size of an array.
But we can give it a size, right?
Let's say that the size, we want this to be a,
let's make this a 2 by 3.
So that's going to be two rows, three columns,
filled with those random numbers.
Let's run it.
And here's our random array.
Sure enough, it's two rows, three columns,
filled with random numbers.
And again, you can see that nice color coding.
The lower the number, the more dark red.
The higher the number, the more dark blue.
And then different values in between, OK?
So that is how you can do those.
It's possible to concatenate arrays.
Remember in Python, when we were learning about strings,
if we had string 1, string 1 was equal to cat,
string 2 was equal to fish.
We can make string 3 just by adding those together.
We could say string 1 plus string 2.
And when we run it, this will now be catfish.
It just adds them directly together.
If you do that, if you try that same thing here,
we're going to run into an issue.
First off, let's create a couple of arrays.
We've got one array that's random, 2 by 3.
Let's do this for another one and try adding them together
and see what happens.
So we're going to make two random arrays,
doing them the exact same approach.
And now we're going to try and just add those together.
Let's see what happens.
Array underscore random 3 is going to be equal to.
What happens when we run that?
OK, random array 3, if we look at it, what has it done?
Well, it's 109, 109, 134.
Let's pull up these other ones.
So here's array 2.
Here's array 1.
So hopefully you can see with this,
they've just added these boxes together.
So at the 0, 0 indices, 97 plus 12 is 109.
56 plus 53 is 109.
68 plus 66 is 134.
So they've just added these together directly.
So that's something that you can do in NumPy.
But what if we wanted to concatenate them?
Well, we can do that.
We can define it like this.
We can say array 3, array random 3,
is now going to be NumPy concatenate.
We're going to concatenate.
We can tell it the axis that we want it to do it on.
Let's go ahead and run that.
OK, sure enough, now we've got random array 3.
And by doing axis 1, that took our 2 by 3 one over here
and our 2 by 3 matrix over there,
and it added them along the columns.
If we wanted to add along the rows, well, we could do that.
We could just change our axis from 1 to 0.
And you can see in the documentation
what these would have been.
So let's run it now.
And we take a look at our array of random numbers now.
Here's our first 2 by 3, and here's our second 2 by 3.
So they've been concatenated going down the rows
rather than along the columns.
NumPy does something called broadcasting.
So if you take an operation, it allows
you to do something really cool.
Like, let's take our regular list from before.
So if we were back in regular Python,
we could say a new list is equal to 1, 2, 3, 4.
Now, if we multiplied that by 2, remember what happens.
New list, if we come here, look what it did.
It just created 1, 2, 3, 4, and then it started over.
1, 2, 3, 4.
So it didn't do what we hoped it would maybe do,
which is multiply this, distribute it
through to each member of the array.
But NumPy does allow us to do that.
So now we can take the array from above.
We're going to say array equals array times 2.
And it's going to multiply it by each member of it.
So it was 0.7, 3.1, and 5.5 originally.
But now if we take a look at it, it's been multiplied by 2.
It's 1.4, 6.2, and 11.
So that's a really handy thing that you can do across these.
NumPy has loads of really interesting tools
related to statistics as well.
For example, within any of our things, we could do,
for example, let's take array random 3.
We could add up all of these things.
That would be with the sum command.
So we could do NumPy sum.
And then we're going to do array random 3.
Let's run that line.
And it's going to spit out and say
that the total of that array is 566.
But there's lots of other things.
You can have it deliver the minimum, the maximum,
the standard deviation, the variance,
lots of really cool things.
When you do it sum, rather than summing the entire thing,
you can make it sum along certain directions.
So let's have it sum up on a certain direction.
Axis 1 means it's going to be going down the rows.
So let's try this and see what it gives us.
And sure enough, now it's going to return an array.
But whereas the original array was a 3 by 4,
now what it's returning to us is a 4 by 1,
where it's taken each one of those rows.
Here's our array.
So it's added them across.
So 35 plus 71, that would be 106, plus 44.
That gives us the 150.
Adding this way would give us that number.
If we wanted to add them going down the rows,
then we have to change the axis to axis 0.
Let's try that.
So now we take a look at that, and it gives us three numbers,
like you'd expect because there's three columns.
So really lots of cool things you can do there.
If you wanted to know the minimum of your entire thing,
then we could do that.
We could do, instead of sum now, we're
going to do minimum of array random 3.
So it's going to look over the entire array,
and it's going to deliver us the smallest number.
It tells us that 7 is the smallest one in that array.
And sure enough, that's the smallest one that we see there.
There's something called cumulative summation.
What this does is it adds each row to the previous value,
so it cumulatively adds it up.
Let's show you what this looks like.
So that's what it spits out.
Here's what it had to start with.
Let's have them going down the rows.
So we're going to apply axis equals 0,
and then we're going to run this line.
So what it returns is four arrays.
The first was just the regular row.
Then the second array is the first one plus the second one.
So 83 plus 33 is 116.
66 plus 10 is 76.
So it's made that there.
Now the third array is going to be the previous one,
this previous value, plus this third one.
So it took 76 and added 61, and that gave you 137.
So some really cool tools that you're
going to be able to use when you do data analysis.
In addition to tools specifically
to describe descriptive statistics,
something really great that NumPy can give us
is just lots of really useful math things,
like square root or exponential, log, sine, tangent,
hyperbolic tangent, all these different things.
It can give us dot products.
It can do the transpose.
Let's do that one.
So let's say we take our array, our initial one up here,
and we want to dot it with itself.
So this is going to be array dot.
So to do this, we're going to do NumPy dot.
We're going to apply it to array.
And then the other array, we can't just
do it array times an array.
It has to be the transpose.
That's how a dot product works.
So we can do that.
We do dot t to apply that to the transpose.
Now we run it.
We have array dot, which is the dot product, which
should be a single value, which it is, 161.
You can do things like correlation coefficients.
So these are really cool.
It'll return the correlation coefficient
of all the components in your array.
So let's do that one real quick.
So we're going to do NumPy core coef.
We're going to apply that to the array random 3.
And when we run that, we're able to get this new one, which
actually shows all the correlations.
Let's actually store that into variables
so we can visualize it.
So now under var, we can see the correlation coefficients.
So obviously, all things are correlated with themselves,
and then others have more or less degrees of correlation
coefficients with one another.
What else can NumPy do?
Well, something that's really cool
is you can apply logic to the arrays.
So this is rad.
You can go through, and you can say, for example,
tell me all the points in my array
where a certain condition is met.
So let's do this one over here.
Let's take our random array 3, which is right here.
And we want it to know all the positions where
the value is greater than, say, 75.
So let's go ahead and ask it that.
We're going to do that by doing, we'll
call this logic, this new array.
We're going to do NumPy.
So this is going to be our array random 3,
where it's greater than 75.
So let's go ahead and run that.
When we run that, now we have this new array logic.
It has the same size as our one from before,
except if we compare these two things, here's random array 3.
You can see all the spots correlate.
These two true ones up here, that's greater than 75,
and so is that one.
This is not.
So you get this really cool map like that.
And then you can use this array of logic
to do operations on your previous array.
So for example, we could come down here,
and we could say array random 3 at all positions of logic.
Let's have those equal to, say, 99.
So when we run that line of code,
and now we take a look at our random array 3,
all those spots that were above 75 before
have now been turned into the number 99.
So this is just a really handy tool
for dealing with big lists of numbers.
Now, because we're dealing with arrays,
you need to be able to talk about specific positions
in an array, right?
You need to be able to slice and visualize through them.
Let's do that by taking a look at an image.
To do that, we're going to have to bring in something that
knows how to read images.
So it's a different library, and we'll
talk about some of these other ones later.
But for now, we're just going to drop it in here.
So the code is this.
It says, from skImage, we're going to import io.
And then the photo is going to be equal to io.imRead.
So that's a method of the io that
is allowing us to bring in a picture of my dog Scout.
So first, we need to tell it where the picture is located.
So this is in data for exercises folder.
And let's go ahead and run that.
Hopefully, we'll now have a photo variable.
OK, that's our photo variable.
This one, if we open it up, it's an array.
But it's an array that represents
the RGB image of my dog.
So there's three channels on here.
If we look at this, it's a 1,333 by 1,000 pixel image
with three channels representing the red, the blue,
and the green.
So that's the picture of my dog.
It's much more useful to look at this in terms of the plot.
We can take a look at these things.
So let's go ahead and show this.
So let's go ahead and show this with another command.
To show it, we're going to use the command.
To show it, we have to bring in another set of commands.
We're going to bring in import matplotlib.pyplot
as plt.
We'll do a whole video later on matplotlib and pyplot,
really cool stuff.
But it has a command for imshowphoto.
So let's go ahead and bring all of this in, run it.
And now, over here in Photos, we have a picture of my dog,
Scout.
What's great about this is it shows us
that it has from 0 to 1,333 rows.
And it has columns going from 0 to 1,000.
And that's a picture of our dog.
Now, we could access this array.
Let's do some examples.
Let's do the following.
We're going to show, instead of just the regular one,
we can index this.
Let's do all the numbers, but do them backwards.
When we run this now, let's take a look at what it produces.
It has reversed the order of our rows.
So that reversed the order of our rows.
We could reverse the order of our rows and our columns.
So rows and columns, let's run that one.
It's going to reverse and mirror the dog.
If you wanted to just mirror the dog, we could do that.
So we're going to do like this.
Let's run that one now.
It's just going to mirror Scout from the first image.
So now the table's on the left-hand side
rather than the right.
What if we wanted to just show a picture of Scout's face here?
Well, then we want to plot from maybe 100 up to maybe 800
along the rows.
So we can do that.
We can go from 100 to 800, taking all the data.
And then on columns, what do we want to take?
Maybe from 100 to 800 as well.
So from 100 to 800 on columns.
When we plot that, now we're going
to get a zoomed-in picture of just Scout's face.
And sure enough, there's Scout's face.
We can do some really cool things, though.
We could show the transpose of this.
Right now, this whole thing photo,
and then with these square brackets,
that represents an array.
We could take the transpose of that,
and that's going to rotate the picture of Scout's face.
For this one, we'll do, let's take that transpose.
So there we're showing a transpose
of one of the layers, right?
One of the RGB layers of Scout.
And we've rotated it by using the transpose command.
You can do some really cool logic, though.
Let's ask it to find, if you've used Photoshop
and you've ever used the threshold command,
let's build our own threshold command, right?
So we're going to say the threshold photo is now
equal to numpy, and then we're going to use the command where.
So photo is greater than, let's say, 200, right?
So RGB values can go from 0 to 255.
So we're going to say everywhere that it's greater than 200,
we're going to set that to 255.
And everywhere else, we're going to set it to 0.
So let's now try plotting that photo.
So we're going to do pyplot.imshow.
Instead of photo, we're going to plot the threshold photo now.
We won't turn on the transpose.
Let's try running that.
And what you ended up here with is a picture of Scout
where only values that were over 200 are showing up with any value,
and everything else is getting turned to 0, right?
And we can move this threshold, right?
If you wanted to make that 100, it's going to look a little bit different.
Here it is if you move the threshold to 100.
So all the filters that you're seeing on many, many images
to mask images in different ways are essentially
applying matrix transformations to the different RGB components.
And this is an example of how you could do that.
And again, that's showing all three RGB channels.
But if we wanted to, we could just show an individual one of those, right?
Let's start with, like, 0 and show that one.
So there's one of our channels.
You could show another one, though, and so forth.
All right.
So what else can we do with NumPy?
Another powerful thing that it can do is it can reshape, right?
Let's say we have an array, and we wanted
to change it to a different shape.
So let's say we have a 1 by 12, and we want
to turn that into a 3 by 4.
Well, we can do that.
So let's start by making an array.
We'll call this array 1D.
And it's going to be starting with 0.
We could type it out like this, right?
We could do that.
Or remember, we could do our command.
Let's do our command.
So we're going to do, let's do NumPy a range.
And inside of a range, we want it to go from 0 up to 12
in increments of 1.
Remember, 12 is exclusive.
It will not actually include 12, but we don't want it to,
because we want 12 elements.
And 0 to 11 is going to be 12.
So let's run that and take a look at it.
Over here in our Variable Explorer
now, we can find array 1D.
And sure enough, we have a 12 by 1 array.
If we look at this, it's just these numbers
listed from 0 up to 11.
Great.
Well, we can transform this with the reshape command.
So we'll call this reshaped array is now
equal to array 1D.
And now we're going to apply that, the reshape method.
And we can tell it what the new shape is.
Let's do a 3 by 4, for example, and go ahead and run that.
So now we're going to have a new variable called reshaped array.
It's right here.
And we went from having 0 to 11.
And now it's turned it into those same numbers, 0, 1, 2,
3, 4, 5, 6, 7, 8, 9, 10, 11.
So it's reshaped it into a 3 by 4.
Now, what would happen if we tried to do a 4 by 4?
We're going to get this error that says, hey,
I cannot reshape an array of size 12 into a 4 by 4,
because 4 by 4 needs to have 16 elements.
And we only have 12 elements.
So there are limits to the types of shapes
that you can convert it to.
You could convert it to a 6 by 2.
Let's do that.
6 by 2, because that has 12 elements.
So now when we take a look at this when it's finished running,
we'll see that it has 6 rows and 2 columns.
Now, notice that it went 0, 1, 2, 3, 4, 5.
What if you wanted to count straight down like that?
Well, we could combine the transpose.
So we're going to take, so let's have it go 6 by 2.
And then let's have it reshape that or take
the transpose of that.
There we go.
It counts now from 0 to 5 and then 6 to 11,
because we took and we reshaped it with 2 rows and 6 columns.
And then we took the transpose.
So NumPy is an extremely powerful tool.
If you get into data science or managing large data
or linear algebra where you have to do matrix
multiplying and all that, you're going
to have to learn how to use it.
But in general, a lot of these tools
that it has like exponential, log,
all these trigonometric functions
are just really useful.
And it's a really handy way to use them.
So that is NumPy.

