Build A Python App That Tracks Amazon Prices

developedbyed
17 followers
Check out my courses and become more creative!
https://developedbyed.com/
🎁Get all files, projects, exclusive videos and more on my Patreon: https://www.patreon.com/dev_ed
In this episode we are going to build out a project using python. If you are a python beginner this simple application will give you some good practice.
We will be using python to make requests and do webscraping on amazon.de. If you have difficulty following along, I highly recommend my python for beginners tutorial which will teach you all the basics of python in 1 hour.
🛴 Follow me on:
Twitter: https://twitter.com/deved94
Instagram: https://www.instagram.com/developedbyed/
Github: https://github.com/DevEdwin
#python #webscraping
Credit: developedbyed
Okay, if you're poor you're gonna love this episode so ever since I did this YouTube thing
I really wanted to get a new camera, and that's the Sony a7 mark 3 not that that matters
But it costs so much money 2,000 euros so the plan was I'm gonna build a small app that checks the price on
Amazon and if that price falls down to whatever whatever I want basically like
$1,600 then it's gonna automatically send me an email. That's what we're gonna build
we're gonna flex our snake skills, okay, or Python skills, and then we can celebrate and
We still don't have enough money to buy the damn product
So yeah, you can use this on multiple websites. Whatever you prefer. Let's stop talking and let's get going
Okay, let's get going. So I just have a scraper.py opened up here. It's just an empty
empty
Python script okay, so you can do this on your own in VS
Code and what we need to do is actually install two separate things so we can open up here. Oh my god
this is from the
Finished project okay, so what you can do is say npm actually npm is is not what we want
Okay, we're working with Python. We can say pip install
requests and
Ps4 all right, I'm gonna explain what these two things do so install those two after you do that we can import
Let's say import
Requests and what this basically does is we can get a page we can access a URL
So maybe Amazon and we can pull out the actual data from that website
Okay, so I'm all I'm gonna. Do is I'm gonna say
URL and I'm gonna set this equal to and the one I used is this from amazon.de
So as you can see here, I just heard Sony a7
You can get this one to
Amazon.de search for Sony a7 I think it's the first or second link if you want to follow along with exactly this
Yeah, and we can paste this back in here and our URL
Perfect now not every website that's gonna work with web scraping some block it some don't this one works
So this is the one we're gonna use okay, and that's the first thing the second thing
I want is headers, so I'm gonna set this equal to the dictionary here
I'm gonna say user agent all right
And this is just basically gives us some information about our browser, and we need to pass this into our header
So how can you get it you can serve for?
My user agent all right, and you can just copy this and paste it in so let me just paste this in here
Like so perfect. That's the second thing I have and now we can actually make a call
So I can say page equals to requests all right the library that we're using yet
And I can get the URL and the second argument is gonna be requests equal not requests my bad
headers equal to the headers
Okay, that's it. That's all we need perfect and this actually returns all the data from that website
Now what beautiful soup does for us is we can parse it and we can pull out individual items from it
So I can say from
Bs4 import beautiful soup okay now if you're from JavaScript you will do this reversed
So it would be important blah blah blah from well in Python. We are cooler, and we do it the opposite way, okay?
So once we have that I can create another variable called soup
I can set this equal to beautiful soup and pass in the page dot content and
The second argument is gonna be a string called HTML dot parser all right
This is gonna parse everything for us
And now we can actually pull out
Individual pieces of information if you want to see what this did actually we can I can show you we can say print soup
And there's a method on it called pretty fine
And let's run this we can call scraper
dot
Python
Scraper dot pie okay run this take a look bro. Oh my goodness
Hacker man
There's so much. It's not stopping it will never stop just like my mental state
Okay, there. We go take a look. We have a lot of this here with spans and everything
I don't know Amazon is crazy, and we're just gonna close this up, okay?
So what we can do is we can go back here press f12 and we can just pull out any data
We want I can hit this icon here click on this take a look
We have a span with a product title
How can I pull that information out I can create a div here called not the div a variable called title I can just say
Soup dot find all right
And you have a lot of different ones that you can experiment with as you can see you can have name
Attributes and you can add a class in here if you want to search by class and we can also do by ID
That's what I'm gonna show you so I can say find ID and set this equal to
Product title because that's how its name there now this actually if we print this out
Title this actually returns me the whole div I believe
Yeah, take a look. We have a span here, and we have the
Span with the actual information, and there's a lot of empty white spaces for some reason Amazon. What you doing?
I'm not sure okay, so to actually get the text out of it. We can do dot get text here
All right, that's only gonna return the text
But we still have that problem of having having a lot of empty spaces here as you can see so we can actually strip that
Out as well, and how do you do that? Well?
Saying dot strip. I just said it so there we go
Let's do another Python scrapper dot pi take a look and we should get back only Sony alpha blah blah blah perfect now
I don't actually need this, but hey why not let's have it here
What I actually want to have is the price, so let's analyze the price click on the price
It has an ID of price block our price, so that's exactly what I'm gonna
Do I'm gonna say price is equal to soup dot find I'm gonna say ID equal to price block
our price
Alright, make sure you don't misspell this and we can also extract the text here and get text now the problem
here is that this is actually a
String all right, and we cannot really compare
The value so I cannot compare a number to a string so we need to actually modify this in just a bit
So how can we do that? Well first of all what I can do is I can add a converted
price
And set that equal to and I can just extract
The first five characters all right, so I can say price add these brackets and say zero to five
All this does is it checks the first five so
It's gonna be
One two three four five so it only extracts this
Information it doesn't extract the dots 99 with the euros as well alright, so if we console log this out print
converted price
Let's take a look at what we have run this and it's gonna give us back
1.9
8-9 all right, otherwise it would give us back one point nine eight nine
Dot nine nine and a euro symbol, okay, so that's why we did this
So we only have back the first five characters, but it's still a string which is not good
So what we can do is we can add a float here wrap this around in the float
And now it turns it into a float if you don't know what a float is well basically an integer
An int is a whole number so like five and a float is like five points eight nine
All right, so you can add a dot to it. Okay, so we have this back or converted price
so what we actually want to do here is something like this if our converted price is
Smaller than a thousand eight hundred or whatever you want the price to be like that
Then I want to send an email all right, so I'm gonna say send
mail
Perfect so that's what I want to happen
Okay, and we can actually leave everything like so what I'm gonna
Do is take everything we have up here and wrap it inside a function, so I'm gonna say def
Define check price all right. This is just a function, and I'm gonna add everything that we have from here
Inside this function all right get rid of some empty spaces
Perfect, so we need to define this send email. I'm gonna just go down here
It's a def send email
It's gonna be another function and now we're gonna take a look at how we can actually send an email with the updated price
So the first thing we need to do we're gonna do this with Gmail
and
For this to work. We're gonna have to enable the two-step verification. There's another way we can do it the simpler way
Let me quickly show you so what we can do. I'm gonna pull this up here as if you search for something like Google
Allow less secure apps or something like that. Let's scroll down here less secure apps Google account
Okay, so you can enable this one. There's gonna be an enable button and that's gonna work
Just fine you can use your Gmail passwords
And you can send emails just fine if you don't want to use your Gmail passwords
What you can do is you can enable these two-step verification
So just Google two-step verification or whatever is called so here
Already second link. All right, just enable your two-step verification. And once you do that, what you can do is
You can Google for
Google app
Passwords. All right, and you can create passwords for different things. I'm gonna sign into my Google account and
as you can see, you can create one you can hit here select app and you can create a
Generate a new password for your email for you your YouTube calendar
Whatever you want. So in this case, you would select mail and
You would select Windows computer. Alright, that's what I'm on. You might probably have Mac or whatever
I'm gonna delete this old one I had and I'm gonna generate. Alright, so this just generates us a new password that we can use
Whoo. Okay
Let's go back to our scraper and see what's up. Okay, so we actually need to import another package here
We don't need to install this separately. So what we can do is
We can just go up here and say import
SMT lib. All right SMT flip what this is is basically a simple mail
Protocol or something like that. I'm not sure exactly what it's called, but it enables you to send
emails
Let's go down here. And what we need to do is actually establish a connection between
our connection and
Gmail's connection. So to do that we can say set up a server here and set it equal to SMT B blip
dot
SMTP and what we need to pass here is
SMTP dot gmail.com. Alright, so this is Google's or gmail's
SMTP and the connection number is going to be eight
Five eight seven. All right, that's what we need to pass in and then what we're gonna do is we're gonna say server
Dot hello
All right, if you don't know what's in low is let me pull it up for you here on the screen
Basically, it's a command sent by an email server to identify itself when connecting to another email
All right, so it's kind of establishes a connection between the two
Okay, so we need to call that after we do that
What we need to do is go down here and say server dot
Start
DLS which basically encrypts our connection and then we can call server dot. Hello again. All right
I know this is getting pretty pretty jargony. But hey
That's what we need to do. So that's what we're gonna do
If you don't remember this you can just always come here and check so
Next up what we want to do is we actually need to log in so we can call server dot login
The first argument is gonna be the user as you can see and the second one is gonna be the password
So my user is gonna be ad magician. I know
Dot gmail.com and the second one is gonna be the password now again
Depends what you did if you did the two-step authentication, then you can just generate this password, which is exactly what I'm gonna do
Okay, so I'm gonna pass this in here. We have the connection and now we can actually set up our email
I can add a subject set this equal to hate the price fell down
The second one is gonna be a body set that equal to check the Amazon link
All right, and I actually can copy this link down here. So our email
Actually gives us the link back with the product perfect and then we can set up a message
I'm gonna format this string by adding an F which means we can just basically interpolate
Subject here and just add subject
All right, kind of like JavaScript with the dollar sign and curly braces and then we can add a new line new line and say body
Okay, which is the actual message?
Perfect, and then we can send the email we can say server
dots and mail and
Here what we need to pass in is from as you can see from to and the actual message
So from I'm gonna say ed
Magician and gmail.com and the second one is gonna be I'm gonna send it to myself again. So to another email
I'm gonna say my full name here
I know and these third argument is gonna be the actual message which we did up here
so from to and the message and
Finally, I'm just gonna print out that. Hey
Email has been sent
Yeah
Perfect. And finally we need to close up the connections. I'm gonna call server dot quit
Quit my life. Okay, so that's what we have. So now up here what I can say is
Hey, if the converted price falls below
one point
7,000. Alright, so this is like
1,700
euros Oh
JavaScript then
We can send an email
That's it. That's all we need to do now in this case. This is not gonna work because our price is
This one is bigger. So let's give it a shot
boom
Well, what we actually need to do is call this function this check price because we added it into the function
So we're not actually calling anything here as you can see, so let's go down here and just call this function
Okay, let's take a look
Please all right
So we get back the price and the title because we just printed that out on the screen, but nothing really happens
We don't get this. Hey email has been sent. However, take a look if our price changes in the future
So I'm gonna just add a bigger sign here to make this true
All right, cuz this is bigger but you can add any price you want here
Well now this is gonna be true and it's gonna send me an email so let's take a look boom boom boom come on
Hey email has been sent. Whoa
Let's take a look here. Let me go to my email and prove prove it to you
So I'm gonna zoom in here. Oh, never mind. Alright, so take a look. Boom. Boom. Boom
Email has been sent price has fell down and we get the Amazon link to it. So everything works just fine
Perfect now what we can do is we can actually make this run for longer
Some multiple times and to do that what we can do is we can just import something
this is again, you don't need to install this we can just import time and
Let's see what we do rather than just running this once I can just add a while loop here while true
So this is always gonna run
Like that what I can do is say check price and
Then I can add a time sleep and I can add the number of seconds. I want this to
Sleep, so basically it runs once and then it pauses the execution for a number of seconds
So if I add 60 here, then it's gonna pause it as you can see delay the execution for a given number of seconds
So this would check every minute for you, okay, however, that's probably not a good idea
So you would want to check this at least like maybe once a day so you would do I don't know
My math is really bad. So you would do I mean 60 times
60 times 60 would be probably an hour
So there you go now if you acquired the skill of scraping the web and also sending some simple emails
So I highly recommend you to try it out with different websites see what information you can get now again
Not every website is gonna let you do this
Some have some checks anti web scraping kind of checks with future captures or whatever. So not everything is gonna work
Unfortunately, but yeah experiments and also don't send like a thousand requests to mess around with their server
Yeah, have fun with it. I also did one with getting all the recent movies of the
IMDB so the latest movie releases and it also saves it down into a file for you a CVS file
If you want to see that I might put it up on patreon
Just leave a comment in the description and I might upload the source files for that as well. Okay. Thanks again so much for watching
I apologize for no magic tricks for today
I'll do this one because this is super simple and stupid, but I'll catch you in the next one. Have a great day. Bye
