noc19-ma14 Lecture 34-Fitting of Linear Models: Least Square Method- R Commands

IIT Kanpur July 201
20 followers
Dec 3, 2023
The method of least squares introduced for linear model is extended for more than one variables. Matrix plot is also discussed.
[Music]
[Music]
welcome to the lecture on the course
descriptive esterases with our and
welcome to the last lecture of the
course
yes we've been able the last lecture
comes this gives us happiness for the
students and for the teachers also so we
try to understand what are we going to
do in this lecture you may recall that
in the last lecture we had discussed the
principle of least squares and
we also had understood that how one can
obtain the estimates of the parameters
or the values of the parameters on the
basis of given set of data
in case of a model y equal to alpha plus
beta X
we had taken an example and we had
solved it manually
the idea was to show you or to expose
you with the basic concepts now in this
lecture
we will learn that how to obtain the
same result using the our software and
after that
considering only one input variable is
not a very realistic thing so in case if
you want to extend the principle of
least square where where you have more
than one independent variables or more
than one input variables then how to do
it and how to use the our software to
obtain the least square estimates of
parameters in that model so this is what
we are going to do in this lecture so
the first thing comes what is the
command for obtaining the least squares
estimate or fitting a linear model using
the our software so in case of our
software the our command for fitting a
linear model is LM these applets this is
an abbreviation of L means linear and M
for model and this command LM is used to
fit the linear model
well you can see here that this LM
has various arguments
and so on
but here we are going to use a very
simple thing just basis on formula
and you should know why why I'm not
going to discuss all the details I had
explained you in the earlier lecture
that whenever we try to fit a linear
model
this does not end the story after that
you have to check how the model is going
to perform
with the entire population
there are different types of hysterical
assumptions which are needed to
expose the stimate adverse were
different types of static a tool
like a test of hypothesis goodness of
fit and so on so well here I am NOT
considering the course on linear
regression analysis but I am simply
trying to use
one of the methods for finding out the
model which is used in the case of
linear regression analysis and this
command LM in our software was developed
in order to find out the further
stat scale tool related to the linear
regression analysis
so that is why you will see that in the
command L M there are many many
arguments
but I will not go into those details but
I will request you that if you want to
understand them first try to have a
course on linear regression analysis and
then try to understand those concept the
interpretation of all the terms inside
the argument they can be be studied from
the help on LM command right so now my
objective here is to show you that how
to get the things but briefly I will try
to give you the idea here what we are
going to use that there is an option
here formula so basically we will be
adding or using here the option of
formula so this is going to give us a
sort of symbolic description of the
model to be fitted right and what are
the details of the model they are given
under as separate specifications and
next the data this is also an optional
argument
where the data is given in the frame of
in the structure of a data frame
or see a list of environment is given by
a as data dot frame and so on but we are
not going into that detail and similarly
there is another option here upset and
this gives you an option to use an
optional vector and which is specifying
a subset of observation to be used in
the fitting process but anyway we are
not going to use it we are simply going
to use the option here formula so now
how to use this formula I will try to
show you with an example the same
example we had considered earlier where
we collected the data on 20 students on
their marks and the number of hours in a
week which they studied and this data if
you remember in the last lecture was
obtained here that beta hat was obtained
as 6.3 and alpha hat was obtained at 160
8.65 and model was in like it but this
all was done manually all the
competitions were done by using the case
simple calculator
now I will try to show you how to do it
on the art software so as we had
discussed we already have stored this
data in that two data vectors marks and
of us and how to store this data that I
discussed in the last lecture right that
this all this data is coming in the same
order so that these observations are
paired I mean the first observation X I
Y I mean first observation of marks and
first observation of of us now
this is the most important part that how
to give the command
first you try to write down the
instruction LM and then try to write
down here the
output where you will hear Y and
then use this equivalent sign this is
present on your keyboard the keyboard of
the computer which you are using
and after this you try it to use here
the
we are able to give the input data so
why is your hair output variable and X
is your hair input variable
right so and they have to be given in
this format so in our case our output
variable here is Marx and input variable
here is ours and they are given in this
framework just joined by the sign
equivalent
right and the sign I can make it here
more bigger something like this you will
see on the on your keyboard
now once I try to do it this will give
me this type of outcome so first we try
to understand what is the meaning of
this outcome
once I say that
that LM Marx equivalents hours this is
going to indicate that our model is Marx
is equal to C here alpha plus beta times
C hours
plus some error
right and this will
inform the our software that we had
obtained say beta hat I goes from 1 to
here and X I minus x bar and y I minus y
bar upon summation I goes from 1 to N X
I minus X bar whole square so this
command will inform
the our software that the value of x is
are coming for example I will use a
different color so this values of here
excise are coming from the data which is
given here in hours and
similarly the values of Y's are coming
for computation of these things from the
first theta vector marks
right and so it uses the formula is
equal to marks equivalents hours and the
outcome comes out to be like this so you
can see here this is written as
coefficients the coefficients means the
value of alpha and beta
and this is specifying here the value of
alpha that is also called as intercept
so this value is coming out to be one
six eight point six four seven and the
value of the beta
which is coming here here so this is
here the value of alpha hat and this is
here the value of beta hat so this value
is indicating that this is the intercept
term and
this is indicating that this 6.30 4 is
the coefficient
associated with the variable of us and
you can see here these are the same
value which you had obtained manually
you can compare here with this and here
this so you can see here this is pretty
straightforward to obtain such a result
inside the our software now
I
will try to show it on the our console
and this is here the screenshot of the
same thing so first I will try to
prepare my data
so this is here marks data which I have
entered and
this is my here
data on of us
which is entered here
so you can see here marks data is like
this a worse data is like this and now
I'm trying to find out here the model LM
say marks
equivalence say hours
so you can see here that this is coming
out to be like this right and here you
have to be careful that how you are
going to specify the value of x and Y
for example in case if you make a
mistake and if you say in place of marks
hours and hours in pairs of marks then
this linear model will be fitted like
hours and
between hours and marks and you can see
here that this is entirely different
than the first one you can see here the
first value is 168 but here the value is
point to 2 but this is a wrong thing to
do because here in this case when you
are trying to fit a model like this one
you are trying to say that number of
hours is your output variable and input
variable is your marks which is not
correct here in this case so this is how
we you try to obtain this thing
right so now let us try to come back to
our slides and now I try to give you one
more concept now you see we have learned
how to find out a model or the least
square estimates of alpha and beta in
the case of a linear model where you
have only one variable but in practice
you can imagine that any process is not
going to be controlled only by one
variable but there will be more than one
variables and we had discussed such
example in the last lecture for example
the yield of a crop will depend on
several factors
quantity of fertilizer quality of
fertilizer temperature rainfall
irrigation and so on
so now
in case if you want to extend this
principle of least squares
to the case when you have more than one
input variable then how to do it
well this is a
topic which is basically taught in case
of linear regression analysis under the
topic multiple linear regression model
but here my objective is not to teach
you the regression analysis my objective
is to tell you or show you that how you
can obtain the values of the parameters
on the basis or given sample of data
using the our software in a case when
you have more than one input variables
so in this case I will try to take an
example of the linear model and I will
show you logically that how you can
extend the model which you have
considered here into a multiple
framework when you more than one input
variable but yeah this is not a very
general technique that is valid for all
the models miss every model has its own
way to extend it to a multivariate
situation but here you please try to
learn that if I try to extend my linear
model in this particular way then how
the our software can be used to find out
the values of the parameter estimates
okay so now you may recall that we had
considered the model here y equal to
alpha plus beta X
now I'm trying to say that here you had
only here one
input variable
but now I'm trying to extend it and
suppose I'm saying that there are more
than
one input
variables
so in the first case I have denoted the
input variable by the notation X so now
suppose I say there are p number of
variable
of input variables
so I can denote them say here X 1 X 2
see here X P
right miss I deal in that symbols and
notation of regression analysis this is
small X 1 is small X 2 is small X B
should be capital X 1 capital X 2
capital X P but here my idea is
something different than the objective
in linear regression analysis so I try
to extend it and since I am going to
consider here and edit your model so I
try to write down the terms like beta X
for each of the variable so for X 1 this
will become say here beta 1 X 1 4 X 2
this will become here beta 2 X 2 and
similarly here for Expedia's will become
here beta P X P and then I try to add
them together so this is what I am
writing here in the second line that y
is equal to alpha plus beta 1 X 1 plus
beta 2 X 2 plus up to here beta P X P
plus E
so easy also here a
header and this is random error involved
in the observations
no similar to the simple scatterplot
that we had obtained in the last lecture
here in this case you have more than one
independent variable so
matrix plots are more useful in
verifying whether the relationship
between y and X 1 X 2 XP is linear or
not one thing which you have to keep in
mind here that in the earlier case we
try to establish the linear association
between x and y but now you have here a
group of X's X 1 X 2 XP inside the one
group and there is here one simple
output variable Y so we are trying to
verify the linearity of a single
variable Y with respect to a group of
variable of X 1 X 2 XP
this is not so straightforward
one option is that I can make individual
plots y versus X 1 y versus X 2 y versus
X 3 and so on and now what we can
conclude that if all the relationship
bit of Y is with respect to each of the
X 1 X 2 XP is coming out to be linear
then we can expect that their joint
effect may also be linear but this is a
little bit tricky situation and you need
some experience to handle this condition
but here but here I would like to just
inform you that how you can verify our
how you can construct such plots in the
our software so now the first I would
try to show you that how to construct
such mattress plot in this case if you
try to see how you are trying to obtain
the observation you are trying to
conduct the experiment say in x and for
every
experiment you will have an observation
like
observation on the first variable
observation on the second variable and
observation on the pilot input variable
and then here value of y
and if you remember earlier it was
simply here XIII because there were only
one variable so here the observations
are going to be something 1 I 2 I and I
and Y I so this is going to represent
the I yet and it tuple of observations
on say Y and X right so now we assume
that each set of observation will
satisfy
this equation so we can write here for
the first set of observation
we can write that this set of
observations circus wise the equation
like this so instead of here X 1 I have
X 1 1 instead of here X 2 I have here X
1 2 instead of here P I have here X 1 P
and
associated random error here is even and
the obtained value of y here is y 1 and
similarly we have obtained here say an
observation which are satisfying this
equation now using a very simple theory
of mathematics based on vectors and
matrix this entire set of equation can
be expressed in the form of vectors and
matrices
so this I'm trying to give it here I am
NOT explaining the theory of metrics
here but I assume that you know it so
all this observation they are contained
inside the n cross 1 vector here y1 y2
yn and all those parameters alpha beta 1
beta 2 beta P they are contained in say
another vector here
consisting of P plus 1 elements alpha
beta 1 beta 2 beta P and the
Associated matrix of input variables on
the data on say here x1 x2 and say here
X P this is given here like this and the
first column is indicating the presence
of intercept M 1 1 1 1 1 so now I can
express this Y vector here as say small
Y and this entire matrix here to be here
X this entire matrix to be here beta and
this entire e 2 e n vector to be here
small e
right so now we have this model here y
equal to X beta plus e now in case if
you try to use the principle of least
squares then principle of least squares
if you remember in the last lecture we
defined the sum of square of this e is
and
this were defined as s summation I goes
from 1 to N AI square so this quantity
can be written as say here e transpose e
where is now a vector so E is now here Y
minus X beta transpose Y minus X beta
like this now in case if you try to
differentiate this quantity with respect
to here beta put it equal to 0
then after solving you get here say beta
equal to beta hat which is equal to X
transpose X whole inverse X transpose Y
which I am writing here
right I am NOT giving you here the
details but if you try to see this is a
simple extension of the least square
estimate that you obtain in the model
with one input variable so now these
things are replaced by metric so you can
see here that X is the metrics of X here
is metrics of observations on say input
variable
so this is known to us similarly here Y
this is a vector of observations on
the observed values y 1 y 2 y n so this
x and y are known so I can compute the
value of beta hat and
this will be called as least squares
estimator of parameter beta and beta hat
will look like if you see beta is
looking like this then beta hat will
look like say alpha hat beta 1 hat beta
2 hat up to here beta P hat so this also
has the same order as of beta which is
of order P plus 1 Cross 1
right now I would try to show you the
computation of beta hat
using
example and I have just taken two input
variables in this example to make it
more simple and in this example 25
observations have been collected on the
time taken by a courier person in
delivering the parcels
so
it is recorded that how much time the
courier person takes in delivering the
parcels and obviously this time is going
to be dependent that how many parcels
are being delivered by this courier
person and also how much time the
courier person has to travel so this
number of parcels are going to be
denoted by x1 and
distance traveled by the courier person
is to denoted as here say x2 and
whatever the time
is taken that is denoted by here Y so
the interpretation of the data goes like
that there is a courier person who has
to deliver 7 parcels and
suppose the courier person travels 560
meters and that total time in doing
this job is sixteen point six eight
minutes
similarly
there is another courier person who
delivers three parcels and percent
travel to 20 meters and
person takes 11.5
minutes and so on so this is how we have
obtained this 25 set of data and all
this data on Y x1 and x2 has been stored
inside the data vectors inside the our
software right as we have done it many
times in the past so essentially now we
are going to fit here a model y is equal
to alpha plus beta 1 X 1 I plus beta 2 X
2 I plus CI where all these observations
are denoted by here I and all
observation will satisfy this model y
equal equal to alpha plus beta 1 X 1
plus beta 2 X 2 so all this data in the
same order has been
stored in three vectors the time of
delivery and Delta in the number of
parcel in Marcelino they have been a
parcel numbers and traveled distance
inside the data vector whose name is the
distance right so once again I will
explain you that this first observation
correspond to in the first observation
of parcel number and this correspond to
the first observation in that distance
which is given here I say observation
number here one right
so this data isn't given over there so
that we already had understood in the
earlier example now I would try to first
show you that how to create a matrix
plot
right what is a matress plot you see
when we had an idea or when we had
discussed the simple case where I had
only one independent variable or one
input variable and one output variable
then I can make a scatter plot and a
scatter plot will give us an idea that
how the things are
going to look like whether there is a
linear trend or a non linear trend but
definitely when we have more than one
input variables then what we are looking
we are looking between the joint effect
of one variable Y with respect to a
group of variable X 1 X 2 XP so finding
outers the Kutcha curve is more
difficult so what we try to do we try to
create the scatter plots pairwise
between y and x one y and x 2 y and x 3
and so on and then what we interpret is
that if all the relationship that means
the relationship of Y with respect to
each of the X 1 X 2 XP is linear so we
can expect at the joint
effect or joint association between y
and all X 1 X 2 XP is also expected to
be linear well you need some experience
in interpreting search results but here
I would like to show you that how to
create such matches plot ok so that
command for creating the matress plot
here is pairs Pai RS and inside the
argument what we try to give here the
data vectors and
with some Orion form and another mode
general structure is say disappears and
inside the argument we give the formula
just in that case of say linear models
the operator LM and after that we there
are different options to be given data
subsets any action and so on so here you
can see here I have given the details
here that X is trying to give us the
coordinates of a point given as numeric
columns of a matrix or a data frame and
similarly here formula that this is the
same thing what we discuss in the case
of linear model so we have to write the
formula as say with an equivalent
science given by the variables and
separated by plus sign
right so each of this term will give a
separate plot with respect to Y and
then data is a data frame in which the
data for the formula is
has to be used right then there is an
option here subsets and data exactly in
the same way as we did in the case of
linear model so I am NOT discussing it
here now I would try to create here a
matrix plot of
the given data set just by writing the
formula formula you see I am writing
here now piers and then inside the
argument formula for that I write here
an equivalent sign and then I need to
find out the scatter plot of three
variables
delivery time parcel number distance so
I'm not discriminating here what is my
input variable and what is my output
variable and it means you can get more
information about this command pairs
from the help but here I am using one
option which is the mean to give the
title of the graph so you can see here
this is my graph and this title is the
same hair matrix plot of delivery time
beta which is printed here I will try to
show you on the our console also but
first we try to understand this picture
this matches plot will look like this so
you can see here first try to understand
this graphic so you can see here what is
there on the y-axis
what is here on the y-axis this is the
same thing which written here and what
is here on the x-axis I will use a
different color so that you can observe
this is the same thing which is written
here so in this box
the x-axis is denoting the parcel number
and Y axis is denoting that the delivery
time and this is the scatter plot of
two variables parcel number versus
delivery time or the number of parcels
versus the delivery time and you can see
here that this trend is nearly the
say this a linear trend and a positive
trend rather
similarly if you come to this block
again now what is being mentioned here
here this is the same thing which is
written here say distance so now this is
the plot between distance and
what is happening on the y axis here
this is that delivery time so you have
to take the y axis from the y axis side
and excesses from the xs/s side so this
is a graphic between the distance
travelled by the courier percent versus
delivery time and you can see here that
this curve is also coming out to be a
sort of a having a linear trend and
similarly if you try to look here in the
third case
see here so you can see here on the x
axis here this will be here distance and
on the y axis we will have here number
of parcels so this is a graph between
the distance versus parcels parcel
number or the number of parcels and you
can see here here also the if you try to
look at the trend this is again shows a
nearly linear trend and then if you
observe direction of my pen it is here
this crossing this diagonally now what I
have shown you here is
the plots in the upper diagonal in this
side now what is happening in the lower
diagonal this is the same thing for
example
these two will match this two will match
and these two will match so usually we
try to look in either in the upper
diagonal or on the lower diagonal so
they are going to give us the similarity
information
okay and yeah means some of these blocks
which I am indicating here by cross
these are not used why because this is a
plot between for example in this case
this is the plot between say here
delivery time versus - delivery time
which has no meaning right so by looking
at this matrix plot we can have an idea
that
whether the joint relationship of Y with
respect to X 1 and X 2 is a linear or
not so in this case I can take a call
yes it is approximately linear well in
this case I will try to show you one
thing more that is since we are dealing
here only with two variables and I want
to see the joint effect of Y with
respect to X 1 and X 2 so there are
three variables Y X 1 X 2 so I can also
take the help of this three dimensional
plots and if you remember we had
critical saw we had discussed earlier a
three dimensional plot which is
by using the command scatter plot 3d so
we try to use it here but it is always
not possible because sometimes the data
is more than three directions but this
is your judgment what you want to do so
if you try to use the command here the
scatter plot 3d then first you need to
upload the library so use this command
library inside the argument a scatter
plot 3d and now I try to plot this
scatter plot 3d with these three
variables delivery time number of
parcels and distance and this comes out
to be like this so this is trying to
give you here the sort of here panel
which is here in this case
so you can see here this panel is
looking like yes there is a that most of
the points are lying close to the panel
now you can also use the option to
change the direction of this cuboid for
example I use here the option in the
next command as angle equal to 120 and I
try to rotate this figure by handed 20
degree so you can see here this that the
excesses are change these are change and
this is see here changed but now I
looking at different structure now this
is something like this so just by
looking at different types of picture
you can finally conclude whether you
want to have a linear model fitted to
this data or not now we have concluded
on the basis of given set of data yes we
are confident that a linear model can be
fitted so now we use the our command
so now you can see here we are
interested in this model where I have
two variables X 1 and X 2 and
coefficients are alpha beta 1 and beta 2
so I need to estimate three parameters
alpha beta 1 and beta 2 so I will try to
use the same command that we use earlier
but now I am making here a small change
I will use the same command L M now
whatever is my here output variable I am
trying to give it an a here as such and
now there is an equivalent science which
is trying to indicate the formula that
now the variables to be used as input
variables are starting so now I'm using
here two variables named parcel number
and distance so they are given in this
format that they are separated by this
plus sign so if you have more variable
suppose if the model is beta 1 X 1 plus
beta 2 X 2 plus beta 3 X 3 or so plus
beta 4 X 4 so all those variables will
be added here
something like say here X 1 plus X 2
plus X 3 plus X 4 and so on so this
formula will remain the same and if you
try to operate it on the our software
you will get this type of outcome so
this is giving us the formula right
so this is essentially writing that Y is
equal to something like beta 1 X 1 and
say here alpha plus beta 1 X 1 plus beta
2 X 2 and now this outcome has to be
read like this that these three values
are giving us the values of coefficient
associated with these values so two
point one nine five seven nine it is the
value of intercept term one point six
seven eight zero three this is the value
of the coefficient associated with
parcel number which is here actually
beta one hat
and this is the value of the coefficient
associated with the variable distance so
in our symbolic this is the value of
beta 2 hat and yeah this intercept term
this two point one nine five seven nine
this is the value of alpha hat so my
model becomes here del time this is the
delivery time is equal to two point one
nine six plus one point six eight times
parcel number or the number of parcel
plus point zero one three times
distance
so you can see here now we have obtained
a model with two variable or model with
two input variable and the same story
you can continue with more than one
variables and
I will try to show you these things on
the our console also and you can see
here this is the screenshot what I'm
going to show you now so let me first
try to
create this data vector here
so you can see here and then similarly I
try to create the
data vector on parcel number
it is here and similarly I try to create
the data on
distance table like this so you can see
here this is my data del time like here
like this parcel numbers number of
parcel it is say like this and
distance here like this
now I first clear the screen so that you
can now understand what is going to
happen so I will first try to create
here a three dimensional scatter plot
and for that you know that first you
need to create here
copy this command here so you can see
here what I'm doing that I try to copy
and paste this command and you get here
this type of
so this is the same plot which I just
explained you inside the slides and now
after this I try to obtain the
scatterplot in three dimension so first
I need to load the
library here so I load the library of
scatter plot 3d we are we already had
installed this library earlier
so this is already there because you
need to install this package only once
and after that whenever you want you
just use the library so you can see here
this is the
scatter 3d plot and if you try to change
the angle here say angle is equal to
suppose 120 so it will give us this
change picture and suppose if you try to
make it here see angle equal to see her
90 so it will give you a different
picture like this one so by using these
things you can have an idea that what
really you are going to do
okay now next I try to fit here a linear
model with this data set right
I close this pictures and I clear the
screen by control L and I paste the
command over here and this gives me the
this command so you can see here these
are the values of the coefficients that
you have obtained so this value 2.19
this is the least square estimator of
intercept term this one point six seven
eight zero three quickest is the least
square estimate of the coefficient
associated with the variable number of
parcel and this zero point zero one
three one one is the value of the least
square coefficient
or the value of the least square
estimate of the coefficient associated
with the variable distance travel and
this same screenshot has been given in
the slides also
so now I
would like to stop in this lecture I
have tried to give you the idea of
principle of least square how to
implement it inside the our software
sometimes it is possible that there are
some nonlinear curves and
in case if it is possible to make some
transformation
to change those nonlinear form into a
linear form say by taking log or by
taking exponential
then
you can use the command L M to find out
the least square estimate in that
transformed linear model that was
originally nonlinear but in that case
you have to keep in mind that if the
original data is given as Y and in case
if you transform the data into R the
variable into log of Y then the curve
becomes linear then you need to input
the data on log of Y so these things say
you have to keep in mind and
using this technique you can estimate
the least square estimate very easily
using the our software but definitely
you will need
more things to learn if you want to use
it further and
which is usually not possible for me to
to do in the same
course there is a usually there is a
entire course on the topic of linear
regression analysis
but here now we come to an end to this
course and this was the last lecture of
this course I
hope you enjoyed it you understood it
well I am saying that this is the end of
the course
but practically for you this is the
beginning of the new course I
have given you only the basic
fundamentals I
have told you very basic things
but believe me
these are the things which are going to
create the foundation for further things
in case if you want to
conduct a Monte Carlo simulation
you want to find out a
good model
or you want to do any data mining
or anything
the first step is the tools of
descriptive statistics and
even inside those things for example in
data mining and other things
you use the tools which we have discuss
here under the course descriptive
statistics and
you have seen that one tool will not
give you the complete information about
the entire data sets there are different
types of features
which are hidden inside the data
data is so now that it is not telling
you that I have this I have this I have
this you are the only one who has to use
different types of tools and
get the information from the data and
it is very important that
you try to use the correct tool on the
correct Decatur in case if you try to
use anything else like as wrong data on
correct tool or correct tool on wrong
data you will not get a correct
aesthetical outcome
and
then people try to make different types
of stories that aesthetics is not
telling the truth
because they don't know what is the
appropriate tool for an appropriate data
so my request is that please try to
understand these topics in more detail
before you try to apply them on a real
data set
definitely you need to also study a book
which will give you more properties of
these tools and
more application of these tools before
you become eligible to use the tools in
a real situation and in a real situation
you cannot say that I can use only the
means of central tendency or the
variation or the Association all the
tools can be applied to any data set but
this is only you who is going to take a
call take a correct decision that which
of the tools have to be used in a given
situation in their graphical or say
analytical or a combination of them
so you learn more in statistics and
enjoy the course I will take a leave and
see you sometime soon once again
till then good bye and
God bless you
[Music]
[Music]
[Music]
