noc18-mg33 Lec 41

IIT Kanpur July 201
20 followers
Dec 4, 2023
[Music]
[Music]
welcome back my dear friends and do your
students this is the D a DM which is
data analysis and decision making one
course under the NPTEL MOOC series and
as you know this is a 12 week course
total duration being 4:30 hours which is
60 lectures because these lectures for
half an hour and we are going to start
the 41 lecture which is basically
starting we have already finished 40
which is 8 weeks are gone we are going
to start the ninth week and each week we
have as I just mentioned five lectures
each being for half an hour and my name
is raccoon and insan' Gupta from Miami
Department IIT Kanpur now if you
remember we but technically I'm going to
go a little bit into the past starting
from the 29th 30th lecture we started
studying the concepts of multivariate
statistics and for the multivariate
statistics we took the concept initially
for the different type of distributions
like like the multinomial distribution
and the wizard distribution multivariate
t distributions the multivariate normal
distributions then we slowly covered all
the concepts and we also studied only
related to the multivariate normal
distribution the concept of hypothesis
testing and the same basic concepts
continue remain remaining as in the
univariate case that given the standard
deviation you want to find out the mean
so if the standard deviation is known we
just simply take the sample mean
similarly even if it is not known the
best estimate is still the sample mean
whether for the univariate case or the
multivariate case then for given the
mean value for the population I am all
all calls considering the normal
distribution remember that we care we
want to find out something to do with
this
darah understand the variance of the
sample because we do not know the
population variance so they can be
technically to formulas in the one case
we use this - another case we use this
without the - and it actually means that
if the mean value of the population is
known we take that we do not lose any
degrees of freedom and we D right
divided by N and do the calculation the
formula is no I am only highlighting in
the main difference and obviously inside
the formula it will be basically a
summation of in the numerator summation
of the in the bracket X I minus mu whole
square because mu is known now in the
next case when mu is not known the best
possible course of action would be try
to find out the best estimate of the
population mean which is the sample mean
so as you use the set of observations x1
to xn for the first time and try to
calculate the best estimate for the
population mean which is the sample mean
you lose one degrees of freedom hence in
the formula it is now it will be divided
by n minus 1 and in numerator the actual
equation would be summation bracket X I
minus X bar n whole square because that
mu has been replaced by X bar n which is
the sample mean now later on we saw I am
sorry I am going going to go back a
little bit more in the future instance
recapitulate whatever we had discussed
even though it that may not be very
relevant here you will not understand
why I am doing that later on so given
for the point estimation problems we
found out the best estimate for the
normal mean best estimate for the normal
standard deviation for the population of
course then given exponential
distribution we found out the best
estimate for the the parameter for the
population then for the Poisson
distribution we form of the best
estimate and obviously they were with
proofs which we didn't state but the
main contention of the analysis or the
discussion was that you basically try to
analyze the unbiasness and
Stanzi unbiasness mean the case where
you want to find out the expected value
of the sample statistic or the sample
characteristics with its distribution
such that in the long run the expected
value is equal to them and the actual
parameter value from the population and
the consistency means that as the sample
size increases observation number
increases the variance should basically
tend to zero so what I mean by the
variance actually it will be the
probability between the difference
between the parameter value and the
statistic value as ends n which is the
sample size increases the probability
slowly of the differences being less
than equal to some epsilon epsilon is
very very small value so it will always
tend to 1 because it is both the values
are equal that means the population
parameter is equal to the sample mean
which you are trying to utilize as the
best estimate now later on we saw that
we are given in this was the products of
the point distribution and we tried to
use the same ideas in the multivariate
case also in the interval estimation we
saw that they were basically for tower
distributions coming out and the cons
why the consequence were that given the
normal distribution you want to find out
something to do with the normal mean
given the standard deviation of the
population known we took the Z
distribution and obviously there was no
concept of degrees of freedom then we
found out that and we gave the logical
sequence of the understanding was that
given the population variance is is
unknown and yet you want to find out
something to do with the population mean
we took the T distribution and the T
distribution had n minus 1 degrees of
freedom the question is why n minus 1
and because the T distribution coming
for the case when as I have just
mentioned the population variance being
known but do you under 1 to find out
something to do with the population mean
and this degrees of freedom as you know
is basically coming from the fact for s
- and hence without a - now these two
formulas
to Z and T would always be utilized when
we are trying to just do some studies or
go into the depth of analysis of some
problem related to the population mean
and proportion variance or the standard
deviation given being given in the first
case standard deviation being known and
in the second case time dilution beam
unknown that means there n T now if you
want to find out something to do with
some studies related to the population
variance so again there would be two
cases population mean being known and
population mean being not known if the
population mean is known then obviously
you'll use the Chi square with n degrees
of freedom when there is only one
population and if you want to basically
compare to population standard
deviations and if both the mean values
of the population 1 and population 2 is
known
we'll use the F distribution with M
comma n m being the number of
observation in the first population n
being the number of observation the
second population so it will be F with
degrees of freedom M comma n now when we
want to find out something to do with
one population its standard deviation on
the variance and the population mean is
not known then we know that we will be
utilizing the Chi square with n minus 1
degrees of freedom and when we go into
the realm of trying to understand two
different populations coins
corresponding to their population
variance given the population mean
values are unknown then we will use the
F distribution again but the degrees of
freedom would be M minus 1 n minus 1 the
M minus 1 being fact that we are losing
one degree of freedom from the first
first population and n minus 1 being the
case when we are using the degrees of
freedom for the second population so
this concept would come and the
corresponding distributions which we
have for the T we had the student T
distribution multivariate case then the
video distribution was basically
replacement of the Chi square which we
discussed and if you remember I did
mention very fleetingly
and though it is absolutely true that as
we utilize the
concept of the the sample mean and the
sample variance are independent for the
case of the normal distribution this
will continue to be true in the case of
the multivariate case also which means
that Weiser distribution and the the
mean value measured if you are
corresponding to the standard deviation
then later on we we came into trying to
utilize different other distribution
that means the possession the
exponential and hypergeometric other
things considering the the sample set of
observation is large and we can easily
replace them their mean values those
distribution mean values and the
variance in the case with the normal
distribution and still continuing with
our work now this was also true in in in
sense for the normal case for the
multivariate distribution and the
multivariate distribution we did discuss
the simple examples of multinomial
distribution and all those things which
we have seen now later on we found out
that after the proofs and few of the
results we went in the concert of
Coppola theory Coppola theories
basically the linkage function which you
want to find out using the marginals and
the joint distributions of the of the
random variables x1 x2 x3 and so on and
so forth and consider you have basically
pete or k number of random variables and
you want to find out what is the couple
living cases it's basically a marginal
distributions concepts and obviously
coppola comes into the picture when we
want to find out something to do with
the independence structure at the
extremes i had drawn that if you
remember now when we come to the in the
concepts of trying to find out later on
for the independent structures the first
different type of ideas we considered
was basically the principal component
analysis now the principal component
analysis which we went a little bit slow
and i would really like you to
understand that this was the first case
that we are basically going through only
the formulas and trying to basically
utilize this formulas not derivation the
concepts and and try to utilize this
for the problem-solving so the main idea
was basically in principle computer
analysis you are given a set of four
data and some random variables x1 to XP
and we will consider this x1 to XP are
are not independent of each other there
is independent structure so obviously
they would each of them would have a
mean value each of them would basically
the combined variance covariance matrix
now our main question would be that is
it possible to find out a certain set of
convex combination of this random
variable x1 to XK or x1 to XP says that
we can find out the maximum variability
step by step and basically formulate
those variabilities in some orthogonal
direction to each other so that we are
able to explain the variability to the
maximum possible extent using the
combinations of x1 to XP which are all
orthogonal to each other so it looks
absolutely perfect conceptually that
means we want to basically big break the
dependence of the variability into
orthogonal direction and obviously the
next question would come that how you go
into that will take basically the
variability for the case which is the
maximum then the next level and the
third level fourth level will go on and
so on and so forth so obviously it
becomes very intuitive that why not
basically break those principal
components in the orthogonal direction
can using the concept of con eigenvalues
and eigenvectors now the eigenvectors
would basically be give would give me
the so-called values depending on the
eigen vectors which I find out the cell
values will give me that in what
combination I should combine x1 to XP
such that I am able to get the first
principal principal component which is
y1 then the print second principal
component is y2 and so on and so forth
till the pH 1 so obviously these y1 to
YP would all be independent of each
other and we can find out what is the
variability so the variability is found
out and when we find out the variability
of Y 1 or Y 2 or Y 3 then we found we
calculated obviously this those will be
calculated by using the covariance
covariance matrix which I have for x1 to
XP you find out the variable to the
to the eigen vector values and obviously
they would give us the information that
the the orthogonal T actually the
breakages of the orthogonal vectors of
y1 to y2 2 YT 3 white 4 on alright and
also we found out what is the
variability so the variability or the
actual variance for y1 with the highest
then the y2 and y3 and it will decrease
continue no decreasing we use this
example for a 3 by 3 matrix and
basically proved on the orthogonality
the concept of how eigen values and
eigen vectors would be utilized later on
we come came to the factor analysis in
the factor analysis what we are trying
to do which we'll continue discussing
now on further also will continue that
the factors are are decided such that
will break them into two parts one of
the factors which which are basically
have their effects and one will
basically be the white noise now we will
also consider that white night concept
we didn't actually consider to be
existing in the even though it was
extreme we basically assumed that they
could be known in the factor analysis we
will try to utilize the concept of the
factors along with an error terms and
basically procedure accordingly again
the same concept of the eigenvalues and
eigenvectors would be used the
eigenvectors would be denoted by lambda
1 lambda 2 lambda 3 lambda p and
obviously the eigenvectors direction
would be given by the bold II 1 then e 2
then e 3 E 4 and so on and so forth
where e 1 would basically be the
eigenvectors corresponding to lambda 1 e
2 the diag invectives corresponding to
lambda 2 now we we were basically
discussing that we'll come to that later
on I am sorry that had to basically
recap all this thing because the concept
which are being utilized discussed may
be a little bit intense so I thought
I'll go a little bit slow in order to
basically clarify the points em as may
be necessary for the students another
point which is are also nothing to
relate it to the discourse directly but
technically this being a 12o UV codes
which is about 60
just thirty hours so we'll basically
have have a one-to-one session with the
students and the instructor would would
me and obviously there would be queries
and we will try to basically split the
set of queries which which would have
been basically after the first four
lectures then next four lectures and so
on and so forth which is four eight and
twelve but after the eight lake lecture
which were started obviously there would
be a weekend or a weekday in the evening
where based on the the communication
which will be sent from the NPTEL office
I will definitely be available to answer
queries
both for the first for a set of lectures
and the next set of forum lectures which
will basically eight lectures and then
depending on how your queries and
whatever the queries in the forums
we will to basically try to tackle all
the queries which may be coming up both
from theoretical point of view as well
as the practical point of view and then
on the end of the twelfth lecture we
will have the third session of
one-to-one video conferencing and I'm
sure in in both considering the
assignments which will be twelve in
number also the three different one-hour
sessions minimum of one hour sessions
which we plan to have with the students
for the first four then the next four
and the twelfth one and twelve lectures
I'm sure and also obviously they would
be twelve as assignment so this is the
eight week starting so you will
definitely have the salt at least seven
set of assignments and by the time when
you are basically coming for the
one-to-one sessions you definitely would
have reached it received the eight week
assignments and after they assault you
can definitely have queries accordingly
so the reason which I am why I am
spending so much time in the first part
is basically to discuss what we have
covered and the next part in order to
basically tell how things will be
handled from our end in order to answer
the queries was because in the forum
there are many different questions which
are coming up which may be a little bit
easy but unfortunately due to either
positive time from your end or
on the wave the the whole concept of the
course are considering that a huge set
of coverages are to be done do justice
has to be done from your end in the
sense that spend some time for with the
concepts clear our doubts read daily
later the books if you remember I have
given you a lot of books and the
references and still if you are not able
to do that after solving the assignments
and trying to basically get your queries
answer in the forum and also after the
one to one session video conferencing
still if you have some questions you are
most welcome to basically collate them
and send on the forum we will definitely
take our time out and answer all the
related questions which may arise for
this course okay so I'm basically
continuing with the factor analysis
sorry for a little bit more exhaustive
introduction and what things we should
be planning in order to basically cover
the course in a much better way so in
let me read it and throw the factor
analysis I have come back to the course
so you the number of common factors that
means the factors based on which we will
try to basically do the analysis is not
known beforehand or is not determined a
priori
one can use the knowledge of previous
researchers so the set of information
which will be there from by the previous
registers can be utilized in order to
answer the question a thumb rule is to
find the value of the residuals in the
matrix s s if you remember either said
the variance covariance matrix from them
or the square root of that from the
point of view of the sample because we
are nothing to do with base the
population so L 1 l l1 l2 l3 till the M
the values of L which will basically be
till M because we are taking em out of
the p number of such factors so they
would basically would have two sets one
is the known level of dependence one is
the white noise so that should
technically be in the long run if the
divisions of the amount information
so-called how should I put it the
information being relevant from the
point of view very able to give the
the reasons for those aberrations and
one set of white roses for which you are
not able to give the informations so
they should technically be equal to the
the sum of the eigenvector eigenvalue
square so a small value of the sum of
the squares of the and
eigenvalues means a small value of four
the sum of the squares errors of AK
process approximation so obviously
depending on the problem we are able to
answer it or not will come out from that
value the contribution to the total sum
sum sample variance will technically be
if you consider the principal diagonal
so the first dye of value would be s 1 1
second won't be s 2 2 which is basically
the counterpart from the population if
you remember it will be Sigma 1 1 which
is the way covariance of the first with
respect to itself then the 2 comma 2
element would be Sigma 2 2 which is
basically s 2 2 so Sigma 2 2 is
basically the covariance of the second
with itself until the last element which
in the principal diagonal will be s P P
which is basically Sigma P P which is
base thickness all these are suffix s
sorry
- take so which are basically the
covariance of the pH vector with itself
so they so this is the total sample
variance which is s 1 1 plus s 2 2 plus
dot dot till s PP from the highest
common factor is given by this I so
these are basically coming out from from
the values of the lambda so they would
basically be given by I square when I 1
square plus I 2 square plus I 3 square t
like p square so p is in the suffix and
the squares are on the peas are in the
pre from in the base and they would
basically be power in the square so they
technically they should be equal to the
values of lambda 1 plus lambda 2 plus
lambda 3 multiplied by square root of
them multiplied by there I invested
estimated value the proportions of the
total sample variance due to the J
affected would be given by the ratio of
the eigen values divided by the sum
for the principal diagonal which is is
one one till s P P such that depending
on whether it is to be is to do with s
or are s would basically be
corresponding to the case when we have
the sample and R would be the case when
we have corresponding to the population
value so the part I am what I am saying
is that population means that I have the
counterpart from the population which is
the correlation coefficient so based on
that we can basically do the
calculations so what is the general plan
of working for the factor analysis so
what first will generate the variance
covariance matrix of the observed
variables that is s which is basically
the estimate of summation or the MU
value and this capital new value is
basically the variance covariance matrix
of size P cross P once we have that we
will basically select the number of
factors that is out of this P obviously
M would be less than equal to P by
finding out the lambda hats so lambda is
basically vector and which is the eigen
values and E's baldies are basically the
eigen vectors so technically the e1
value would be corresponding to the
first element of the first vector then
corresponding to the second value would
be in the first eigen vector would be
basically e 1 2 then e 1 3 e 1 40 e 1 P
then similarly we will basically have e
2 e 2 1 e 2 2 e 2 3 te e 2 P where the
first value here would basically be
corresponding to the the eigen vectors
which is the second level command for
lambda 2 now with with respect to the
estimates from the sample size n we get
it in general trying to find out as I
mentioned lambda hats mean basically
estimates divided by the sum of s 1 1
till SP P would be utilized when we have
basically the standard error of the
sample and in case we want to basically
divided by the value of P
that means summation of lambda J's
divided by P would come be coming out
when we have basically the the
correlation matrix corresponding to the
problem of the sample so we will extract
our initial set of factors and based on
that we will try to find out F 1 F 2 F 3
until FM remember that what we are
trying to do is that trying to utilize a
set of factors M which is less in size
with respect to P and utilize them in
such a way that we get the maximum
amount of information which is plausible
and the other type of white noise would
basically be putting into the the error
term will perform the factor rotation to
a terminal solution so basically we will
standardize them rotate the factors as
that they would both canal to each other
again the same concept being taking up
where we try to utilize the set of
information which is coming up from the
eigenvalues and eigenvectors and
rotations are being done so that the
maximum variability plausibility is
there and once it is for the first set
is fixed will basically keep that and
then try to find out the rotation of the
second set with respect to the first
that they are orthogonal to each other
so we'll keep rotate treating it in this
way such that each of them are
orthogonal and the variabilities are
basically being um being subsumed or
coming out from the discussion the
factors which are M in number M if you
remember is less than equal to P so once
we find out the factors so we have the
variance covariance matrix from the
sample which is s and that would
basically from that will try to
basically subtract to two sets of
information one set of information with
the factors information which is
plausible and for which we do have some
some some formulas to say that yes the
factors are coming out from the the M
number of so called back vectors
corresponding to the lambda those eigen
values lambda 1 lambda 2 lambda 3 and
obviously the eigen vectors would be
they're given as e 1 e means cap all
these are capital e 1 e 2 e 3 and they
would basically be a set of white noise
which will try to basically captured
using the peak row
P matrix of capital fine so once we
construct this you will basically have
now in this case of the of the principal
component we have the principal
components as that they're orthogonal to
each other that it had nothing to do
with the error terms but now and
obviously because they're the convex
combination we were being utilized
because we consider the relationship was
really linear based on this we proceeded
so we'll consider the cost factor scores
in such a way that the first set of
factors which will we have would
basically be the convex combination
corresponding to the first factor which
will be L 1 1 into F 1 plus L 1 2 into F
2 so on and so forth till the M F 1
which is L 1 M into M F M so this whole
set which will we have this will give me
the plausible reasons which are for
which we I can assigned reasons and this
would be the the error part similarly
when we go to the second one it will be
X s
so if I have this and highlight this is
the error term this is all the error
term of the ph1
and the factors for which we will have
the information is for the second for
the first and for the last one so we are
trying to divide the overall variance
covariance into two parts the
variability one on the factors which are
possible and nama is the white noise so
with this I will close the 41st lecture
and continue discussing more about
factor analysis and multiple linear
regressions later on have a nice day and
thank you very much
[Music]
[Music]
[Music]
