noc21- ma35 - lec45

IIT Kanpur July 201
20 followers
Dec 3, 2023
Hello friends, welcome to the course essentials of data science with R software 1 in which
we are trying to understand the basic concepts of probability theory and statistical inference.
So now you can recall that in the last 2 lectures we had discussed the bivariate probability
distributions when we have discrete random variables.
And in the last lecture we discussed the implementation in the R software.
Now in this lecture I am simply going to extend the concept that we have learned for
the case of discrete random variables to a continuous random variable, right.
And as I discussed towards the end of the last lecture that how are you going to handle
the data on such variables in real life that we already had covered.
So I need not to show you once again that how are you going to do such an analysis in
the R software.
So my objective in this lecture is merely to give you the expressions of the same quantities
that we had understood in the earlier lecture like as probability mass function, marginal
probability, conditional probability functions.
Now I will try to define them in a different cut up when my random variables are continuous.
So we will have the concept of joint probability density function, marginal probability density
function, conditional probability density functions and so on.
And you know one rule that whenever we are trying to deal with the discrete random variable
we are trying to obtain different probabilities by summation.
Summation over x or summation over y or summation over x and y etc.
But when we are trying to deal with the continuous random variable we simply have to replace
them by integral sign but appropriately, yeah you cannot close your eyes and just replace
it, you have to do it following the mathematical rule.
So let us begin our lecture and I will try to give you all the definitions and concept
that we have discussed in the case of discrete random variable.
Now for a continuous random variable, okay.
So suppose we have here two random variables capital X and capital Y which are now both
continuous.
For the joint probability distribution of X and Y can be specified by providing a method
for calculating the probability that X and Y assumes a value in any region R of two-dimensional
space.
Well do not get confused with this R that is the R software, this is the space of real
numbers, right.
So if you try to see we already have defined the concept of probability density function
where we have given this interpretation that if you want to compute the probability of
an event then it is going to be an integration over the range of the variable in which the
event is going to occur.
So similarly, the concept of probability density function of a single continuous random variable
can be extended to a joint probability density function and can be defined over a two-dimensional
space or a multivariate dimensional space.
So now first we try to define the cumulative distribution function but in the case of joint
probability.
So we have the definition of joint CDF, right.
So a bivariate random variable X, Y is continuous if there is a function f X, Y is such that
you have to keep in mind that we had earlier denoted by small f the PDF and by capital
F the CDF, right.
So the same symbol will continue here also.
So this joint probability density function is such that that capital F X, Y that is the
CDF or the joint CDF of X and Y at the values small x and small y which is actually the
probability that X is less than equal to x and y is less than equal to y.
So this can be obtained by the double integral minus infinity to y and minus infinity to
x and the joint PDF f X, Y dx dy holds, right.
So this f X, Y this is the joint CDF of X and Y and if you try to see we have just extended
the definition of a univariate CDF to a multivariate CDF.
The only thing will be that instead of here having a univariate integral you are going
to have here bivariate integral and double integral and in case you are going to deal
with more than 2 variables you simply have to solve a multiple integral that is all.
The main thing is this what are going to be the limits that is the most important part
whenever you are trying to deal with multiple integrals, okay.
Now you can recall that we had given couple of conditions for a CDF and similarly we have
here condition for the joint CDF of X and Y.
They are as follows that limit X tending to minus infinity f X, Y, X, Y should be equal
to 0, limit Y tending to minus infinity f X, Y, X, Y should be equal to 0, limit X tending
to plus infinity limit Y tending to infinity f X, Y, X, Y should be equal to 1, right and
if you try to see these are the simple extension of the same definition that you use in the
case of univariate, right.
There is one more condition here that f of X 2, Y 2 minus f of X 1, Y 2 minus f of X
2, Y 1 plus f of X 1, Y 1 should be greater than or equal to 0 for all X 1, X 2, Y 1,
Y 2 such that X 1 is smaller than X 2 and Y 1 is smaller than Y 2 and this is called
here as a rectangle inequality. Actually, this is like this that here there is an X
1, X 2 and there is an here Y 1 and here Y 2. So, this is the area and this inequality
is simply trying to ensure that the that this quantity here that is going to indicate this
area and obviously, this area has to be always greater than or equal to 0, the area cannot
be negative. So, this is what it is trying to ensure, right.
Now, let me try to take here one example and try to explain you how are you going to compute
it, right. So, that will give you some idea. So, suppose the joint pdf of X and Y is given
by this quantity f X, Y and which is e raised to power of minus X plus Y when X is lying
between 0 and infinity, Y is lying between 0 and infinity and 0 elsewhere. So, if you
want to find out here the joint cdf that will be the integral minus infinity to X minus
infinity to Y and exponential of minus t 1 plus t 2. So, this t 1 is corresponding to
X and t 2 is corresponding to Y. So, this will become here d t 1 d t 2 and if you simply
try to solve this integral, well I am not going to show you here, but I believe that
you can solve such integral without any problem. This will come out to be 1 minus exponential
of minus X into 1 minus exponential of minus Y for X and Y being lying in the interval
0 to infinity and 0 otherwise. So, you can see here this is how the joint cdf will look
like, right. Now, we come to the aspect of marginal probability
distributions, right. Now, I do not think that if there is any need to explain what
are the marginal probability distribution because we already have understood them. So,
now whatever the approach you adopted in the case of discrete random variable in defining
the marginal probability distribution, a similar approach is going to be adopted in the case
of joint pdf also, right. So, if we assume that our joint pdf is f x y, then we simply
have to define the marginal distribution of X and marginal distribution of Y and we will
see that we are simply going to replace the sum by integration, but you have to do it
following the rules of mathematics, right. So, the first question comes here that what
is the joint pdf. First, we try to define or give a formal definition of the joint pdf
and then we will come on the marginal pdf. So, do you remember that when we defined the
joint probability density function in the case of univariate continuous random variable,
the same definition is going to be extended over here also.
So, suppose if I say that there are two continuous random variable f x y and their joint probability
density function is going to be denoted by f x y x y. So, small f is going to indicate
the probability density function. So, we assume that f x y is greater than or equal to 0 for
all choices of x and y and minus infinity to plus infinity minus infinity to plus infinity
f x y x y d x d y should be equal to 1. That means, earlier you had assumed that the total
area under the probability curve is going to be 1. Now, the case is going to be in two
dimension, right and then for any region A in a two dimensional space of real number,
the probability that x y belongs to some region A, this is going to be the integration
over the range of the region of A and then f x y d x d y, right.
So, this typically this f x y is defined over all of two dimensional space by assuming that
f x y x y is equal to 0 for all point for which f x x y is not specified. That is the
same definition that you did in the case of univariate. Also, you define the probability
for example, if you try to recall the uniform distribution, you define the probability space
only between the two points A and B and beside those points the value of p d f was 0, right.
So, for example, if you want to compute here the joint probability that x is lying between
x 1 and x 2 and y is lying between y 1 and y 2, then this can be obtained by the double
integral y 1 to y 2 x 1 to x 2 f x y d x d y, right. Exactly in the same way as you did
in the case of univariate c d f that you used to integrate the probability density function
over the values of the random variable, right.
So, now about the marginal probability distribution, so that we have already understood that if
more than one random variables are associated with the random experiment, we distinguish
between the joint p d f and the p d f of each variable individually, right. So, this individual
probability distribution of a random variable is referred to as marginal probability distribution,
right. So, this marginal probability distribution function of x can be determined by the joint
probability distribution function of x and other variables in case if you are trying
to deal with more than two random variables, right.
For example, in case if you are trying to deal with three random variables, you have
a trivariate probability density function, then you have to remove two variables if you
want to find out the effect of one variable. For example, if you want to find out the marginal
of x, then you have to remove the effect of y and z. So, now we try to formally give
you the basic definition of a marginal probability distribution.
So, in case if you can recall the definition or the way we had introduced or understood
the marginal probability mass functions in the case of discrete random variable, the
same concept is going to be here. For example, if you have a probability joint probability
functions like a say p x y and if you want to find out here only the say here p x, what
you are trying to do? You are trying to sum over the y. Now, I will say simply you have
to replace this summation by integration and that is obvious. You are, you have a joint
effect and you want to find out the effect of one variable, you have to remove the effect
of another variable. So, thinking in this direction, the marginal
distribution of x can be defined as f of x, which is equal to integral over the joint
pdf f x y, but the integration is to be done over the range of y. So, this is the range
of y and then you are integrating it here with respect to y. So, obviously the effect
of y is being integrated out. So, now whatever is left that is the effect of x and this is
how you can remember that whether you want to integrate with respect to x or with respect
to y because many times students get confused. So, in this case, the integral is over all
the points in the range of y and similarly, if you want to define the marginal distribution
of y, then it is defined as the same quantity, the integral over the joint pdf, but now the
integration is with respect to x. Now, if you want to ask me that how are you
going to extend it. So, I will say here simply here, suppose you have a probability density
function of say here 3 variables say f of x y z and suppose you want to find out here
only f of x, then this is going to be integrated to over the value of y and z dy dz. So, that
is pretty simple. So, you can see here once you understand the bivariate extending it
to trivariate or any order or say multivariate setup is not difficult for you and definitely
that means you have to take care of the range of y.
So, now I give you one more concept. Once you can define here the marginal distributions
of x and y, why cannot we define the cumulative marginal distributions of x and y? That means
this marginal distribution is also going to be a just like another probability density
function, which is looking like a univariate probability density function and you earlier
defined the c d f for a univariate probability density function. So, now I can define the
c d f exactly on the this univariate probability density function, which we have called as
the marginal distribution. So, now it is very simple. You simply try to take here the marginal
distribution here f x and we try to define the cumulative marginal distribution of x
indicated by f of x and this is here the same thing means minus infinity to x integral f
x t dt and this is the same as limit y tending to infinity f of x y x y.
Similarly, you can define the cumulative marginal distribution of y as f of y y, where the integration
is going to be over the range of minus infinity to y over this marginal p d f of y with respect
to t and this is the same as limit x tending to infinity x y and so that this is the joint
c d f f. So, you can see here that it is not difficult. You are simply trying to look this
marginal distributions of x and y as an two individual probability density function. So,
whatever you have learnt in the case of probability density function, all those result theorems
concepts can be extended here.
So, now we try to consider the conditional probability distribution as you have means
found the marginal. So, similarly we can find out the conditional distribution also. So,
there are two continuous random variable x and y. So, I can define the conditional distribution
of x given y and y given x and what is their interpretation, how they are going to be used
that we already have discussed many times. So, I will say here if you want to define
here the conditional distribution of x given y. So, this will be f of x given y and definitely
this is going to be now here a function of x that you have to remember because out of
these two values x and y, now y is known. So, this is no more stochastic. So, this function
is going to be here function of only of x. This is what you have to understand in this
case. So, this is going to be defined here as say joint p d f divided by the marginal
p d f of y and obviously, this is f y y has to be greater than 0. This otherwise this
quantity cannot be defined and similarly the conditional distribution of y given x can
be defined as say f of y given x and this is going to be now function of here y because
x is fixed x is known. So, this is going to be joint p d f of f x y divided by marginal
p d f right. So, and here obviously, in this case f of x has to be greater than 0.
Now, if I ask you how will you find out the conditional distribution, it is very simple.
You will have the value of joint p d f, you can obtain the value of the marginal p d f
and then take the ratio means after following the rules of mathematics and then you try
to find out that function that is all. Now, once you can define here this conditional
p d f do not you think that this is just like a probability density function. So, this should
also satisfy the rules on the same lines as of p d f right. So, this conditional probability
density function will satisfy these three properties exactly on the same way as you
did in the case of simple univariate p d f that f of x given y x should be greater than
or equal to 0 integral over y f of x given y d y should be equal to 1 and the probability
that x belongs to A given y equal to y for any set A in the range of x this can be obtained
by here the integral over the range of x for the set A into the conditional p d f
f of x given y d x. So, this is what we have defined for the conditional probability of
x given y and similarly, you can define for the f of y given x that is the conditional
probability density function of y given x right and this and if you want to define it
for the discrete random variable exactly you can define in the same way right.
Now, once you have defined the joint random variable do not you think that we should also
define the expectation yes. So, now, we are going to consider the expectation in the case
of bivariate probability density function right. So, suppose g x y is a function of
random variables x and y. So, the conditional expectation of g x y for given y equal to
y is that expected value of g x y y given y it is equal to integral minus infinity to
infinity g x y f of x given y is equal to y d x. Now, here comes sometimes a confusion
among the minds of a student whether they have to take it here the range of x or y for
example, here we have taken here the range of x. So, if you try to see here there is
a very simple logic by which you can understand this variable g x y given y equal to y what
do you think this is a function of x or a function of y, y is actually here fixed. So,
this is only a function of x. So, when x is random variable you know the basic rule that
if you want to find out an expectation the expectation of x is a integration say f x
d x. So, you have to choose this variable here appropriately right. So, this I can write
down here that this is an integration over the range of x and similarly, if you want
to find out the conditional expectation of g x y for given x equal to x you simply have
to use the same definition, but you have to now integrate it with respect to the range
of y this is what you have to keep in mind. So, you are simply trying to treat this conditional
p d f marginal p d f just like a simple probability density function and the way you have found
different types of things they can be obtained here also right.
Now, I try to give you here some results which are very particular results related to the
such conditional distribution right. So, in case if you try to define here the expectation
of g x y given x equal to x that we have just considered we have now understood that this
is going to be a function of here x phi x one thing do not get confused that this is
the p d f of normal distribution this is some function. So, this is not the c d f of normal
0 1 remember one thing right. So, now the result here is that if you try to find out
the expected value of phi x this is going to be the same as the simple expectation of
g x y and how to find out the simple expectation of g x y that is going to be simply here integration
over the range of x and y g x y into f of x y and d x d y that is all right.
So, you can see here what is really happening you are trying to take the expectation of
a conditional expectation phi x is conditional expectation and now you are trying to take
this here expectation e operator here this is this e is unconditional expectation right.
So, you are trying to take an unconditional expectation of a conditional expectation
and this is coming out to be same as the unconditional expectation right that is a very important
result and this is also indicated here like this what I have shown you here that this
is expected value with respect to the random variable x and this is the expectation with
respect to the conditional random variable x given y equal to y and this is here e x
y this is the expectation with respect to x and y this is the joint p d f right.
So, now if you try to see I am trying to see here that first you try to take the conditional
expectation and then if you try to take once again the unconditional expectation of the
conditional expectation this is going to be same as the unconditional expectation of g
x y this is what I want to see here right. So, now in case if you try to take here g
x y equal to y then expected value of y is simply going to be expected value of a conditional
expectation of y given x and in case if you try to take g x y is equal to x then expected
value of x is going to be the unconditional expectation of a conditional expectation of
x given y right. So, now based on that if you want to find
out the variance of such a variable then the conditional variance of x is obtained here
in a different way right. For example, the in case if you try to simply want to find
out the the variance of say x in the case of the conditional distribution of x given
y then it is defined just exactly in the same way as you have done in the case of univariate.
For example, if you try to take here conditional variance of x given y equal to y this is you
just try to take it as a new random variable here z which is the conditional random variable
and simply try to define here expected value of x square minus expected value of x whole
square. So, this will become something like based on the condition of y given y.
So, you can write down here this something like z is expected value of z square minus
expected value of z whole square. So, this will simply be the expected value of x square
which is obtained from the conditional expectation or the conditional distribution of x given
y this is the expectation of x given y and by solving this you can simply compute the
variance. But now what I want to show you here is the
following that the variance of x in this joint distribution this is obtained as variance
of expected value of x given y plus expected value of variance of x given y this is a very
important result and it is very different from the univariate case. So, variance of
x is obtained as here variance of conditional expectation plus expectation of conditional
variance. So, this is something like variance of conditional expectation plus expectation
of conditional variance. So, this is what you have to use when you try to solve any
problem. Now, I come to the last topic of this lecture
where I would try to give you some idea about the stochastic independence. We already have
discussed this concept while dealing the concept in the probability theory that how do you
define the events to be independent at that time we said if there are two events A and
B then we say that the if the probability of A intersection B can be expressed as probability
of A into probability of B. Then the two events are independent and we also had discussed
the interpretation of this concept that the occurrence of one event is not going to affect
the occurrence of other event. Now, the same thing can be defined here that
the two continuous random variable x and y are said to be stochastically independent
if f of x y is equal to f of x into f of y. That means, the joint probability density
function can be expressed as the product of their marginal density function. So, this
is here joint p d f and this these two are the marginal p d f. If you can satisfy this
condition for a given joint p d f then the random variable x and y are going to be independent.
Now, at the same time I would also like to extend this definition for a discrete random
variable case also that if x and y are two discrete random variable then if their joint
probability that probability x equal to x i and y equal to y j can be expressed as their
marginal probability like probability of x equal to x i into probability of y equal
to y j. If this is valid for all values of i and j then we say that the two discrete
random variables x and y are independent or they are stochastically independent of each
other. The same definition can be extended or can
be written using the concept of c d f also. So, in terms of c d f if we have a joint c
d f of two random variables x and y like as here capital F x y, we say that x and y are
stochastically independent if the joint c d f can also be expressed as the product of
their marginal c d f. So, on the left hand side we have here a joint c d f and on the
right hand side we have here the marginal c d f of x and y.
So, now you can see that here I have given the definition of this stochastic independence
using the p d f as well as c d f. So, similar definition can also be given in terms of the
expectation of the random variables. So, two random variable x and y are said to be stochastically
independent if the expected value of x y is equal to expected value of x into expected
value of y. So, what you have to do? You have to just find out the joint expectation and
then you have to find out the expectations which are obtained from the marginal distributions.
Remember one thing this is obtained from the marginal distribution and this is obtained
from the joint distribution. So, from the joint distribution first you have to have
to obtain the marginal distribution and then you have to take the expectation that will
give you these values. So, if you can satisfy this condition then the random variables are
going to be independent or stochastically independent. So, all this definition whatever
you have given here in terms of p d f c d f expectation etcetera they can be extended
for any finite number of a random variable. For example, if I say here if I have random
variables suppose here x 1, x 2 suppose here x p and we want to check their stochastic
independence. So, in terms of the p d f you will have here
x f of x 1, x 2, x p, x 1 of x 2, x p this can be written as the product of i goes from
1 to p f of x i into x i that means the probability that the marginal probabilities are obtained
for each of the variable and then their product. And then if you want to write in terms of
here joint c d f like this then this has to be written as the product of their marginal
c d f's right f of x i see here x i. And similarly if you want to write down it in
terms of expectation then expected value of x 1, x 2, x p this can be written as a
product of i goes from 1 to p expected value of x i right. So, this is all about the stochastic
independence.
So, now if we come to an end to this lecture and you can see here that this lecture was
only theoretical and, but my idea was only to extend the definition whatever you have
learned in the case of discrete distributions. So, this is what I have done in the case of
continuous distribution. So, after this yes as expected I will try to take some examples
and will try to show you, but the reason why I am stopping here is that I have introduced
here couple of concepts. In the next lecture I am going to take some examples and I will
try to show you that how this values are computed. So, but it is very important for you that
before you understand those applications you have to understand their fundamentals.
So, I would request you that you please try to revise this lecture try to understand the
definition of all these quantities and in the next lecture I will try to take some example
and I will try to implement them. So, try to revise and practice and I will see you
in the next lecture till then good bye.
