Lecture 21 : From Distribution Functions to Probability Measures I

IIT Kanpur July 201
20 followers
1 Video View·Dec 2, 2023
Welcome, to this lecture this is the first lecture of week 5. So, before we proceed let
us first recall what we have already covered. So, we have crossed the half way mark of this
course in week 1 we have discussed about measurable spaces, in week 2 using that information from
week 1 we considered measure spaces and in particular probability spaces, then in week
3 we considered measurable functions, then using this information from measure spaces
in particular probability spaces and measurable functions on top of this probability spaces
which we called as random variables, we defined the law or distribution of random variables
in week 4. So, then in week 4 we considered this law and conversely we have also shown
that given this law which is a probability measure on the real line for the case of random
variables or R d for the case of a R d valued random vector.
So, given such a probability measure you can always construct a probability space and random
vector with that specified law. So, we had obtained this correspondence between random
variables or vectors and probability measures. So, we had seen that correspondence. Then
from probability measures we went to a class of functions which we called distribution
functions corresponding to these probability measures and using that connection we had
said that we will define the distribution function of a random variable or random vector
to be the distribution function corresponding to its law.
We had stated the connection between random variables and vectors and probability measures.
So, that was both way correspondence, both way connection, both way correspondences,
but for the case of distribution functions we had only obtained the one way connection,
one way association from the probability measures to the distribution functions and we had remarked
that we are going to discuss the other way connection soon. So, this is what we discuss
in week 5. So, we are going to see that the standard properties of distribution functions
by that I mean that it is a function defined on the real line takes values between 0 and
1 is non-decreasing, right continuous, limit at infinity is 1, limit at minus infinity
is 0. So, these are the standard properties of the distribution function corresponding
to a probability measure. So, given such a function which a priori need not be connected
with such a probability measure we are going to construct a probability measure and that
will finish that correspondence between the probability measures and distribution functions.
Of course, whatever I have just said is applicable for the dimension 1 case meaning for real
valued case. However, you can extend all these ideas in higher dimensions in Rd. In first
two lectures we are going to discuss the one dimensional case and then we shall appropriately
extend these ideas and these results to higher dimensions and that we shall discuss in the
later part of week 5. So, in the first two lectures we are going to focus on the one
dimensional case. So, as before let us move on to the slides.
So, in the previous week we had discussed this correspondence between random variables
and vectors and probability measures and we had also defined this distribution functions
corresponding to probability measures and that connection we had used to define distribution
functions of random variables or vectors. We are now going to consider a class of functions
which we are going to refer to as distribution functions. Now, this class of functions are
defined independent and without any a priori connection with any probability measure. So,
this is the idea. So, we will first start with certain properties of a class of functions
with which there is no a priori connection to probability measures, but we are going
to show that we can construct a probability measure and this function will turn out to
be the corresponding distribution function with respect to this probability measure.
So, we are now going to discuss this construction of probability measures from this class of
functions. So, this is going to complete that appropriate justifications for the results
and facts that were mentioned earlier in week 4. So, you can refer to notes 8 and 11
in week 4 notes. So, as mentioned in the introduction we are going to focus on the one dimensional
version of this connection meaning given this class of functions we are going to construct
appropriate probability measures and this is going to give us all possible probability
measures on the real line defined on the Borel sigma field.
So, let us first start with this basic definition. So, you consider a function defined on the
real line taking values between 0 and 1 and you are going to refer to such a function
as a distribution function if the function is a non-decreasing right continuous and has
this appropriate limits meaning limit at infinity is 1 and limit at minus infinity is 0. So,
you call such functions as distribution functions. So, as you see here there is no a priori connection
with any probability measure. So, in this definition no probability measure has been
mentioned no random variables has been mentioned, but then note that all distribution functions
that you can find corresponding to probability measures will satisfy this properties. So,
therefore, all distribution functions of probability measures are distribution functions in the
sense of the above definition and what we are going to discuss what I would like to
show is that all distribution functions actually give you all probability measures in this
connection that means that all distribution functions as considered above again is of
this form. As mentioned earlier in this lecture we are going to discuss the construction of
probability measures from distribution functions and in fact we are going to see that given
this distribution function once you construct the probability measure this will also allow
you to make this identification that all distribution functions appear as the distribution function
of a probability measure of this type. To do this we first take some motivation from
distribution functions corresponding to probability measures. So, let mu be a probability measure
defined on this measurable space real line together with the Borel sigma field and recall
that the corresponding distribution function has some nice properties. So, the first property
that we are going to focus on is the value at infinity and minus infinity. So, let me
rephrase we are going to focus on the limits at infinity and minus infinity. So, here we
already know that the limit at infinity exists and its 1 limit at minus infinity also exists
and its value is 0. Now what we do is that we think of the distribution
function as a function defined on the real line. So, that is as per the definition but
now what you can try to do is to extend the domain by adding the two points plus infinity
and minus infinity. So, what is the domain again the domain is the real line we are adding
these two points plus infinity and minus infinity and therefore we are going to get the bigger
domain which is the set of extended real numbers. And now let us say we want to define or extend
the given distribution function on the extended real line. So, we have to associate the values
for the points plus infinity and minus infinity. What you can choose to do is to associate
the value 1 the limit value at infinity to be the value of the function at infinity and
similarly look at the limit at minus infinity which is 0 you associate this value to the
function value at minus infinity. So, you are going to associate the values plus 1 to
infinity they associate the value 0 to the point minus infinity. Once you do that you
can get the function extended to the extended real line. So, you have just had to define
the corresponding values to this extra points plus infinity and minus infinity and thereby
we obtained this distribution function on the extended real line. We are going to see
the usage of this later on in the construction itself but this is a quite a useful observation
that by using the limit values you can extend the function defined here.
But then you have this interesting connection that the increment of the function values
for the points a less than b increments of the function values is exactly the size or
weight or the measure of the interval open a to close b under the measure mu. So, this
we have seen earlier. So, again just to note that here we are following the notation that
a can be the point minus infinity and b can be the point plus infinity. However, mu is
defined on the real line so you have to interpret this interval accordingly. So, for example,
if b is infinity here so closed infinity you have to reinterpret it as open a to open infinity.
So, we are following those same notations that were considered earlier. So, the probability
measure mu is purely defined on the real line. So, that is the given probability measure.
So, we cannot talk about points infinity to be included in the intervals open a to closed
b and more generally for such type of intervals look at this a i open a i to closed b i for
i equals to 1 to n. If these intervals are pair wise disjoint you can now make this observation
that the size of this disjoint union of intervals is exactly given by this summation which can
be expressed exactly as the increments of the function values the distribution function
values. So, you look at this increments of the distribution function values and add them
up. So, that is what we are doing and furthermore you also observe that the measure of the whole
real line using the continuity from below we can show it to be 1. Of course, the probability
measure is defined on the real line, but then note that using the properties of the functions
we can show that the measure of the real line is whole of 1 is 1. So, how do you show this?
So, measure of the real line you use the continuity from below and apply to this kind of sets
open minus n to closed n. Then this sets increase to the whole real line. So, the size
of this sets can be represented as increment of the function values like this. So, look
at this function values now and take the limit. So, this limits now exist as per the assumption
as per the given properties of distribution functions and therefore, you end up with this
value to be 1 and this other limit to be 0. So, with that at hand you have that the measure
of the real line is 1. Of course, as we said that the measure of the real line under the
probability measure mu is exactly equal to 1. However, note that this one we are obtaining
through properties of the function. Now, what is the need for this observation? So, let
us go back once more. So, we said that measure of this type of intervals can be obtained
as increment of the function values. More generally, if you have finite disjoint union
of left open right closed intervals then the measure of that can again be written in terms
of the function values and then finally, we are saying that the measure of the whole real
line can be written and computed by the properties of the function values. So, why you are stressing
on the function values and the properties of the function it will be cleared in a moment.
So, what we are basically trying to say is that given the function which appears on the
right hand side of all of your expressions, we can recover properties of the measure and
using this we hope to construct probability measures from the class of functions that
we have started off with. So, here what we know about is that if mu is a probability
measure then we are obtaining all these properties of the function from the probability measure
but using those properties you can recover the measure of these sets. So, let us use
these ideas, this identification we are going to use and construct the appropriate probability
measures. So, in this regard it is an important observation that if mu and nu are probability
measures on the real line such that their distribution functions are the same that means
that for each point on the real line the distribution function of mu and distribution function of nu
both agree, they match. If that is the case then you can show that the measures must also match
that means for all Borel sets A, the measure of A with respect to the measure mu and measure of A
with respect to measure nu will match. So, you can restate this statement in words like this
that a probability measure is determined by its distribution function. But then there is another
way to state the same result in this exercise. So, it is the following. So, consider these two
probability measures once more let mu and nu be probability measures on the real line then you say
that these two probability measures agree on a class of sets which class of sets is exactly
minus infinity open to closed x. When you say that this agree that means that
for all such sets in your class this specified script epsilon measure of A under mu and nu will
agree. So, in particular note that your sets A are exactly of this type of intervals and therefore
they exactly give you distribution functions. But here we are not using the term distribution
functions we are referring directly to the measures and we are saying if the measures agree
on this collection of sets meaning if the distribution functions agree under the hood
then the two probability measures will match. So, that is just a restatement of the previous
exercise. Once you work that out you will immediately get this statement. So, with that
at hand we are now ready to start the construction of probability measures from distribution functions.
So, let us start with a distribution function. So, we are still working in the dimension one case.
Now we want to construct a probability measure corresponding to the given distribution function.
So, for the duration of the construction we are going to refer to the constructed
set function which will be a probability measure to be mu subscript f just to denote the dependence
on the function capital F. And we want to construct it on the this specific measurable space
real line together with the Borel sigma field. But then before defining this probability measure
what we need first is that mu subscript f should be a well defined set function
on this measurable space. So, and to get a set function what you need to specify
are the values or the sizes for all this type of Borel sets capital A coming from the Borel
sigma field. So, you need to figure out the values mu subscript f of A. Moreover once you
have specified the set function then you need to check the properties of a measure which is that
that set function is non-negative and countably additive. Finally if you want to say that this
set function thus constructed is a probability measure then you also need to check that the
size associated to the whole set is 1. So, once you have verified all these things you can
complete the construction. So, this construction rely heavily on the motivation from the node 2
above. So, in node 2 we had seen that for certain nice type of sets like left open right closed
intervals the size of the set according to the probability measure can be figured out from the
properties or the values of the distribution function.
So, therefore motivated by that we first do this we first think of the function f
as a function defined on the extended real line. So, a priori you are given all the values on the
real line but then you want to figure out the values on the extended real line. So, that means
you have to associate values for the points plus infinity and minus infinity these two extra points
in the domain and as motivated earlier you associate the value 1 at the point infinity
and associate the value 0 at the point minus infinity. So, therefore you get the function f
extended to the extended real line. We will see the usage of this but then our main target is to
define a set function mu subscript f which in preparation for it to be a probability measure
we should define that the size of the empty set is 0. So, we do that right at the beginning.
But then we want to specify the size of certain nice type of sets for example take this left open
right closed interval open a closed b. So, here a and b vary between these ranges. So,
these are the standard ranges that we have used and we are following the same notations for
example if b equal to infinity we are going to take the set open a to open infinity because we
are taking these sets open a closed b to be within the real line. But then we want to specify the
size of this left open right closed interval under the set function and what we do as motivated
earlier we will simply define it to be the increment of the function values. But then
in preparation for mu subscript f to be a probability measure we first of all require
it to be finitely additive and therefore for finite disjoint union of left open right closed
intervals what you should expect is that the size associated to such finite disjoint unions
will be simply the addition of the individual sizes. But the individual sizes you have defined
it as the increment of the function values. So, therefore the size associated to finite disjoint
unions of left open right closed interval should be this summation. So, this is as expected.
And finally in preparation for it to be a probability measure once more you require
that the size of the whole real line should be one. But then this is again I am motivating it
through the corresponding function values with the limits at infinity and minus infinity.
But then what did we do? We specified the values for the empty set and the whole real line then
we specified sizes for the left open right closed interval and then finally to all possible finite
disjoint unions of left open right closed intervals. So, therefore what we have actually
obtained is that this set function which we have just defined is defined on the field script c
of finite disjoint union of left open right closed intervals on the real line.
So, you now have a set function defined on this field script c. So, you will see the usage of this
immediately in the next step. So, observe that since f the given function f is non-decreasing
mu subscript f is a non-negative set function defined on this field script c. Why is this?
Let's go back to the all these values. So, empty sets gets associated the value 0 which is
non-negative. The whole real line gets associated the value 1 which is of course non-negative
and then look at left open right closed intervals or finite disjoint unions of this.
Since the function f is non-decreasing therefore the function value at b will be at least as big
as function value at a. So, therefore f of b is greater equals to f of a and therefore this
quantity f of b minus f of a is non-negative and in using that you can now show that mu
subscript f associates non-negative sizes to finite disjoint unions of left open right closed
intervals. So, therefore in step 2 we immediately observe that the mu subscript f the set function
thus defined is a set function which takes non-negative values and it's defined on the
field script c. Moreover by definition by the construction of mu subscript f it becomes
finitely additive. Why? Because if you are going to look at finitely many sets in the field which
are pairwise disjoint then what are the possibilities. So, first let's take care of the
case when one of the sets is the real line then to get finitely many pairwise disjoint sets you
need to take all the other sets as empty sets and therefore it's easy to check that the finite
additivity holds for the choices real line together with any number of empty sets.
So, that will be fine but then if you are going to use this kind of left open right closed intervals
or finite disjoint unions of them by construction itself you have put in the finite additivity
itself right because for pairwise disjoint such intervals you have built in the values to be
equal to the individual summation of the sizes of the left open right closed intervals. So,
therefore you can easily observe that mu subscript f thus defined becomes a finitely
additive set function on the field. Now you have a non-negative finitely additive set function
defined on a field we are going to claim that mu subscript f is continuous from above at the
empty set. So, we have a certain kind of a continuity for the set function thus defined
but remember that by theorem 1 in week 2 we have shown that a finitely additive set function which
is continuous from above at the empty set becomes countable additive on the field.
Therefore what is happening here is that once you are able to prove this claim
this continuity property for this set function you will immediately be able to say that mu
subscript f thus defined is a countable additive non-negative set function on the field C.
We are going to discuss this proof of the claim in the next lecture but assume this for now.
So, if you get this that mu subscript f is countable additive and non-negative defined
on the field script C we are now going to appeal to the characteristic extension theorem
which we had discussed just in the previous week in week 4 and therefore we will be able to extend
this set function uniquely from the field script C to the sigma field generated by script C
and which is nothing but the Borel sigma field. So, therefore it has a unique extension to the
Borel sigma field and you are going to get this as a probability measure. So,
the extension will be a probability measure. So, for the sake of not complicating the notation
we are going to continue using mu subscript f to denote the same set function. The extension again
we are going to denote it by mu subscript f. So, by step 4 using the characteristic extension
theorem we can extend this countable additive non-negative set function from the field
to the generated sigma field and therefore we are getting this probability measure
on the Borel sigma field of the real line. This is completing the construction.
However, this is not the end goal. We had started off with the motivation to connect
the identifications between the distribution functions and probability measures both ways
and then which we are now going to observe is that this construction is going to give the fact
that the probability measure thus constructed mu subscript f the probability measure thus
constructed is going to have the distribution function as the given function. So, why is this?
So, again go back to the definition of mu subscript f and we want to check what is the
distribution function of mu subscript f. So, let us go back to the definition. So,
look at this left open right closed intervals open A to closed B. So, in particular A can be
taken to be minus infinity and B to be some real number and for that using the notations that we
had mentioned earlier. So, we are going to get the interval minus infinity open to closed B and for
that what is the size of this? So, mu subscript f associates the size as f of B minus f of minus
infinity but f of minus infinity is 0. So, therefore you just get back the function value at
B but what is the left hand side that is mu subscript f of minus infinity to closed B.
So, therefore the distribution function of mu subscript f is the given function f itself.
That is nice. So, we have that if you start with the distribution function go to the corresponding
probability measure by this construction and then go back to the corresponding distribution function
you are going to get back the original distribution function but this is not the
end of the story. You can start off with now a probability measure that is mentioned in note 5.
You now start off with a probability measure again on the this measurable space real line
together with the plural sigma field and look at the corresponding distribution function f
subscript mu. This we have discussed extensively in the previous week but now you know that this
distribution function f subscript mu is a distribution function as per the definitions
in this lecture. It satisfies all those nice properties. Therefore, you can say that you can
now construct a probability measure corresponding to f subscript mu. So, call this as mu subscript
f mu as per the notations of this lecture. So, given this distribution function that you have
somehow got you can now use this construction that we have discussed to construct a probability
measure mu subscript f mu but then observe that by the previous note mu subscript f mu
has the distribution function f mu itself. Of course, the given probability measure mu
has the distribution function f mu. Therefore, both these probability measures
have the same distribution function which is f subscript mu and hence using the exercise
mentioned earlier that a distribution function uniquely identifies the probability measure
you will immediately claim that mu is nothing but mu subscript f mu. So, if you therefore start
with a probability measure mu go to the corresponding distribution function and then
go to the constructed probability measure you will get back the original probability measure.
So, this is the flip side of it. So, in note 4 we said start with the distribution function
go to the probability measure then go to the distribution function you get back the original
distribution function. In note 5 we are saying start with a probability measure go to the
distribution function go to the corresponding probability measure and compare final probability
measure with the first one you are going to get back the same thing. This completes the
identification between the class of distribution functions and the class of probability measures
on the real line. So, we had skipped one of the steps in the construction that was regarding
the countable relativity of the set function mu subscript f. So, we have to verify that through
the continuity property that we had left as a claim. This is what we are going to discuss in
the next lecture and later on in further lectures we are going to discuss the extension of all this
construction to higher dimensions. We are going to continue this discussion in the next lecture.
