Lecture 07 : Probability Measures

IIT Kanpur July 201
20 followers
Dec 2, 2023
[Music]
[Music]
welcome to this lecture
let us first recall what we have done so
far
so in the previous lecture we had
considered
how to look at sizes of sets in a sigma
field
and the motivation was that this will
help us
understand which events are more likely
to occur
or which sets in a sigma field are
more important and this we are
checking by the associated numerical
value
and if the value is larger then we will
say something is
more important or more likely to happen
than the other
so with this way or with this motivation
what we did we defined something called
set functions
and then we understood that for
disjoint sets pairwise disjoint sets we
should be able to add up the sizes
and with that motivation we looked at
the special class of set functions
which we called as measures so what were
measures
measures were non-negative countably
additive set functions
on sigma fields but then we clarified
that these measures may take the value
plus infinity
all right and with that we made this
very specific assumption that
there is a set with finite mass or
finite size
and that allows us to prove that measure
of empty set is zero
and using that we had proved the finite
additivity
of the measures and obtain certain other
basic estimates involving
the measure subsets or the sizes of sets
so with that in mind we now start over
and move on to the slides for this
lecture
so in the previous lectures this notions
of measures and measure spaces were
discussed
now let us recall this important
property that
we proved almost at the end of the last
lecture
so this was this fact that if mu is a
measure
on a measurable space then you pick up
any set from your sigma field look at
the size of that so mu of a is the size
of that set
according to that set function way of
measuring
so that is dominated from above by the
size or
mu of whole omega now right so omega is
the whole set
so what it says is that the maximum
value achieved by this
function mu is achieved at the
whole set and this is the maximum
possible value
okay and with this in mind we are now
going to look at the special subclass of
finite measures
so just to recall we had looked at these
two specific
subclasses of measures one was the case
when mu of omega the size of omega was
infinite
we called it infinite measure and the
other one
where mu of omega the size of omega was
finite
and we called it a finite measure so we
are now going to look at this special
subclass
of finite measures and this allows us to
make this next definition so what is
this
so we define that a menu
a measure mu is a probability measure
if mu of omega is one so we are looking
at
again the size of the whole set
according to the set function mu
according to the measure mu and we are
saying
that if mu of omega is equal to 1
then we are going to say that mu is a
probability measure
on omega script f that measurable space
now as we have seen earlier when you
have a measure
on a measurable space omega f then you
can consider the triple so
take all these three things together you
get a triple and we called it a measure
space earlier
but now if it so happens that mu
is a probability measure then we shall
refer to
this triple this measure space as a
probability space
and we are going to work on this kind of
triples this kind of probability spaces
throughout the course so again if
mu is a measure with the fact that mu of
omega
equals to 1 then we are going to call it
a probability measure
and we are going to look at the
corresponding triple corresponding
measure space
and also refer to it as a probability
space
now as soon as we make the definition
it is a good idea to look at some basic
examples
and to do that the first example and
this is a very important in
example in fact as we shall find out
is called the dirac measure on the real
line
so again just to recall the terminology
that was introduced in the previous
lecture
so we will call that a measure is on a
measurable space
as long as the sigma fields are
understood we can as well say
that the measure is on the domain omega
or equivalently we can also refer to the
sigma field so measure is actually
defined on the sigma field
so we can also refer to it as that
measure on the corresponding sigma field
script f
so on the real line most often we are
going to work with the purel sigma field
so if it is not stated always assume
that the bodel sigma field is taken on
the real line so when you are
considering the real line
if the sigma field is not clearly stated
please consider the bodel sigma field
now what we do we are going to define a
set function which we write as this
delta
subscript small x so we are going to fix
this real number small x first and
going to look at the corresponding set
function which we are going to define
now
so this is delta subscript x so this is
going to be a set function that is
defined on this measurable space
so in fact it is a of its a set function
defined on
the bodel sigma field so how do you
define this
it's pretty simple so the point small x
is fixed for you
a priori and then you vary your bodel
sets capital a
over the bodel sigma field so now you
are going to check this
whether your point small x is in the set
or not
if the point x is there in your set
then i assign the value 1 we say yes the
point is there in the set
otherwise we assign the value 0. so this
is simply
0 1 valued set function okay so
therefore it's also a non negative set
function by definition
correct so it just takes value 0 or 1.
now
this is pretty simple looking set
function but it turns out this actually
happens to be countably additive
and therefore it becomes a measure on
the
borel sigma field on the real line so
therefore
this is a very important example of a
measure which simply checks
whether the parameterized point that
small point small x
is in the arbitrary set capital a or not
if it is there then you put one
otherwise it is zero
all right so please check that this set
function is countable additive
and that will imply that it is a measure
so this is left as an exercise for you
please check this but as soon as it
becomes a measure you can now try to
look at
the size of the whole set which is the
real line here
so once you look at the size of the
whole set all you have to do
is follow the definition is the point x
in the real line
yes the answer is yes because x was
chosen like that
so therefore the answer to this value is
1
so therefore this becomes a example of a
probability measure
so as soon as a measure is with the
property that the measure of the whole
set is 1
then it automatically becomes a
probability measure as per our
definition earlier
so therefore you have a very nice
example of a
probability measure on the real line we
are going to refer to this measure
as the dirac measure supported at the
point small x
d dog subscript x or delta subscript x
so that is the d doc measure supported
at small x
now in an analogous fashion you can
actually extend this notions of
dirac measures on general measurable
spaces
how so all you have to do is that start
with some non-empty set omega
take a sigma field script f then for
any fixed small x in your domain
look at this set function doc or the
delta subscript x
for any arbitrary set coming from the
sigma field you just check
whether the point small x is in the set
a or not
if it is there you assign the value 1
otherwise 0 exactly as done before and
it will happen
that d dot subscript x or del delta
subscript x
that is the dirac measure it will turn
out to be a probability measure so again
please check
that this is countable additive and the
size of the whole set omega now
is also one ok please check this so this
is left as an exercise for you
please check this so the notion of
direct measures can be
extended to arbitrary measurable spaces
but most often we are going to restrict
our attention
to the real line as defined earlier okay
so this is on the real line so we are
going to look at this kind of a
structures throughout the course all
right
now with this example at hand one
concrete example at hand
we are now going to look at certain
properties of this probability measures
which will now allow us to construct
more examples
so once you have some explicit examples
at hand like the dirac measure
you can now use certain properties of
probability measures mix them up
and construct more examples so to do
that we state two propositions
so the first proposition says that you
take a finite measure first
you take a finite measure call it mu so
this is defined on this measurable space
omega script f
consider the set function this we are
now calling
as new so originally the measure was
mu which was a finite measure now i am
calling the new set function that i am
going to define as
new so how it is going to be defined so
its still defined on the
same sigma field that i have started off
with
but then i am going to look at the
values
of the sets or the sizes of the sets
according to the new measure
defined like this so what do we do we
look at the size of the
arbitrary set capital a with respect to
the original measure
mu and divide it by the total measure of
omega okay so here we automatically have
to assume that mu of omega is strictly
greater than zero otherwise this
definite division is not well defined so
we are not allowed to
divide by zero so for a finite measure
as we have started off with
we are going to assume that mu of omega
is strictly positive
is positive it is not zero and we are
going to divide or look at the ratio
of the size of the set with respect to
the whole set
ok so that is the ratio i am looking at
and automatically this
value is less or equal to 1 because mu
of a
the maximum value it can take is mu of
omega itself that we have proved earlier
okay so this ratio is false between
zeros and ones
so nu of a as defined actually takes
values between zeros and ones
okay now all you have to check that this
is a probability measure
so this is the claim of the proposition
so let us try to
go through this so by definition this
set function nu
is non-negative why because mu the
original finite measure
that was non-negative and you are
dividing by this
positive quantity mu of omega so the
ratio is of course non-negative
so that is not a problem so the next
thing that we need to check
for is countable additivity so that will
allow us to say
that nu is a measure so how do you do
this
so you choose a ns a sequence of arrows
design sets
coming from the sigma field script f
then
you look at the summation that you want
to check countable relativity you look
at the summation
so on your left hand side i am just
adding up the individual sizes according
to the new measure
so put in the definition so this one by
measure of the whole set according to
the mu measure comes out so that is a
common factor comes out of the
summation and you just have to add up
the individual sizes according to the mu
measure but since
mu itself is a measure this will have
countable additivity and that
allows us to write this ratio that on
the numerator i have
mu of the whole union countable union of
ns
so this is simply using the countable
additivity of the
measure mu but then i am dividing the
whole quantity by the measure of the
whole set
and then immediately i can write it
according to the definition
in terms of the new measure so according
to new this is simply the new
of the complete union or the countable
union a
n so therefore we have proved that nu
is a countably additive non-negative set
function and
as per the definition it becomes a
measure but then
as per our proposition we wanted to
claim that
this actually is a probability measure
so how do you show this
you look at the size of the whole set
according to the new measure
so what is this as per the definition
this is on the numerator
this you have to simply put that set
itself so that is mu of omega
but then you are dividing by mu of omega
as per definition
and hence this ratio is exactly one ok
so therefore this becomes a probability
measure and completes the proof
so all we are saying so let us go back
as soon as you have have some examples
of finite measures
divide the actual measure by this
constant
and then immediately what you get is
this ratio
which falls between zeros and ones this
automatically becomes a probability
measure
so therefore as long as you have
examples of finite measures
you immediately get examples of
probability measures
okay so that the first result that
allows you to construct more examples
and we will immediately apply this to
look at
some x in interesting examples of
probability measures
so the example is uniform measure on a
finite set
what we do we have to first of all
remember we have to first of all
work with finite measures so take a
finite set
omega call the points as x 1 x 2 x n
and look at the power 6 so power set
sigma field on top of it
and look at the counting measure on this
so remember
this gives you an example of a finite
measure because the
whole set has finitely many elements so
therefore
the mu of omega here is exactly equal to
n the number of elements and that will
be finite
so this is a finite measure so for this
finite measure you can now apply this
previous proposition and look at this
new measure
as defined there so what is this this
simply says
look at nu of a for any arbitrary set
which is now
any arbitrary subset of omega because
you are looking at the power set
so what do you do you look at mu of a
divided by the
mu of omega so the size of omega you
divided by
grid but since you are looking at
the finites at omega all its subsets are
also finite meaning all subsets are
finite number of elements therefore
as for the definition of a counting
measure what you get in the numerator
is simply number of elements in the set
a divided by the total number of
elements in the
set omega so you are just dividing that
number and dividing it by
n okay and as for the proposition
this new measure must be a probability
measure okay
so let us explicitly compute what
happens so here
if you look at more explicitly
on this non-empty set omega
you are looking at all possible subsets
in particular you can also look at
singletons
so if x1 x2 xn's where the individual
points that you have identified then
look at the singleton sets x i
so this set is a subset of omega so we
can look at the size of that according
to the new measure
and what is the number of elements in
this set its exactly one so therefore
according to the formula which we have
just
computed this ratio turns out to be 1 by
n
so for each singleton set you get the
value 1 by n
and now this gives you the uniform
measure
because it distributes the whole mass
the total mass is 1
it distributes the total mass total mass
1
into n equal chunks which is 1 by n
so you distribute according to the
number of elements so if number of
elements is
n you are just getting one by n weights
for each of these points ok so again
if you are having a double turn set
meaning a set with two elements
you will get the size as 2 over n so
that is what the meaning of
the uniform measure ok so this is a
explicit example of a probability
measure
which you have constructed out of a
finite measure
here it was the here the finite measure
was the counting measure
all right but now this is the
interesting way to look at this previous
example
so we say that we can look at or we can
write
the measure mu according to a formal
summation like this
what is this i am saying look at this
summation from i equal to 1 to n
1 by n d dot x i or delta subscript x i
dirac supported at x i
so we have looked at the dirac measures
and i am saying
that i am going to look at this
summation this formal summation
now you are going to ask what is this
formal summation so this means
that for all arbitrary sets capital a
from your sigma field which is the power
set here
we are going to have that the new of a
is simply
this summation what is this this is
summation from i
equal to 1 to n 1 by n that is the same
fraction that was written earlier
then direct measure supported at x i
acting on the set a so we are looking at
the size of the set arbitrary set a
according to the individual dirac masses
and we are just adding them up
with this scaling factor 1 by n
okay so now this is exactly the
same measure that was obtained earlier
okay
so that is another way to write down the
same equality
this is simply counting whether the
point x i is in the set a or not
but then if you check it for all
possible i meaning if you check
for all possible elements you are just
counting up the number of ones there
deduct simply associates the value 1 if
the point is there in
is in your set capital a or not so
therefore if you are adding up this
summation from i equal to 1 to n
you are just checking whether the points
x 1 x 2 x n
is in your set or not and the number of
ones is exactly the
number of elements in the set and
therefore you exactly get back the
ratio number of elements in the set a
divided by n
so that is the motivation for looking at
this kind of a summation
and formally we are going to write it as
a new as
a linear combination like this with the
weightage
1 by n to each of the dirac masses right
so this is an interesting way to
represent the new measure
but then this observation has
been put in the form of an exercise
please try to write it down
please check that this equality holds as
discussed now
but motivated by this example by this
exercise
we are now going to look at some kind of
combinations of such dirac masses
or more general probability measures
what do we do
we look at two probability measures call
them mu 1 and mu 2
both of them are defined on the same
measurable space
now fix a constant p
between 0 and 1. so it could be 0 it
could also be 1 or it could be a
proper fraction between 0 and 1 then
we are now going to look at a convex
combination
like this so for any arbitrary set
capital a
we are going to look at nu of a defined
as
the convex combination of measure
of a according to mu 1 and measure of a
according to mu 2 but we are going to
associate the weights p
and 1 minus v so this is a convex
combination of
weights so we are choosing this weight p
we are fixing this parameter p and
looking at this convex combination
p and 1 minus p so with that weights we
are going to look at the convex
combination of the size of
a according to mu 1 and the size of a
according to mu 2.
look at this convex combination now that
i am saying will give you a probability
measure
so let us quickly check this out so
again by definition
nu of a is non-negative why because
p and one minus p both are non-negative
mu
mu 1 and mu 2 are probability measures
so in particular
mu 1 of a and mu 2 of a both are
non-negative
so therefore this combination that you
are looking at is non-negative
so nu is a non-negative set function so
that is not a problem
but let us try to check the countable
additivity so how do you check this
again
start with the sequence of pairwise
joint sets in your sigma fit
then let's start with that right hand
side of the countable relativity
property
add up the individual sizes so for the
individual sizes
put in the definition so this is simply
will give you
these two individual series one for
the mu one and another for mu two with
the individual weights
coming from the convex linear
combination p and 1 minus p
so just put in the definition of the nu
of a n
and you will immediately get this series
on your right hand side
but then you observe that mu 1 and mu 2
are genuine probability measures
in particular they are countably
additive and therefore you end up having
that this is a convex linear combination
of the measures
of a the union of a ns according to mu 1
and mu 2. so here you are simply using
the countable activity
of mu 1 and mu 2 therefore
all you get is the convex linear
combination and as per the definition
this is nothing but the new of the
countable union of ends
so again you have managed to prove that
nu
is countably additive and hence this
becomes a measure
but then our original claim was that
this measure
is in fact a probability measure so how
do you check this
all you have to check is the measure of
the whole set so we are going to look at
the size of the
whole set omega so nu of omega what is
this put in the definition
so this is nothing but the convex linear
combination or the size of the
set omega according to mu 1 and mu 2 but
with this weights
p and 1 minus p so that is the convex
linear combination
mu 2 of omega is also 1 because this is
a probability measure
and hence you end up with this formula
this is simply p
plus 1 minus p which is nothing but 1 so
therefore with this convex combination
that you have
looked at this gives you total measure 1
for the set function nu so for a convex
combination of measures
you end up having a probability measure
so let us go back so we are saying that
take two probability measures mu one and
mu two
and look at this convex combinations of
individual sizes correct
and therefore we are ending up with a
probability measure
so again as mentioned a few minutes back
we are now going to remove this set
capital a from this equality
as defined here and we are going to look
at this formal summation
as some kind of functions functional
equation
so therefore we are going to write this
as a convex combinations
of p and one minus p so those are the
weights that appear in the convex
combination
and it will be a convex combinations of
the measures mu 1 and mu 2
with the weights p and 1 minus p so that
is the motivation
that we have looked at now this was for
convex combinations of two things
two measures mu 1 and mu 2 correct now
we can do the extension of this result
and this is stated in exercise 3. so
take
n measures n probability measures mu 1
up to mu n
all of these are defined on the same
probability space omega script f
now you are going to choose convex
combinations so you have to choose
scalars
alpha one alpha two up to alpha n
non-negative with the sum
equal to one so this will be the weights
for the convex combinations
then what we are going to look at is
this new measure
given by this convex combinations alpha
is mu wise
now you should be clear what this
equality means it simply means
that if you want to compute nu of some
arbitrary set a
all you have to do is to look at mu i's
of a
so accord the size of the set a
according to mu y
first compute that then multiply by the
weights alpha i and add them up
so that will be the convex combinations
of the sizes of the sets
a in terms of the individual measures mu
1 up to mu n
so then as per the exercise here it will
also be a probability measure
so this proof goes through exactly
the same arguments as considered for the
convex combinations of two
probability measures so the same
argument goes through and you will get
this
and this was in fact the special case
that
was mentioned earlier in exercise 3 as
the motivation
so what we said was that the uniform
measure on a finite set
came out to be these convex combinations
of the dirac masses
so that's exactly what was our
motivation for looking at
such combinations such convex
combinations
okay but with that hand hand we can now
identify that thing so we can now
connect this exercises 2 and 3
which was our motivation that you just
say
that mu y is where this dirac of x is
and
alpha 1 up to alpha n is exactly 1 over
n so that is the convex combination
that is given earlier okay
now we can extend this exercise
three which was for this finite
combinations two
infinite combinations involving series
what you have to do you now look at a
sequence mu n of probability measures
and the corresponding sequence of
weights so this should be non negative
and they should add up to one so we are
now looking at series
now the question is is the set function
new
given by this countable summation alpha
and mu
n ah probability measure and the answer
will be yes
please try to work this out again the
meaning is same
if you take a arbitrary set capital a
then all you have to look at
is first mu ns of a the weights of the
set capital a according to mu n
and then multiply by alpha n and add
them up
so that will give you a probability
measure please check this
now these exercises allow us
to construct more examples so how we are
now going to look at
convex combinations of property measures
remember
the explicit example of probability
measures that we have
are the d-doc measures now on the real
line
consider the direct measures drock 1 and
dig0
meaning the direct meters support deduct
measures supported at 1 and direct
measure supported 0.
if you choose a appropriate p between 0
and 1
and look at this convex linear
combination as per the proposition 2
above
you can immediately get this probability
measure
okay so let us go back yes so this is
the proposition 5
sorry the correction will be made so
this will be
position five sorry so the proposition
five
will imply that probability of delta one
and
one minus p delta zero that will be a
probability measure
ok now you apply exercise 3 which was
for
combinations of more than 2 measures so
again look at
deduct measures but support it at
integers from 0 to n
and look at this combinations given from
the binomial coefficients
so choose some p between 0 and 1 fix a
positive integer n
look at these coefficients these
coefficients as before these are
familiar coefficients they add up to 1
as given by the binomial theorem so
these coefficients add up to one
and this will give you a convex linear
combination for you
of the dirac measures and as for the
exercise earlier
this will be the prob this will be a
probability measure okay so let us go
back
so this is the exercise that i am
referring to that while you are looking
at
convex linear combinations of n measures
in probability measures
okay all right but then we also looked
at the series version of this
where you are looking at a sequence of
probability measures
and as i already mentioned that exercise
4
i left it as a question but i mentioned
that this answer to that easiest
you can get this as a probability
measure so then
you look at this deduct measure
supported at the points k
at the integers k from 0 to infinity so
look at all possible
non negative integers and look at this
weight so
which is this what is this so this is
exponential minus lambda lambda power p
k by k factorial so these weights if you
just look at them
all alone adding them up you get one
so this also denotes some convex linear
combinations
of these dirac meshes so this will again
be a probability measure
now let us just go back and look at all
these examples
so this examples that we have so far
said
from these exercises are following
certain parameters
here it was n and p and here there is a
parameter lambda
which is taken to be positive then
what will happen is that we are going to
connect
these explicit examples of probability
measures with some known random
variables
known discrete random variables
this we are going to see them later on
when we start discussions about
random variables but the this
examples are coming simply from looking
at the convex linear combinations
of the dirac masses okay now
ah some final exercises to finish off
the lecture
so earlier we had looked at convex
linear combinations of probability
measures
and that gave us examples of probiotic
measures once more
but then we had also looked at another
proposition
which said that if you start off with a
finite measure
and scale it accordingly divide it by
the whole mass
then you get also get a probability
measure but then
this is about dividing by the whole mass
which is in some sense multiplying by a
factor
correct but this motivates us to look at
this exercise
and this you can do for any measure you
do not have to restrict your attention
to probability measures but you can do
it for any measures so this
is just a repeat of the earlier
exercises
that was done for the probability
measures so what is this
so if you have mu 1 of mu 1 mu 2 and so
on
b measures on a fixed measurable space
omega f
then you have this set of exercises
first
look at two of them mu 1 and mu 2 then
i say that i can add them up and get a
measure how
so mu 1 plus mu 2 that is defined as
follows
so for any arbitrary set a what you do
you
add up the individual sizes according to
mu 1 and according to mu 2.
add up the sizes and i am saying that
you will get a non-negative set function
which is also countably additive
so all you have to follow are the
earlier proofs or earlier methodologies
that you have done for the exercises
and you will immediately get the
countable additivity by definition
this first example is of course non
negative because mu 1 and mu 2
both are known to be measured so they
will take non-negative values
so that is your first part in the second
part i am saying
instead of adding of two of them look at
the series
so now what will happen is that this
series
will give you a way of defining the
series or summation of measures
what is this so the countable sum of the
measures is defined as follows
so for any set a you look at this left
hand side and this is defined as
adding up the individual sizes of the
same set according to different measures
mu n
so that gives you the second example or
second part of the exercise
and the final exercise is that you can
now scale it by appropriate scalars
alpha
so choose any arbitrary alpha and
choose this indices i from 1 2 3 and so
on
then if you multiply any one of the
measures that was considered earlier
mu 1 mu 2 mu n any measure that you have
considered earlier
and multiply by alpha you get a measure
so how it is defined
so alpha times mu y so that associates
the weight or the size to arbitrary set
capital a
according to this formula that you first
look at
mu i of a meaning you look at the size
of the set a
according to mu y the measure given and
multiplied by
alpha but then this will
be a non-negative set function because
alpha is non-negative
and you can immediately prove the
countable additivity exactly as
discussed earlier but then if you put
them together
you are getting a non-negative set
function which is countable additive and
this becomes a measure
so now these three part exercises
what does it tell you it says that you
can multiply by
positive scalars alpha and also look at
additions so therefore
you can look at in general linear
combinations
of measures so that is the take away
from this exercise
that if you have some finite number of
measures
or a countable number of measures then
you can look at linear combinations
or the corresponding countable sum of
the measures
multiply by the appropriate positive
scalars
you still end up with a measure so this
is a very
interesting construction that will also
allow you to construct
measures in general from known examples
mu 1 mu 2 mu and so on
so that is the take away from this
exercise
we are going to stop here and we are
going to continue the
discussion in the next lecture
[Music]
[Music]
[Music]
you
