Lecture 12: Measuring Central Tendency with Missing Data

IIT Kanpur July 201
20 followers
Feb 2, 2024
[Music]
Hello friends welcome to the course
multivariate procedure with r so you can
recall that in the last lecture we had
understood that how are we going to
handle the missing values in the art
software that means if uh there is some
data set which has got missing values
because of any
reason now how R is going to handle them
that how are we going to identify the
places or uh that where we have missing
data how are we going to identify
whether my data set has missing values
or not then how are we going to identify
that what are the places where we have
missing values how are we going to
create a vector
after removing the missing
values now the next question comes that
how are you going to implement this
concept in different types of
functions related to particular
statistics that in case if there is some
missing data then how are you going to
implement the statistical procedures
with the missing data and our basic
objective is now here is that which will
be remain the same in this lecture and
in the next lecture and in all the basic
concept when we are trying to handle the
missing data in the way I explained that
we want to remove the data and then we
want to compute our statistical
procedures with the remaining available
data please note that I am not handling
here the topic of missing data models
where we try to estimate the missing
values and we try to impute them inside
the
uh data Vector so that the data Vector
looks
complete so please keep this in mind and
if you want to learn how to handle the
missing data there are different
functions available in the r software
but definitely I am not going to
consider them in this course so now in
this lecture uh we are going to
concentrate how are we going to measure
the central tendency of the data when
the data Vector has some missing
values for example suppose I want to
know that uh how much a new medicine is
going to control the body temperature
that means if somebody has got a high
temperature and suppose if the new
medicine is given to the patient for how
much time the body temperature will
remain in control after taking the
medicine now suppose I take a sample of
suppose five
uh
patients and I decide that I will give
them a medicine today and tomorrow they
will come and report me and this process
I will continue for say 5
days now suppose the one patient comes
on first day takes the medicine comes on
second day takes the medicine but does
not come on third day now I don't know
whether the what was the recording of
the body temperature or the duration in
which the body temperature remained in
control for the third day now the
patient comes again on fourth day and
fifth day now I want to
compute suppose the arithmetic mean on
the basis of given set of
data that means first day second day
fourth day and fifth day so how are we
going to compute the mean and similarly
how are we going going to compute all
other measures of central tendency that
we had discussed in the earlier
lecture so let us begin this lecture and
we try to understand that how are we
going to measure the central tendency of
the data when we have missing data
values okay so now you recall the
arithmetic mean of the observations X1
X2 xn is defined as xar 1 / n summation
I goes from 1 to n x i and we had under
Ood that mean X that is the r command to
compute the value of the arithmetic mean
of the data in the data Vector X right
all this data X1 X2 X1 this has been
stored in a data Vector X and just to
give you an example what I'm going to do
here that I am I have created here a
small data set in which the time taken
by 20 participants in a race is recorded
and it is in seconds right so what I
will do that this is my complete
data and after that I will try to remove
some data and I will replace them by na
and I will try to create the uh data
with missing values and I will try to
execute different operation over the
same data sets so now we know that if
you store all these data values in a
data Vector time and if you want to know
the arithmetic mean of this of the
values in this data Vector then I can
find it by mean of time and it comes out
to be here 56 well you are aware of this
thing so you can see this is here the
screenshot now if there are some missing
values in the data then I have to modify
this command mean and we try to write
down here mean X and we try to add n. RM
is equal to true this true is
a logical
variable right so if I try to add this
command here na. RM equal to true so
this na a means this is trying to
indicate the missing
values and uh this here RM this is
trying to indicate here the remove right
so when I'm trying to say here that na
do RM is equal to here true then it is
trying to say here that please try to
remove the missing values or
missing
values have have to be
removed the answer here is yes and this
is indicated by true and if there is
false that means the missing value uh
need not to be removed and this is
actually here the the basic default
operation right so now this command will
give us the arithmetic mean of the
values after removing the missing data
that is any values from the data vector
so now could we try to consider here the
same example in which we had recorded
the time taken by 20 participants in a
race in seconds so now you can see here
I have uh made the first two observation
here as na well I have made it here
underline just to uh just to have your
attention otherwise you don't have to
make it underline now I try to store
these values in a data vector and I
indicate it by time. NA so time was my
original data vector and na is simply
indicating that there are some na values
yes there is no compulsion to put here a
doubt that that you please don't
misunderstand that you have to give the
data Vector name with a DOT I'm just
taking it for the sake of convenience
right otherwise this can be time one
time two also or whatever you want so
now you can see here this is my here
data vector and it is entered in the r
software also now if if you try to find
out here the mean of this data Vector
time. na it will come out to be here na
because it is trying to find the
arithmetic mean of na+ Na plus all those
values which are here like 45 83 74 like
45 + 83 + 74 and dot dot dot divided by
20 so this will come out to be here na
but now what you are trying to do you
are trying to remove this na values and
you want to compute here 45 + 83 + 74 up
to here divided by 18 so for that I try
to add here a command na. RM is equal to
true and this will give me here the
value
58.5 and in case if you try to use here
the command na. RM is equal to false you
can see here this is here the na which
is here just like here Na and this is
actually the default mean option right
now I would like to have your attention
once again that if you try to see when
you had the complete completed data set
then the arithmetic mean was 56 and now
when you have removed the two values
then the mean is changed and it is
coming out to be 58.5 so that is
indicating that when there is a missing
data in the values then the values which
could have been based on the complete
data set will change and that is why it
is important to understand how are we
going to handle this missing data right
so this is here the screenshot but
before going forward let me try to show
you these commands on the r console also
right so I already have taken here this
data I have entered you can see this is
here time and this is your here time. na
right if you try to find out here mean
of here time this will come out to be
here 56 and if you try to find out the
mean of your time do na this will come
out to be here na now if you try to use
here the command mean time. Na and na.
RM is equal to here true then you can
see here it is coming out to be here
mistake error why because you are trying
to put here additional this bracket so
this is the concept of your error I
explained you in the last lecture also
that means once there is error then the
command will not move forward but if you
try to correct it then it will move
forward and this value will come out to
be 58.5 on the other hand if you're
getting a message like warning then this
value will come but then you will have
to look into your command that how are
you that where is the mistake where is
the slip right and if you try to use
here this command that na. RM is equal
to false you can see here it will come
out to here na which is equal to here um
finding out the mean of those values
which are having the missing values also
right so this this is how we go forward
this thing right so this is where I'm
trying to explain you that what is
happening that when you are trying to to
find out the mean of all complete values
20 values then it is computed here as a
1 upon 20 summation I goes from 1 to 20
XI and this value here is 56 but when
you are trying to compute the mean in
the missing data Vector those so those
two values they are removed and then we
have here 20 - 28 observation and the
arithmetic mean of the remaining
complete observation is obtained here as
xar is equal to 1 upon 18 summation I
goes from 1 to 18 XI and this value is
here
58.5 so now this type of changes will
happen in each and every function the
reason why I'm trying to show you here
this difference that well in the case of
this arithmetic mean I have shown here
after that we are going to consider
different types of function in this
lecture and in the next lecture so this
type of changes are going to happen in
each and every case but uh your
objective should be that you should
actually know what is happening so that
will help you in better understanding
otherwise R will do in the way it has
been programmed right now in case if you
recall the in the last in the earlier
lecture we also considered the concept
of median to find out the central
tendency of the data so now if you have
all the data available that mean there
are no missing values then you can use
here the command median X that is m e d
i a an inside the parenthesis data
Vector X which is all the values no
missing values
here right no missing values but in case
if your data Vector has some missing
values then the same command can be used
as median X but now you have to add here
the option na. RM is equal to true and
after that if you go to the help menu
there will be many other options well
I'm not going into those details but I
request you to please look into the help
menu and try to understand what those
different procedures are trying to
explain you as I said in the beginning
that our software is not a black box it
has all the options the only thing is
this you have to customize them
according to your need and requirement
so if you try to use this command then
all the missing values which are
indicated by NA they will be removed and
the median of the remaining observations
will be obtained right so if you try to
see here the uh mean I have taken here
the same example which I took earlier in
which the some time it taken by some
people to complete a race is now
recorded but I have change the data name
so that you don't get confused between
the two so if you try to to store all
these values in a data Vector I have
given it a name here minutes m n u
minutes and then I if I try to find out
the median of this data set minutes it
will come out to be here 26 right it is
here like this now in case if you try to
remove here first two data points
suppose if I remove here and I try to
create here another data Vector min. na
then if you try to compute here median
of minutes. Na and WR here the command
na. RM is equal to
true then it will come out to be here 26
right so this is how it is going to be
computed right but let me try to show
you first these examples on the R
software so let me try to first take
this data set
minutes you can see here minutes and
then if you try to find out here minutes
of your median of your minutes it will
come out to be here like this and now if
you try to take here the data sets
minutes.
Na and if you try to find out here the
median of this minutes. na you will see
here it is na but if you try to add here
the command na do RM is equal to true
you can also write here capital T also
it will come out to be 26 now in this
case actually if you try to see both the
values are coming out to be 26 because
it is a robust estimator so you should
not think that the values are not going
to change in the median median is a
robust estimator that is why a small
change in the data values may or may not
alter the value of the median but anyway
mean I'm trying to convey that this 26
and 26 are here same in this particular
case but if you take another data set
they may change right so don't think
that this missing values are not going
to impact uh this median right okay so
now we come to an end to this lecture
and you have seen that okay I have taken
here the two measures of central
tendency which are here mean and median
in which I have explained you that how
are you going to compute them when we
have missing data in the data
Vector now similarly if
you want to execute another type of uh
uh this uh measures of central tendency
like geometric mean harmonic mean
Etc they can also be executed on the
similar lines but definitely I'm not
going into those details here but I
leave it up to you because the concept I
have given you and now I expect that you
should also move along with me so that
uh you can handle them very easily so my
request will be you try to take the same
data set here time. na or median Na and
try to compute uh this geometric mean
harmonic mean Etc and try to see
yourself are you able to do it or not so
in the next lecture I will try to now
consider the measures of variations and
I will try to show you that how you can
handle the different measures of
variation when the data Vector has
missing values so you try to practice it
and I will see you in the next lecture
till then
[Music]
[Music]
goodbye
[Music]
