noc19-ma14 Lecture 06-Introduction to Descriptive Statistics - Objectives, Steps

IIT Kanpur July 201
20 followers
Dec 3, 2023
Objectives of statistical analysis of data. Different tools available in descriptive statistics 3. Idea of population and sample
[Music]
[Music]
welcome to the lecture on the course
descriptive statistics with our software
now you may kindly recall that in all
the earlier lectures we had discussed
and we had an idea that how our software
is going to help us in different types
of computations now from this lecture we
are going to start the discussion on the
topics of Statistics but here I would
like to tell you one thing or I would
like to clarify one thing the topics in
descriptive statistics whatever I am
going to consider here they are pretty
elementary and I will try to go to the
depth as much as possible under the
given time frame but my idea is not
really here to teach you statistics my
idea here is that most of the topics you
will see you know and my objective is
that I would like to make you
comfortable that in case if you want to
use the our software for the computation
of those topics then you should be
comfortable you should be confident once
you are confident in handling the basic
topics I am sure and I am confident that
there should not be any problem in
handling the the advanced topics in
statistics
besides this thing people are using
these statistical tools very often but
sometime they don't know why they are
using it sometimes they don't know what
is the interpretation of different
quantities so that will be my another
objective on which I would try to
discuss here
that whatever the the tools of
descriptive statistics I am going to
handle I will try to discuss about their
concept their implications and their
competition using the our software right
so in this lecture I am definitely not
going to use any our software for the
competition but I would try to give you
an overview that what is descriptive
statistics and how it helps and what are
the ways in which it can give us
different types of hidden information
and a given set of data so only I will
be considering on the basic aspects in
this lecture and possibly in the next
lecture also so one of the basic
fundamental question comes here what is
really statistics so I would say simply
that statistics is a science which turns
the data or which tries to take out the
information contained inside the data
and converts it into a form which is
useful for making a decision this
decision can be at your office level at
policy formulation for forecasting at
country level or say anything else so
what are we going to do here we are
trying to collect the data we are trying
to analyze the data and based on that we
will try to make some statical
inferences and those inferences are
drawn from the numerical facts which we
are going to call as as data so data is
a very important thing in statistics and
it gives you some information and this
information is hidden inside it and what
is descriptive statistics this is
essentially the starting point for
knowledge discovery on the basis of data
yes
the discovery of knowledge can be done
from different types of ways by looking
at the picture by reading some subject
and similarly aesthetics also gives us a
tool to discover the knowledge that is
contained inside the data so data is
essential
a very important source of information
data contains many information inside 8
but the problem is the following if I
know something then I can speak and I
can inform you but data cannot speak
data cannot listen data cannot
understand the language what we speak
for example if I am speaking Hindi or
English language possibly you can
understand it but if I try to speak in a
language which you don't understand then
I will not be able to transfer the
knowledge so similarly data has its own
language for example you have seen a
first somebody has some problem in
speaking or hearing then there is a
Chinese language and that language can
be understood only by those people who
understand it so similarly data also has
a language and which is based on
different types of symbols notations and
interpretation so our objective here is
that that we want to know the tools by
which we can draw the information
contained inside the data using the
tools of descriptive statistics so
essentially I can say here that
statistics is a language of data and it
provides a scientific way to extract and
retrieve the information hidden inside
the data and remember one thing its
taxes cannot do any miracle sometime if
you try to see you might have heard that
people try to make different types of
joke different types or comments on the
statistic that is that 6 lies or
something like this so I would like to
inform you that it started 60 never lies
right the statuses is simply based on
the data and now it is our capability
that how much we can retrieve the
information in a from inside the data
for example if somebody is speaking to
us in a
sign language then it depends on our
capability that how much I can correctly
interpret it so remember one thing that
aesthetics also cannot change the
process or the phenomena whatever
process is happening that will happen
and as a statistician I am not allowed
to alter or change the process I simply
have to collect the data on the basis of
the process which is happening which is
continuing and on that basis I have to
take a call I have to take a decision
that which of the tool is most
appropriate tool in this situation to
draw or to extract the information
hidden inside the data right ok and
definitely the inferences what we are
going to draw on the basis of a shtetl
tool they are going to use for different
purposes
for example one of the basic purpose is
forecasting so for example stat scale
tool provide forecasting but remember
one thing this is not like a astrologers
parrot that the parrot chooses a chit
and then one reads it that what is my
future
it does forecasting but on the basis of
some scientific principle and the
principles of statistics now whenever
you are coming to this aspect that you
have to choose a tool on a data set to
extract the correct information then in
this process I can divide the entire
canario of decision-making into two
parts one part I would say here suppose
my data is here and there are two option
that the data is correct and the second
option is this data may be wrong what do
you really mean by data is correct and
data is wrong suppose I want to know the
average height of the students in a
class then obviously I have to collect
the data on the height but suppose I am
trying to collect the data on the weight
and based on that I am trying to infer
the data on the height then it is not
appropriate right so in that sense I am
trying to say that the
data has been chosen correctly which is
matching with the objective of the study
and second aspect is choice of tool
there are many many state scale tool
which are available and we are going to
study them in the future lectures in the
further lectures but the main thing is
this one has to choose what is the
correct tool to diagnose the problem and
to solve the problem just like if you go
to the shop of a medicine or if you go
to a doctor there are thousands of
medicine but what doctor does doctor try
to decide that which medicine is most
suitable for the given problem so
similarly in statistics also we have
many many tools and we need to make a
decision that which tool is going to be
appropriate to draw the correct
aesthetical inference for a given
problem so I know how the choice of
statical tool can be correct or the
choice of statical tool can also be
wrong now there are four options first
option is this suppose I am trying to
choose the wrong stat scale tool over a
wrong set of data then in this case the
decision is not going to be the correct
the next option is this means I can
choose the wrong start school tool and
the data is correct even in this case
I'm trying to use the wrong tool over a
correct data I will get a incorrect
decision and similarly in case if I try
to take here the wrong data and if I try
to take here the correct aesthetical
tool so using the correct start scale
tool all are wrong data will also give
us the incorrect decision now the last
option which is correct is that I have
to use the correct stat scale tool over
our correct data so this is the only
option though so unless and until you
try to choose the correct esthetical
tool and you collect your data correctly
you will not be able to get the correct
aesthetical inference out of it the
rule is very simple garbage in garbage
out right
and one thing we wish I would like to
inform is that sometime if people do
come to us and they ask us to do the
start six very quickly for example I
have to submit my thesis tomorrow I have
this data please try to help me write
well it is not so simple at that stage
because you are the one who has
understood the entire process and as a
statistician I have not been told about
your process so first you need to
explain me the entire process then I
will try to understand the data
generating process and only then I can
do something and it is also possible
that the type of tool which you need
that may not always be available but it
needs to be developed so statistics
always need some time to understand the
phenomena now another popular question
with which I am asked that when I come
to the aspect of descriptive statistic
there are two types of tools one is
graphical tools and C and another is
analytical tool so there are different
types of graphical tool like as to
dimension three dimension plots a
scatter diagram pie diagram histogram
bar plot stem any plot box plot and etc
there is a long list in this case people
do ask me which of the graph is more
suitable or they also feel that if they
try to use more number of graphs then
their analysis is going to be better I
would say this is only a myth you simply
have to choose the correct graph and you
have to use the correct number of
graphics so the appropriate choice of
graph and appropriate number of graphics
that is only going to help you in
getting the correct information and
similarly when I come to the aspect of
say they say analytical tool their
analytical tool I can there are
different aspects on which we try to
analyze the data for example I would try
to find out what is the central tendency
of the data what is the variation in the
data and what is the structure of the
data and what type of relationship are
existing inside the data so for example
if when I come on the aspect of measure
of central tendency then we have
different tools mean median mode
geometric mean harmonic mean quantiles
etc and similarly when we are trying to
understand the variability in the data
then we have different types of tools
variance standard deviation standard
error mean deviation absolute deviation
range etc so you can see here in this
case there are two aspects one is the
central tendency of the data and another
is desperate or the variation in the
data now in case if you want to study
the central tendency of the data then
and then out of this list you have to
choose the appropriate tool and
similarly in case if you want to study
that nature of the variation in the data
then you have to choose the appropriate
tool and similarly in case if you want
to find out what is the structure of the
data in terms of symmetry peakedness etc
then you have to choose a proper tool
for symmetry and then there are concept
of skewness concepts of kurtosis these
concepts are going to give us more
information that what is the structure
of the data and then another aspect can
be if I have a data on say more than two
aspects something like height or weight
or say height weight and age then there
may exist some relationship in the data
also or there may exist some coherent
structure inside the data then in order
to study those aspect we have the tools
of correlation coefficient rank
correlation multiple correlation partial
correlation coefficients correlation
ratio intraclass correlation coefficient
linear regression nonlinear regression
etc so there is a long list so when we
talk about the descriptive statistics
descriptive in statistics is not a tool
but this is the collection of the
appropriate number of tools that may
include the graphical tools as well as
that may include the analytical tool and
the choice of analytical tool also
depends what exactly do you want to
study many times people do come to us
and they ask us sir can you please do
some static analysis on my data
in that case I would always request them
please let me know what really you want
to know from this set of data and based
on that I am going to take a
appropriated decision and I'm going to
decide that which of the statical tool
can give an answer to your query and
then I would try to use it so another
question crops up here that which of the
tool is a better option graphical tool
or say analytical tool my suggestion is
that
please use both of them because if you
try to see this descriptive is that six
is the point of starting of a name for
this analysis what you have in your hand
you simply have a set of data data are
some numerical values so you can always
imagine that in front of you there are
20 values there are hundred values there
are 2,000 values or they can be two
million or so two billion values and all
those values are sitting silently and
you are the one who is going to start
the knowledge discovery on the basis of
given set of data right so I would say
don't make a rule but depending on the
condition try to use both types of tool
later on in these lectures I will show
you that how the graphical tools and and
how these analytical tools can be used
under what type of condition and how
they can be computed on the basis of our
software now what is the difference
between between the use of graphical
tool and an analytical tool graphical
tools provide us a visualization this
will give us a first-hand information
and what about analytical tools they
will give us the information in
quantitative form and they will give us
the quantitative information so graphics
will give us information but we have to
look into this and then we have to draw
a proper esthetical inference and this
analytical tool will give us a number
which we have to interpret to make a
correct hysterical inference so I would
say usually are in most of the cases
this graphical tools and analytical
tools both work together because the
process is the same data is the same and
data is propelling you that please use
only the graphical tool or please use
only the analytical tool this is only
you who is going to take a call that
whether graphical tools have to be used
or say these analytical tools have to be
used so please try to make a appropriate
decision keeping in mind the objective
of your study and the type of
information which is contained inside
the data and in statistics
why do statuses comes into picture it's
Texas comes into picture because
variation always exists in all the
process what do you mean by variation
for example if I say suppose you try to
take a plot and you try to put say this
hundred grams of seeds and say after a
month you will get a crop and suppose
you will get one kilogram of seeds now
in case if you try to repeat the same
thing try to use the same plot or the
same sizes or plot and put the same
quantity of seeds do you think that in
all the plots you are always going to
get exactly one kg of field this is
practically very difficult there will be
some difference one plot may be given
you one kg another plot may be given you
one point one kg and say another plot
might be giving you 900 grams and so on
so the variation always exists in all
the process and in statistics our basic
objective is that that we want to
understand the process of this variation
we want to control this variation and to
draw a statistical inference out of the
data with minimum variability so this is
one of the basic objective so in
statistics or in descriptive statistics
what we really want to do we have a set
of data now we are going to use that
Gator
on a aesthetical tool that may consist
of say analytical tool as well as
graphical tool now I will be getting
some information from the graphical tool
I will be getting some information from
the analytical tool and now this is my
responsibility that I have to combine
the information coming from both the
aspects together and I have to convert
it into our piece of information which
is useful which is interpretable or that
can be conveyed to the experimenter who
might not have any knowledge about the
statistics right so I can say that using
the information gained by the tools of
descriptive statistics and combining
them together to reach to a meaningful
conclusion to depict the information
hidden inside the data is the objective
of any stats color analysis and proper
interpretation of those outcome is very
important and all these outcomes all
these inferences are made only on the
basis of data so P next question comes
how this data is coming so there are two
processes one deterministic process and
say another is non-deterministic process
deterministic process means you know the
outcome in advance but non-deterministic
process are where you really do not know
the outcome in advance so in statistics
whenever we are calling or when we are
and whenever we are trying to understand
the data generating process the data
generating process is always random or
say non-deterministic and that is why
the role of statistics comes into
picture once there is no random
variation things will become purely
mathematical so a simple question that
arises here why should we collect the
data so data is collected with different
types of objective first is to verify
the theory
all findings for example suppose if I
say in children the height increases as
the weight increases or the weight of
that child increases as the height
increases suppose that is my theoretical
finding and ever and if I want to verify
it whether this is really happening in
real life or not so I need to collect
the data and I need to verify this
finding secondly I have some objective
and I really want to know the outcome of
that process so I have to collect the
data which is being generated from that
process and then I have to use a stat
scale tool to get the correct
aesthetical outcome and remember one
thing the inference what we are going to
draw that is just on the basis of the
collected data you cannot argue that
some statical inferences is coming and
which is from some other source beyond
the data so but clearly when we are
talking of the tools of descriptive
statistics we try to speak of
information or we try to report the
information which is coming only from
the given set of data and yes the
information which is coming from this
data that we try to convert it in the
form of statical inferences which is
further use in the development of state
scale models which are used for policy
decisions gradually a classification
forecasting and many many other things
possible now in case if you want to make
a study on a statical experiment then
what are the steps involved right the
first objective is that please identify
the objective of the astatically
analysis which is missing in most of the
cases in my experience right people
simply try to collect the data and after
collection of the data many times they
try to decide what type of statical
inferences they can draw from this data
well that is not bad but at least my
suggestion is that
before you collect the data please try
to decide the objective of your study
and try to ask why I am collecting the
data and based on that you have to take
further steps okay how to get the data
the data can be obtained from a
laboratory experiment from a survey from
some primary sources or from some
secondary sources called as primary data
or say secondary data but whatever is
the data I am not bothered about that my
objective is this I have to use the
correct statistical tool and I believe
that by using the correct aesthetical
tool I can get a correct correct valid
and meaningful interpretation of the
result that is my belief and with this
objective I am moving forward so the
next question comes what is an
observation so the unit on which we try
to measure the data that is called an
observation what did this mean suppose I
want to measure the height so first I
have to decide height of children or say
height of elders suppose I decide that I
need to measure the height of the
children between the ages say 5 years to
7 years so what I would say that I will
try to collect is some children whose
ages are between five years and seven
years and then I will try to record the
heights of those children so the heights
of those children which will be some
numerical value and they will be called
as an observation so similarly in case
if I want to find out the number of
persons number of cars monthly
expenditure on say food in any family
then these are also my observations
which are trying to cater to some
objective of statical analysis next
definition which I would like to discuss
here what is called a population the
collection of all the unit is called a
population for example in the in the
earlier example when I wanted to
the data on the height of children
between five and seven years you are
trying to collect the data only on some
of the children but do you think they
are the only children no there are many
many children in that city in that
locality in that country what what are
you trying to do
you are simply trying to choose some of
the children and then you are trying to
record the data so the collection of all
the data that can be locality that can
be City that can be country that depends
on the objective that is called as a
population of children whose ages are
between five years and seven years
similarly suppose I want to find out the
average age of the of all the female
students in class ten in a school on the
basis of a sample then by population is
going to consist of all the female
students in class ten in that school
that will be my population but if I want
to study the average age of all the
female students in that city then all
the female students in that city who are
studying in class ten that will consist
of my population and similarly in say in
another example if my objective is that
I want to know that how many female
employees have salaries more than the
male employees in a given company on the
basis of a sample so in that case my
population will consist of all the
female employees in the company from
there I will try to choose a small
sample now the penis question is what is
a sample so sample is only a subset of
the population a basic question comes
why we use this sample well that is the
main objective and main advantage of
using statistics we are always
interested in finding out a statistical
conclusion which is for the entire
population maybe of country maybe of
skitty or maybe of village and there
will be large number of people who have
to work to collect the data that is very
difficult so the advantage of statistics
is that that the statistics says that
instead of collecting the data on the
entire population if one can collect the
data on a small fraction that we call a
sample then on the basis of sample of
the data the statuses can help in
getting a reliable esthetical inferences
which are going to be valid for the
entire population so that is why
collection of sample is very important
in statistics so whenever we are trying
to collect the data in a sample we
believe that whatever are the
characteristics whatever are the
features which are present in the
population they are also present in the
sample for example you have seen that if
you go to a market and you want to buy
some wheat and there is a bag of hundred
kilogram of wheat usually you will not
open the entire bag but you will try to
take a small sample maybe consist of say
this 20 grains 40 grains hundred grains
you simply try to look at those grains
and based on the quality of the grains
you try to make an inference for the
entire bag which is of hundred kg so now
this is my sample sample is possibly
consisting of the grains of wheat maybe
20 gram 30 gram 40 gram and based on
that whatever we are going to continue
that is going to valid for the for the
entire bag it's not even the entire bag
but I will say the entire wheat
available in that shop so that is why
the collection of data in the sample is
very important and we believe that the
data has been collected in such a way
such that the sample
representative so this will be our basic
assumption and that goes without saying
that in all his tactical analysis the
sample means sample is representative
what is the representative sample that
means all the characteristics which are
present in the population are also
present in the sample for example in
case if the quality of the meat is not
so good suppose there are ten grains
which are infected by some insects then
we assume that in the population also
means a similar type of proportion will
continue in case of the ten percents
seats in my sample in my hand are not
good so we so I believe there 10% of the
wheat in the entire bag is also not good
so this is how we go so about basic
foundation like that in case if my
sample is good sample is representative
then my statical inferences are also
going to be good and there are various
ways in statistics which help us in
choosing the representative sample or or
say character sample so so we have
different types of sampling scheme like
simple random sampling stratified
sampling cluster sampling systematic
sampling multi phase sampling multistage
sampling etcetera and which helps us and
guide us that how to choose the correct
good and representative samples but
definitely this is not the objective of
the course and I'm not going to discuss
that what are the different sampling
procedures to collect a good data so now
in this lecture I have given you a brief
background that may not look very
mathematical but believe me this is very
important for us to understand that what
are we going to do under what type of
condition only then I will be able to
take the correct decision and I have
recorded or I have given this lecture
with this objective only so I will try
to continue with some more basic
definition in the next lecture and you
try to understand this lecture
and try to create a foundation inside
your mind to understand the tools of
descriptive statistics and I will see
you in the next lecture till then
goodbye
[Music]
[Music]
[Music]
