noc21-ma36-lec12

IIT Kanpur July 201
20 followers
3 Video Views·Dec 2, 2023
In this lecture, steps in conducting a survey and variability control in survey is explained.
[Music]
[Music]
hello
welcome to the course essentials of data
science with r software
part 2 where we are trying to learn the
topics of
sampling theory and linear regression
analysis in this module we are going to
talk about the basic fundamentals of the
sampling theory with
r software so you can recall that in the
last couple of
lectures we started discussion on the
basic fundamentals
and basic definitions which are related
to the sampling theory
now in this lecture also i will continue
with
those topics and in this lecture
i would like to explain you
that if you want to conduct a survey
then what are the different steps means
how you
have to collect the data what are the
different ways to collect the data
and what is your objective the the main
objective is that one has to collect the
data in such a way such that the
variability is as minimum as possible so
how to ensure it
so these are the few things which i am
going to address in this
lecture so let us start our lecture
ok so first thing which i am going to
address is that
what are the different steps
in conducting a survey
well these steps are going to be like a
story
telling but these are very useful
when you want to conduct a survey in a
real life
well there are several steps but i would
like to
make it clear that these steps actually
are not independent
they are inter related i mean the first
step will also indicate
that what is to be done in the next step
and and something what is
what is to be done in the further step
that has to be taken care in the first
step so let us start this thing and i
will
try to explain you well ok
so what are the principal steps in
conducting a survey
the first step is you have to define the
objective of
survey this is very important
that first you need to decide what you
really want to find
out from the survey and try to write
down
very clearly what are the different
questions which you want to answer
and many time people start the
survey they just collect the data and
after collecting the data they
come and they ask okay
what type of information can be obtained
from this type of survey
well those things are not really correct
so
first you have to define the objective
of the survey that means you simply have
to define
different questions which you want to
address
now in the second step you have to
decide and choose the
population from where you would like to
take the
sample now you can see
that step one and step two they are
interrelated
whatever are the objectives which have
been defined in step one
they will basically decide
that what should be the population and
what type
of sampling scheme is to be used and
what to be sampled
now in the third step one has to decide
what is the data which is to be
collected
for example if the objective of the
survey is to find out the height of the
students
then in the second step we are
going to draw a sample
of students from the population of a
students
that means all those students who are
the students of a university or a
college
then in the third step we have to find
the
or we have to record the height of the
student
sometimes it happens that that the
objective is to find out the height of
the student and people try to collect
the data on the weight of the student
that should not happen these type of
things
now after this we have to decide what is
the degree of
precision what is required for example
if i take the same example if the height
has to be measured
we have to very clearly indicate that
it is in terms of meter centimeter up to
what degree of precision
for example if you want to collect the
data on age
then one has to very clearly specify
that the age has to be in terms of
number of completed years
or number of years number of months or
number of
years number of month number of days and
so on or it is
or is it going to be in terms of number
of days
whatever is the objective based on that
whatever is the requirement
that has to be mentioned very clearly
before the
survey starts then in the next step we
have to decide
what method of measurement we have to
adopt means how are we going to
record the observations you want to
find out the age of the person suppose
then would you like to record the age
or you would like to record the date of
birth
and based on that you have to decide in
your
questionnaire that how are you going to
record the
data then you have to decide the
sampling frame
sampling frame means in the step number
2
you already have decided that what is
the
population which has to be sampled now
you
would like to identify the sampling
units and
give them a particular tag a particular
address
so that the same sampling unit is
collected in the sample which is drawn
by some sampling
technique after this you have to decide
which scheme you want to opt for
selection of the
sample there are different types of
sampling schemes simple random sampling
stratified sampling cluster
sampling two-stage sampling etcetera
so depending on your objective depending
on the
type of population and depending on the
environment and the condition in which
you are going to sample
decide the sampling scheme every
sampling
scheme has some conditions and every
sampling
scheme works under a special type of
environment
especially types of requirement so you
have to choose
once you decide all these things so you
are basically
ready to start the survey but
do not go to the field but conduct the
pre-test
pre-test means before going into the
field
try to conduct your survey as if you
are in a real field for example if you
have prepared a
questionnaire try to give the
questionnaire
to your colleagues to your friend or to
people in your office
and ask them to fill it is possible that
they may face some problems in giving
the answers to different questions which
are
written inside the questionnaire then
you have to
revisit them you have to see that how
you can
modify them so that the answers to those
questions are
clearly obtained this is very important
suppose
if you do not conduct the pre-test and
if you go
directly to the field then it is
possible that the
surveyors may get get stuck or the
people do not understand the question
they will try to give the wrong
information and which will result into
wrong statistical inferences so it is
important that you
try to conduct the pre-test and then
based on the feedback which is received
from the pre-test
you try to modify your questionnaire
your sampling scheme or
whatever type of this drawbacks you
observe once you are completed
with this pretest now you are revise
your questionnaire and other steps
now you are in a condition to enter into
the real field
but before going into the real field you
have to look in the organization of the
field work
that means how are you going to work
when people are working in a real field
for example if i take an example
suppose if the government if
of india wants to conduct a survey at a
so the main offices are located in
the capital of the cities in new delhi
now they have to administrator
administer the survey in the entire
country
so what they can do or what they have
actually done
that they have their offices in
different states
in the capital cities mainly
then they have offices in different
cities
then i mean they can also have
some small setups in
in different locations now whatever
they want to get it done from the main
office they will
send it to the states the states will
will understand those requirement and
they will try to divert it to different
cities and then that may go to different
districts different villages and so on
and at every step they have to very
clearly define the
role of the investigators or role of the
persons who are involved in the
survey
means everybody is not allowed to take a
decision in case of any confusion
it has to be very well defined that who
is going to address
problem at that particular level
and based on that the reverse process
happen
for example the data is collected into
the villages
that is submitted to the district the
district will compile the data of all
the villages
then that will come to the city the city
will
will compile the data of different
districts
and then city will send it to the main
office
in the state the state will compile the
city of
the data from all the cities and then
will finally transmit it to the
the main office so these type of
organizations
are required for a successful conduct of
the
survey so and then
once you have this data
then it has to be defined before the
start of the
survey that how are you going to present
the summary
and how are you going to analyze the
data
now you can see once you come to this
step then all the previous
steps are interrelated
for example in the analysis if i
want to have only the frequencies or if
i suppose if i want to conduct
some test of hypothesis
then the requirement of the statistical
tool
will be different
for example if the requirement of the
statistical tool is that
the data has to be in the quantified way
then this has to be immediate in the
into the questionnaires
and they have to be mentioned very
clearly that what should be the
precision of the data because that is
the quantified data
height say hundred
sixty centimeter or hundred sixty point
two centimeter or hundred sixty point
two three centimeter
whatever it is
means if you want to conduct a
statistical tool
whose requirement is the data has to be
con quantitative but if you have
collected the data which is qualitative
then you are stuck and this happens in
real life so you can see first you
fix your objective based on that you
have to decide that
what you really want to find and then
what
statistical tools are going to be
utilized
then whatever are the requirement of the
statistical tool
based on that you have to you have to
decide for the variable
then you have to decide whether the
variables are going to be qualitative
quantitative or indicator variable
whatever it is then those types of
variable have to be embedded into the
questionnaire
so that the the answers of those
questions are going to give you the same
data set
which you are going to use at the end
for the analysis
ok so once you do
this thing then possibly you will
be able to conduct a good survey so
the next step is this that we need to
decide
how the data has to be present presented
how the summary of the data has to be
presented and how we are going to
analyze the
data what type of statistical tools are
needed
and this is also an objective that once
somebody is trying to conduct the survey
they are trying to collect different
types of data they are trying to collect
different types of information
if that information can be collected in
such a way
such that it is also useful in
near future or to fulfill some other
objectives
so these are the main principle
steps which one needs to follow
in conducting any samples
survey well these steps look
very simple
but when you really try to conduct
a real survey different types of
problem come they will crop up
and you have to handle them
what i have given here these are the
basic steps but depending on the
environment depending on the objective
depending on the type of survey
depending on the place of survey there
can be
different types of things for example if
in a country like india if people are
conducting the survey in different
states they have got different types of
language
the same vegetable which is
being called as name x might be called
as name y
in say in another state
or name z in different state
the unit of measurement of a
variable may also vary
for example say oranges at some places
they are sold in numbers by counting
one dozen two dozen at some places they
are sold by weight
so if you really want to have the
information on the oranges
you have to be very careful that how are
you going to
measure it how are you going to prepare
the question so that the question is
clearly
understandable by the surveyor as as
well as by the
respondent now
when we are going to conduct the sample
survey one of the
main objective in any sample survey is
that
the data has to be collected in such a
way
such that the variability
which is going to occur when the data is
used in the statistical tool
has to be as minimum as possible
so control of variability is an
important aspect
in any statistical analysis this
variability can be controlled at
different
steps for example i can use a
statistical tool
which will give me a statistical
inference with lower variability
but at this moment we are trying to
conduct the survey
so in case if i start controlling the
variability
right from the beginning then it is
expected
that at the end the variability is going
to be as minimum as possible
so the first step where you can control
the variability is the
survey or the sample survey where you
are trying to collect the
data so the question
is that how we can do it in
sampling theory okay so now
the variability control is a very
important issue at any statistical
analysis and we would always like to
have a statistical inference which
gives us the result with the minimum
variability
and in order to control the variability
through the sample service there are
various types of
sampling schemes which are adopted under
different types of
conditions and these schemes
essentially help us in controlling the
variability at different
stages now what are those stages
based on those stages we have
classified the the different sampling
scheme for example
there are three stages the the
variability can be control
before the selection of the sampling
unit after the selection of the sampling
unit and during the
selection of the sampling units so
when i try to control the variability in
the sample
before selection of sampling units then
there are
different types of sampling schemes
which
help us in doing so for example
there are there is a stratified sampling
scheme there is cluster sampling scheme
there are
there is two stage sampling there is
double sampling and so on there are
multi-stage multi-phase samplings
sampling schemes
which help us in controlling
the variability in the data
right what are those thing that is the
topic we wish will come in the
further lectures now sometimes
the variability can also be control at
the time of
collection of the sampling units for
example
the systematic sampling varying
probability sampling
etc they help us for example systematic
sampling is used
in a condition like forest right suppose
you want to see
what is the amount of the forest wood or
the amount of wood
of some particular trees now inside the
forest
you cannot create the sampling frame
because the sampling frame has to be
created
before the start of the survey one
cannot enter into the forest first
count the number of trees indicate the
number of trees
create the sampling frame and then start
sampling that is not possible
so under those situation what somebody
has to do
the person has to enter into the forest
and then yeah
with some pre specified rule he have to
he or she has to identify the
trees and then the required measurement
have to be taken
so similarly varying probability scheme
is also there that assigns
say different types of weight to the
sampling unit depending on their size
or probability of selection and so on
and third option is this once you have
collected the sample
then we always try to estimate some
parameters those parameters can be
population mean
population variance and other things so
once we have got the sample
then which of the statistical tool can
be used which can give us the
statistical inference with
lower variability and under this clause
there are two popular methods in
sampling theory which are very popular
one is ratio method of estimation and
regression method of estimation
ah one thing here i would like to make
very clear
that sometime people try to use the
terminology like as ratio and regression
methods of sampling
but these are not the methods of
sampling actually
actually they are the method methods of
estimation of parameters
they are not the method for drawing the
sample these are not the
ah samples sampling scheme these are the
estimation
methods so this is what you have to keep
in mind
ah before i move further
let me try to tell you here that as i
said in the beginning
that sampling theory has lots of topics
and this is a
big course but here in this course i am
not going to cover
all sorts of sampling scheme i will try
to cover only couple of
sampling schemes and my objective is to
give you the basic fundamental and to
train you
so much so that if you want to learn the
remaining sampling schemes you can
learn it yourself ok
now the question comes
in a typical survey how would you
collect the data there are different
ways by which we can collect the data
so i will try to give you here a brief
introduction
about those methods now that will be
your choice that
which method you would like to use
and that depends on the objective that
depends on the environment
that depends on the requirement and
practically that is a decision which the
surveyor or the person who is conducting
the survey
has to take ok so there are different
methods of data collection the first one
is
physical observations and measurement
that means
whatever is the variable on which the
the observation will be
required for example if the variable is
a is height or weight
then the height or weight of the persons
have to be
recorded on the on the
on those using some tool for example
weighing machine
or the scale to measure the height and
so on
second option is personal interview that
the
that the server will contact the persons
and yeah they can contact now now our
phone or
something like that or they can
personally visit the person
and they will ask the question
the person will reply the question
and the interviewer or the surveyor will
fill up the
questionnaire or in some cases yeah
means
the person can give the question add to
the
respondent and in consultation with the
surveyor the respondent can
fill up the questionnaire and can give
the data
third thing is mail inquiry
that can be a a sonic mail or a physical
mail
or a postal mail so the questionnaire is
sent to the people and they try to
answer the questions
nowadays web based
surveys have become very popular
there are some websites
which helps in creating the
questions and their possible
responses the advantage is that whatever
are the response responses
they are automatically compiled and
the time to compile the data is saved
sometime the data is collected from the
registration forms for example if there
is a contest
the then different people want to
participate they are asked to fill up a
registration form in which they will
give
all their personal details and then from
there
we can collect the data right
and then
there are some transcription from the
record for example
we have municipality offices there are
different types of government offices
where various types of informations are
available
and if one needs to go there and then
from those records one can
draw or record the required data set
nowadays some online forms are available
are also available for example google
forms they are one of the most popular
things nowadays similarly it's not only
google but then
different sites also help us in creating
a different types of form they can mean
they can be sent to the responder
through email
and link is actually sent by clicking on
the link
they come on the website and they try to
answer all the queries
and based on that their data is recorded
so now these are some possible ways
now another very important
source of data collection are different
types of cards
for example i had discussed this thing
in the
last lecture also some shopping store
some different types of chains office
stores
they have started issuing a card
and they give you some incentive for
example if you
buy ah buy something for hundred rupees
they will give you one point two point
three point four point
and they can be redeemed with some gifts
or with some cash or kind something like
this
so now what happens you go to the store
you try to from from a very big store
you try to
buy something and whatever you buy that
will indicate your choice and
preference and your requirement also
now once you come to the counter to pay
the bill
they will ask you do you have a card and
if you say yes they will swipe your card
and those amounts and entire detail
whatever you have purchased today all
the details will be entered in the
database of the card and the database of
the card is being managed by some other
company
which is trying to collect the data
and they are also not giving they are
also not
collecting the data from us for free for
that you are
you are given some points which you can
reimburse
so but this is one one of the
say cheapest way to collect the data
and now using that data they can analyze
the pattern of your shopping they can
analyze what type of
uh requirement do you have based on that
sometime they will send you offer
to attract you to the store for doing
more shopping and so on
for example if they say if they see from
the pattern
that this particular card
makes a good shopping on clothing from
this store
so they can sometime they can give you
an offer that ok if you
purchase the clothing for say
x amount of rupees then you will get
this offer some
some more discount and that will help
them in
increasing their sales so that is
another
way of collecting collection of the data
so this is how the data is collected in
a real survey
now i have given you almost
all the basic backgrounds different
types of definitions
concepts which are related to the
sampling theory
now from the next 10 i will try to take
up
the sampling schemes
and i will start with simple random
sampling
and then i will show you that
how the data is collected or how the
sample is collected
and when we are trying to implement a
scheme
how it is ensuring that the statistical
principles or the basic assumption of
statistics are also satisfied
and remember one thing they are not
going to happen automatically
you have to conduct the process in such
a way
such that those assumption get satisfied
so you revise your basic concept and i
will see you in the
next lecture till then good bye
[Music]
so
[Music]
[Music]
you
