Lecture 16

IIT Kanpur July 201
20 followers
Aug 9, 2024
[Music]
let's start with this case study problem
on auto parts manufacturing company Z
Limited in this case study we'll apply
the concepts learned on the topics
related to inferential statistics and
hypothesis
testing an autop parts manufacturing
company company Z limited conducts
regular training and development
programs for its thousand
employees it wants to check the current
level of satisfaction of employees and
that after the training program you're
the chief data scientist in the
consultant firm engaged by Z
limited the firm has given you the
primary survey response data and the
feedback taken from these thousand
employees pre and post training program
and wants you to examine the impact of
the training program on the employee
satisfaction
the data provided which is in the data
file data. CSV to you comprises three
key variables first variable is employee
ID second employee response data before
conducting the survey this data is taken
on scale of 1
to8 the data file uses the variable name
pre to reflect this data third we have
employee response data after the
survey this data is taken on a scale of
1
to8 the data file uses the variable name
post to reflect this data both of these
variables are extremely important from
the analysis
perspective very often individuals are
slightly biased in their
responses some are biased on the upside
While others are biased on the
downside based on the response of these
individuals to survey questions the
responses are corrected for this
bias this bias is corrected through
scaling the data and that is why the
data is in the form of fractions unlike
the original data which was collected in
integer form on a scale of 1
to8 we'll analyze this data using our
programming language Let us start the
case analysis by summarizing and
visualizing the survey response data to
conduct a preliminary analysis using R
tool as a first step towards the data
analysis we'll understand different ways
of reading data in R next we'll
summarize the data in R there are
various commands and functions that help
us in getting a brief overview summary
and structure of the data there are
various ways in which summary measures
help us gge broad Trends available in
data using these summary measures we'll
try to understand the key differences
and contrast the pre and post trining
survey
responses next we'll plot data using
plot functionalities of R we'll also
understand how to improve the aesthetic
aspects of these plots subsequently
using plot technique we'll superimpose
the survey response pre and post
training data we'll also examine the
density distribution of these variables
to understand the impact of
training next we'll examine the measures
of central tendency for our pre and post
variables the measures of central
tendency examined here are mean median
and mode in addition we'll also analyze
the quantile wise distribution of data
overall these statistics will help us
gge whether there is some significant
impact of training on imply
satisfaction next we'll examine the
measures of variability the measures of
variability studied here include range
variance standard deviation mean
absolute deviation that is
mad these measures will help us
understand the variability in the
variables pre and post training on
survey
responses very often distributions are
simply defined by two parameters
including mean and variance and and
therefore the variability of key
parameters in a study is very important
to
understand lastly we also examine the
shape
parameters this includes skus which is a
measure of symmetry curosis which is a
meas measure of pness and normality of
data we'll first compute these measures
for a simulated normal distribution and
then compare these measures for our
variables pre and post that is survey
responses before and after training did
moreover we'll also use our
functionalities to statistically test
whether these deviations from normality
are indeed significant or not this
includes test of normality like Haka
test test of skewness and test of
curtosis lastly we'll also plot the
histogram of the data to visually
inspect and confirm the deviations from
normality now that we have understood
the data let us start by reading and
writing it
first we will try to analyze the data
and as a best practice I would recommend
that we set our working directory
first so we'll click on the session
button we'll click on this set working
directory and then choose the directory
appropriately once you click on this
open you will notice that a command has
appeared on the console
window as the best practice I would
recommend that you copy paste this
command on your current
script the idea here is that in future
whenever you are working on this script
later you need not remember the exact
location where your current working
directory and all your data is
available notice on the file tab here
you will find all your current working
directories and data available directly
there if you want to
read a data file let's say data. CSV you
can click on it once you click on this
you will get an import data set button
and if you click on this import data set
button it will give you the shape and
form of data how it is looking if you
click on import this data will be
downloaded and uploaded in your current
B directory in the current form there
are some options for example you can
choose first row as names trim spaces
and so on as may be found suitable and
you click on import and you will again
notice First Data is visualized in short
form on your current Tab and also you
can notice that a command has run which
you can directly use in future as well
that is one way to read the data
here also the data variable is appearing
on your current working environment that
is also there if you click on it again
the data will be visualized briefly on
your current
script another way to import data is to
click on this data set appearing in the
environment Tab and then text button if
you click on this text button it will
give you some options to read the
current working directory and you can
click on the data file double click and
it will again give you some form of
visualization of data how it will look
like if it is imported into the current
working directory if you click on it
again command file is run you can see
that on my console window the command
file that has run
lastly there's another way to read this
data is
through read. CSV command for example I
can write this
command notice I have not given the
exact location of the data from where it
is to be read simple reason is I have
already set my current working directory
it is one of the major benefits of
setting the current working directory
that you need not specify every time
whenever you are reading or writing the
data it will automatically take this
particular
location however if you want to read and
write from some other place then you
need to specify that location separately
but if you're reading the data from the
current working directory or writing it
there then you need not specify the
address
location now that we have read the data
let us start by examining the nature of
this variable
as a first step you would like to know
the nature of this data and we'll
examine the nature of this data by
running this class
command notice that this data is a data
frame variable a very important class of
variable in our
format a data frame is a kind of
variable that is a collection of
different columns each column itself is
a variable and each column variable can
be of different nature for example it
can be numeric it can be character it
can be factor and so
on let us have a brief overview of this
data now so I will use the head command
and once I run the head command on data
I can see some of the elements initial
elements appearing on my console tab I
can see those elements there are three
variables it seems as we have discussed
earlier about this data first is
employee ID then pre which is the survey
response variable before training and
development then post which is the
survey response variable post training
and
development we can also see some of the
last Elements by running this tail
command and you can see some of the last
elements starting from 995 to 1,000 have
appeared if you want to check the
dimension of this data we can simply run
dim command and and you can see there
are thousand rows each row is an element
as we already know from this data frame
and there are three columns which is
also appearing on the
console now let us summarize this data
and we'll use the summary
command it'll be interesting to note
that all the three variables have been
summarized the pre and post both of the
variables have been summarized like they
are numeric variables you can see their
minimum first quartile median mean Max
Etc are appearing also you can clearly
see as per this summary there's a shift
from pre to post there's a certain shift
that is appearing it gives you a look
and feel that there's a certain shift
after training but it's still more
analysis more statistical procedures are
required to establish that to a good
extent one problem as you would notice
that employee ID variable has also been
summarized like a numeric variable of
course it is not a numeric variable but
the way R works it tries to Summarize
each variable in a manner that there's a
minimum loss of
information so it has summarized it in a
numeric
form next another way to summarize the
data or to see what are the contents of
this data frame are to use structure s
Str command and as I run this s Str
command
notice that it tells us that there are
three variables we can clearly see on
Console bu it gives me a brief overview
of this data frame it tells me that
there are thousand observations and
three variables that is first set of
information then it also tells me that
there are three variables employee EMP
pre post and what is the nature of those
variables for example employee EMP ID is
taken as integer although it is not
exactly an integer variable but R is
recognizing it as that
we can also change the way I recognizes
certain
variables then we have pre which is the
survey response variable before training
R has recognized that as a numeric
variable it's a fraction of course
again this post variable which is again
a numeric variable in a fractional form
remember from our earlier discussions
that the data has been scaled from
integer to numeric for Bas correction it
also tells us that in order to access
these variables inside the data frame we
need to use dollar sign for example if I
want to access the pre variable I need
to type data and then dollar sign this
is called tab completion a very
important property of our data so with
this tab completion I can click on Tab
button after dollar sign and you can see
all the three columns are appearing I
can select one of them for example I can
select three so this is how we can
summarize the data see the structure of
data frame in which all the variables
are there and we can also get a brief
summary and brief overview of our data
that is available for analysis with this
now that we have understood the
summarizing the data and we have now the
overview of data we will move to the
visualization part and plotting of the
data now that we have got a brief
overview of data it structure and
summary let us start by visualizing the
data in this video we'll see how to
visualize the data using r a lot of
interesting insights can be obtained by
simply visualizing the data for example
you can see if there are any outliers or
with the help of density plot you can
also see if data follows a particular
distribution like normal
distribution a very basic and simple
plot command available with our for
example plot command we'll apply on a
data
variable notice our plot tab I'll Zoom
it to improve the
visualization notice both the variables
pre and post are appearing here for pre
variable the range is 3 to5 which
indicates its
observations and similarly the value
range for post is 4 to6 this clearly
suggests that because of the training
there is a clear increase in the
employee satisfaction pre to
post this provides us with some initial
intution
to the impact of training and
development program we can also use this
plot command for a single
variable pre and the plot appears on my
plot window I can either save it as
image for example I can save it as PNG
or jpeg
image I can change the width and height
of this
plot also you can export this as PDF or
you can copy it to clipboard as well and
subsequently paste it on your current
working directly maybe a word or PP
document then you can do a lot more to
improve its Aesthetics for example as a
starting point let's plot this pre
variable and in the argument type I'm
specifying P this will tell r that the
plot needs to be made in point form
which I've already done so it will not
make a difference however I could have
instead of P used L as well which would
have plotted in the line form but as we
can see the line plot may not be too
appropriate for this kind of data so
we'll stick to the point
plot we can also add some colors here
for example I can specify the color with
Co argument as
red you can see the points are plotted
now in red
color we can further improve this plot
by adding Legends and access labels for
example we can add the main Central
heading maybe I can name it as graph
I can also add access labels for example
X access label by xlab argument set to
data similarly I can set the color to
Red which is already there so we'll run
this command and
notice once I run this command Central
heading
appears
and I can also add y AIS label Maybe I
can put it as
frequency so it will change the y-axis
label and so on lot of aesthetic
Improvement we can do here to make it
look more professional now many times we
are interested in density plots a simple
command or function which is density
function I can use here and apply it on
my predata to plot
density now here in the central argument
main heading I can put it as density
graph I can also specify my xais label
by xlab
argument as
data I can add a color
of
red I can add certain more arguments
maybe I can add a line with let's say a
line bit of
one let me run this command as you can
see now I
have obtain the same plot that I wanted
in red color the density plot probably
the line with his slightly low so I can
increase the line width by using lwd
command setting it to three so now my
line width has improved now I would like
to superimpose the post observations
after the training and development
program for that I need not write the
plot command again I need to use the
same plot because I want to superimpose
and in that I'll use the lines command
and I use the density function on my
post
data to tell r that the line
corresponding to density plot on post
variable are to be added superimposed on
the original plot I give it a color of
blue to differentiate between the old
and new plot and I give a line width of
three if I run this you can see that now
both the graphs are appearing
superimposed on one plot but there's
some problem with this because X XIs
width is limited up to five however it
seems the post variable because there's
a shift in the variable is not
completely coming inside the graph so I
will make some small modifications by
adding a limit xlim argument which
specifies the limit of x-axis labels and
I'll specify the limits between 2 to 7
so on the right side I'm increasing
the graph width a little bit and you can
see now when I'm plotting the plot
original plot there is certain
additional space on the right side which
will give me some allowance and now I'll
learn the lines function where post
variable is there and you can see now
both of these plots are appearing within
my plot boundaries and clearly they are
showing that there is some increase in
the post variable after training and
development now I can also add Legends
here very simple I will use the
Legends function I'll specify the
location as maybe top
left then I need to specify the names
I'll go with the same Convention as post
and
pre and then I'll add the fill color
I'll again follow the same Convention as
blue for
post and red for pre
notice that Legend has appeared I can
change the location of this Legend very
easily for example instead of top left I
can give it top
right or for that matter on the bottom
side also I can give
it and so on so a lot of permutation
combination are possible however this
graph clearly indicates that there is a
certain impact of training and
development pre and post on the employee
satisfaction
now that we have understood the data
visualization we'll move to examine the
measures of central tendency with the r
implementation that gives us more robust
and quantitative
estimates now that we have understood
the data visualization we'll move to
examine measures of central tendency
that provide us a more robust
estimate a measure of central tendency
is a summary statistic that represents
the center point or typical value of a
data set
in statistic the three most common
measures of central tendency are mean
median and mode we'll start with mean
and it is a sum of observations divide
by the total number of observations it
is also defined as an average which is
sum divid by count in R the syntax
mean is very simply implied like this
mean inside I'm putting my variable of
interest which is pre and I can get the
value by running this command which is
4.7 now instead of printing this value I
can also assign it to a new variable
which is M1 by running this kind of
command this is assignment operation now
the variable M1 contains this value of
mean for pre variable similarly I can
use almost similar command to generate
the mean for my variable post which is
the survey responses after the training
program
it is quite easy I can print the mean of
post which is 4.9 98 I can also assign
it to a variable
M2 notice that this variable
M2 is 4.98 which is more than the mean
of pre which was
4.7 which is an indication that the mean
has
increased very often
mean also represents the expected value
of a variable for example when you are
Computing expectations often you do not
have probabilities assigned to you so
you used historical observations
assuming that each observation has a
equal weight and therefore the averages
would represent equal probability
weighted values and therefore also
represents the expectations of the
variables and that is why mean is also
very
important sometimes the variables
contain
na observations which is not available
in those cases if you hunch or your data
collection exercises suggest that there
may be some na kind of variables in your
data then you use na. RM equal to T true
that means you are telling your our
console that this variable may have any
observations and if there are kindly
remove them if you run that now your
mean will be computed ignoring all the
na observations so this is how you
tackle na observations in the
data next we'll compute median median is
also very important measure of central
tendency it is the middle value of data
set that means it divides your data set
into two
halves so
first we computed mean now we are
Computing
median the syntax to compute median is
also very
simple we'll type this median command
and first we'll try to see the median of
our variable
PR the median of this variable is 3. 99
we will also assign the value to a
variable called M1 we can do that very
easily like
this similarly I can compute the median
of my post
variable notice the median is 4.98 I can
again assign this value of median to my
new variable M2
the value of this variable M2 notice is
4.98 now again an indication that M2 is
greater than M1 which suggests that
there is some impact of training on the
median of the
data after
median we will compute another important
measure which is
mode for mode uh this is the value that
has the the highest frequency in a given
data set also if you're talking in terms
of probability density mode is the
variable where highest probability
density is there for given set of values
now one way to run this is
install a package called staup so we'll
use this install do packages command to
install this new package called
staup once you install this package STP
what you need to do is you need to
Simply add this to your current working
Library by running this Library startup
command once you install this Library
EST static you can very well use the
function mfv to compute probability
density the way this function works it
organizes or ranks observations
according to their frequency that means
those observations which are highest in
terms of frequency or they have the
highest probability density are put
earlier than those that have lower
probability
density so in case if you want to see
the top five 10 or some initial values
you run head command as we are very
familiar with the head command we can
print a set of initial values for
example let's say I want to print
initial five values I can run head
commands like this and it will print my
initial five
vales so you can see these are some of
my initial five values printed on my
console for my pre variable
aable similarly I can run this for my
post command post
variable
notice that the mode of my pre variable
versus post variable is not same in fact
it appears that there's a considerable
increase in the mode of the data after
the training and development program so
in this fashion like median and mean
mode is also helping us understand the
impact of training on our satisfaction
of
employees lastly we'll also examine the
quantile measure a very important and
interesting measure what are
quantiles again quantiles are a measure
that divide data into different
different buckets for example if you
dividing your data into four buckets of
equal sizes you will call it quti that
means q1 Q2 Q3 Q4 and each bucket would
25% of the observations similarly if you
divide your observations into 10 equal
buckets you will call them desiles that
means each bucket would n 10% of the
data sets as a smallest Division if you
want divide them into 1 percentile 1
percentile bucket then you call them
quantile now it is quite easy to compute
quantile in r with functional quantile
use function quantile let's see the pre
variable quantiles by default our
computes quartiles for you starting from
0 percentile to 100 percentile it
divides data into four buckets and you
get the corresponding observations in
this particular case you get 0
percentile at three 25% at 3.53 50% at
3.99 and so on similarly
I can compute the quantiles for variable
post as you can see for each of the
quantiles 0 25 50 75 I get a relatively
higher value as compared to variable pre
which again confirms to our earlier
intuition that there is definitely some
increase in the custom in the employee
satisfaction now let's say instead of
these default quantiles you want to
compute certain specific quanti let's
say you want to
compute 95 percentile for your pre
variable quite easily computed you can
simply type this syntax you can specify
the quantile that you want to compute
and it will tell you that 95 percentile
is 4991 for pre variable similarly you
can compute 95 percentile for your post
variable and it appears to be
5.89 7 which clearly indicates that
there is some increase in the
satisfaction now you need not stick
yourself to only a particular set of
poti and you can use a very important
function called sequence function to
generate quantiles as you may want for
example I can use a sequence function
like this and specify the sequence
starting from zero ending at one with an
interval of 0.2 so it will generate a
new sequence of quantile which will
cut off at points 0 percentile 20
percentile and so on let's see the
output as you can see a very interesting
set of observations are appearing for
five buckets that cut at 20 percentile
40 60 80 and 100 exact same sequencing
we can produce for post
variable and it will give us again the
five divisions but you can clearly see
that observations corresponding to each
of the quantiles that is 20 percentile
40 percentile 60 percentile have
increased considerably as compared to
their pre-training
values overall these observations
clearly suggest that now there's a shift
in the distribution of our variable of
interest which is employee satisfaction
after the training so with this we
understand that measures of central
tendency clearly indicate that there is
some shift in the distribution of for
employee
satisfaction in the previous video We
examined the measures of central
tendency in this video we'll discuss
about the measures of
variability and how to estimate
them so we'll talk about measures of
variability in this
video measures of central tendency
yielded information about about the
central or middle part of the data set
however business researchers can use
another group of analytical tools to
measure the measure of
variability to describe the spread or
the dispersion of data set using
measures of variability in conjunction
with measures of central tendency makes
it possible for a more complete
numerical description of data there are
a number of measures of variability some
of the most important ones include range
variance standard deviation and mean
absolute deviation that is
mad so we'll start with the range
measure range measure of
variability the range describes the
difference between the largest and
smallest data point in our data set that
means the bigger the range the more is
the sprit of data and vice versa
the syntax to measure the range is quite
easy
range and then we'll apply a data set
which is
pre we can see the range of pre data is
3 to
4.99 I can also check the range of my
post data which is again quite
easy it is 4 to
5.99 the results clearly suggest that
post training exercise the range has
shifted
although the spread of the range is not
very different but there is a shift in
the range that is shift in smaller and
larger
values our next measure of variability
is standard deviation and
variance so first we'll talk about
variance variance measure is the average
of square deviations about the
arithmetic mean for a set of numbers
the syntax of this variation measure is
VW which is short for
variance and then we'll put our data
which is pre variable the variance is
0.32 very similarly I can compute the
variance of post
data very close to earlier value which
is 0.3 to5 which we also got some
incling from our range measure that
there's not much change in the the
variation only there's a shift in
data the third measure of variability is
standard
deviation standard deviation measure is
square root of variance that is standard
deviation measures the dispersion of
data set relative to its mean it is
defined as the square root of variance
and if the data points are further from
the M there's a higher deviation within
the data set the the more they spread
outside the data the higher the standard
deviation the syntax to compute standard
deviation is quite simple St data dollar
pre which is our pre variable the
standard deviation which is
0.57
similarly I can compute the standard
deviation of my post
data which is very similar to the
earlier value again 0.57 which again
confirms that the spread is almost
similar
the last measure that we are discussing
is mean absolute
deviation
mean absolute deviation often referred
to as M ad measure this
measure is the average of absolute
values of deviations around the mean for
given set of observations the
computation and syntax is quite simple
M data
dollar pre so the mean absolute
deviation is
0.721 similarly I can compute the mean
absolute deviation for data
post again not very different slightly
lower but
0.70 so not much shift in variation pre
and post training program however as we
have seen the range and other measures
of central tendency we can clearly say
that while there's not much difference
in the dispersion of the data before
training and after training but clearly
there's a shift in the distribution that
means the lower as well as higher values
have indicated a kind of shift on the
positive side after training however to
get some more idea about the variables
and the impact of training activity
we'll also examine the measures of shape
of inferential
statistics now that we have made
ourselves familiar with the measures of
Cent tendency and the measures of
variation in this video we'll discuss
about the measures of shape and their
implementation with our
software we'll talk about measures of
shape in this video and in order to
implement these measures we need to
install two very important packages
first
is time series
package the second one is m ments
package these two packages provide very
important functionalities for
implementation of some of the shape
measures once you install these packages
you need to put them in your current
working library with the library
command like this first I'll run my time
series package and then I'll run Library
command for my moments package
now once you run these libraries they
will be installed in your current
working
directory first and foremost we'll talk
about the measure of
skewness the distribution of data in
which right half is mirror image of left
half is said to be
symmetrical one example of the
symmetrical distribution is normal
distribution or bell curve skewness is
obtained when distribution is
asymmetrical or La symmetry as you would
have guessed normal distribution has
zero
qess so in order to check the qess of
different
distributions first let us start with an
normal distribution let's generate a
normal distribution and give it a name
NM we'll use our simple our Norm command
to generate 10,000 observations for
normal distribution and we'll keep using
this for our investigation of shape
measure so first let us compute the
skewness of this
Norm
variable as you can see it is very close
to zero which suggests that the
distribution has a the skewness which is
closer to zero let's compute the
skewness of
our pre
variable it is also quite close to zero
which suggest the data has a SK of Zer
let's compute the skewness of our post
variable
as you can see this is also very close
to zero that means our pre and post
variable conforms to zero skewness and
which is quite similar to normal
distribution however this just a measure
and it doesn't give any statistical
confidence or Assurance so we can also
perform a guo test which is a more of a
stat statistical measure of
skewness so this austino test let us
first start with our NM variable as you
can see the P value is 0.34 which
suggest that the null hypothesis cannot
be rejected that means the data does not
have a
skewness the same austino test we can
perform on our
predata you can see again the hypothesis
cannot be rejected that means the data
does not have a skewness same test we
can perform our post data also
so again the hypothesis cannot be
rejected which suggest that our data
does not have skewness whether it is pre
or
post next we'll talk about curtosis so
curtosis is a measure of peakness and
we'll discuss more about this kosis
measure kosis describes the amount of
pigness of a distribution distributions
that are high and thin are referred to
as lepto cartic
for them the curosis value is greater
than three similarly the distributions
that are flat and spread out are
referred to as plat cartic distributions
for this the value should be less than
three and as you would have guessed
between these two types of distributions
there are normal distributions which are
also called mesokurtic distributions for
them the value should be equal to three
so let us try and compute this curtosis
value for
our normal distribution first C CIS very
simple to compute let's use this for our
normal distribution and it seems the
curosis is very close to three which was
expected because we simulated a normal
distribution let's compute this curtosis
for our pre-
dat you can see the curosis not close to
three it is in fact much far away than
three similarly I can compute curosis
for my post dat also
also again the kosis is further away
from so it indicates that probably these
pre and post variables do not have kosis
that resembles a normal distribution but
to perform a statistical test which will
give us more confidence and more
statistical significance we'll use anom
test the test is provided in the movs
package first we'll perform it for the
normal data
and as you can see the hypothesis that
it is a c it curosis equal to three
cannot be rejected that means curtois is
indeed equal to three for that normal
distribution we'll perform the same test
for our
predata you can see the kosis is clearly
not equal to three the hypothesis is
rejected very
significantly so the kosis is not equal
to
three let us again test for the Post
data again the cutos is not equal to
three which means that for this
distribution the cutos is not equal to
three now we'll test for an overall
hypothesis the data is normal or not so
we'll test for the hypothesis of
normality in statistics Hara test is
widely available to test the normality
of the
data Hara test is available in uh time
series package as well as moments
package so first we'll
implement this Hara test for time SE
package which uses Hara do test command
and we'll use it for our normal variable
which is Norm and you can clearly see we
cannot reject the hypothesis that it is
normal and that means this obviously we
simulated the normal data itself so we
cannot reject the
hypothesis let us test this for our pre
variable
now as you can see for pre we are able
to reject the hypothesis with lot of
confidence indicating that the data
probably is not
normal for the Post variable if we test
this hypothesis again we are able to
reject this with a lot of confidence let
us try to run this har Bea test with the
different
package so we'll run this for our
moments package which is the command is
the har do test and first we'll run it
with our normal variable
for the norm variable again obviously it
was a normal distribution simulated so
it cannot be rejected the hypothesis
cannot be rejected similarly we'll test
it for
our pre
variable and again as we obtained the
same result earlier we have to reject
the null hypothesis that means it is not
a normal distribution in a very similar
manner we can run our post data also
which also suggests that it is not a
normal
data now that we have run
normal distribution test let us
visualize this data again to see if
indeed the visual inspection suggests
some kind of presence of
non-normality and for that very easily
we'll first start with a normal
histogram we all know what is histogram
so first let us run with this hist
command and I'll perform this hist
command on
our normal data to see how it
looks and you can see it's a very nice
shaped well shaped normal curve
currently on y- axis you have frequency
Dimension if you want instead of
frequency one
can use
probability I can put this feature
probability equal to two and instead of
frequency now you'll have probability
density the diagram will remain similar
moreover if you want to superimpose the
lines of density over this histogram you
can very simply use lines command and
you can plot
density now to make it more aesthetic
appeal you can add some further features
like line withd equal to three so
there's a solid line will appear and
let's give it a color of
red so now this histogram will give
me a dotted solid red line which shows
as we can see on the
graph a normal distribution and now
let's plot the pre and post variables on
the same plot so let's start with
plotting the pre variable instead of
Norm we'll use our pre variable data
dollar pre let's plot it program
obviously it's slightly different in
fact than the normal distribution it
appears to be symmetric but obviously
the peak is not very similar which we
would have anticipated given our earlier
estimations instead of frequency let us
plot the probability density
now as you can see obviously the diagram
remains similar but we get an idea of
probability density as well we can also
plot the lines probability density lines
and we can see the density lines
indicate that the curtois is very
different from a normal distribution
again in a very similar manner I can
compute the histogram and the density
plots for post
variable look at the histogram again
it's not in terms of curtois it is very
different it's symmetric although around
its mean but it's still it's curtosis
very different than a normal
distribution we can again plot the
density plot and plot the density lines
also so now it is clear that this is not
a normal distribution it is symmetric
though but since curtosis is different
and why it is not normal as we already
saw the overall normality hypothesis was
rejected but it was not rejected on
account of skewness but it was rejected
on account of
curosis but an interesting question
arises while we have been able to reject
the hypothesis of normality for both of
these data sets in subsequent
applications of hypothesis testing
interval estimation Etc will be making
use of normal distribution why so as we
have already studied in centrer limit
theorem that distributions of sample
means approach normal distribution when
you have large number of samples if
sample size is more than 13 using the
central limit Theory we will be able to
use normal distribution to model these
data sets
[Music]
[Music]
[Music]
