In intelligence research, personality research and other research fields of psychology, factor analytic models are used to structurize the variables' jungle. For example: To simplify the variable structure of a personality questionnaire several questions (items), that are supposed to measure the same construct are grouped to one (latent) factor (e.g. extraversion). But does this group of items really measure extraversion? Maybe some of the items "load" on a different factor. And Maybe other aspects of the model have flaws as well... How well a factor model fits the data can be examined by a confirmatory factor analysis.
Keywords: Confirmatory Factor Analysis - Factor Analysis - Psychology - Statistics - Research Methods - Personality Psychology - Extraversion - Conscientiousness - Openness - Neuroticism - Agreeableness - Five Factor Model - Intelligence - Model fit - Model specification - model identification - estimation - loadings - model modification - maximum likelihood - Chi-squared test - degrees of freedom
Now that we know what it's good for, we want to know how to do it.
And obviously, the first step is that you need a model.
You need a structural model.
And this you get by using exploratory factor analysis
or you have a clear theory.
You have a hypothesis how the structural model has to look like.
For example, you say the two latent factors do not correlate.
There is no correlation between, in our example, extroversion and openness.
There is a zero correlation.
This also implies that the first two items, the first two variables,
do not load on openness, whereas the item he is hungry for knowledge
and the item he is happy to try out new things,
do not load on extroversion as well.
So these are model-implied restrictions,
which are accompanied by the restriction that the error variances
do not correlate with each other
and the restriction that the two factors in our model
have a variance of one.
This is just an intervention to have the same matrix.
So all in all, we have eight unknown parameters,
four unknown error variances, and four unknown loadings.
These parameters we will have to estimate.
And as we will see in a minute, we will follow two guidelines.
Guidelines one, these parameters will be estimated
in line with the model restrictions
and this estimation has to be as close as possible
to our observed data.
And the data we observe, the data which is known,
are the variances of the observed variables
and their co-variances.
These are the known informations.
All in all, four variances and six co-variances.
And in order to have the chance to estimate
the unknown parameters, the loadings and the error variances,
we need at least as many known parameters
as unknown parameters.
In our case, we have ten known parameters
and eight unknown parameters.
So there are two degrees of freedom and we can estimate
the parameters of our model.
If you should try to run a confirmatory factor analysis
and you have less known parameters than unknown parameters,
usually your statistics program will report an error.
Here you see it summed up again.
Frequently used methods to estimate the parameters
are the maximum likelihood, which is very often used,
or the generalized least squares method.
For both methods, the precondition is
that you have a multivariate normal distribution.
If this should not be the case, you have to use another method,
like, for example, the weighted least squares estimation method.
I won't go any deeper into this.
Again, you see the basic principle of the maximum likelihood method
is that the model implied co-variance matrix
is estimated along these two guidelines.
It should be in line with the model restrictions
and as close as possible to the empirical co-variance matrix.
So it should be as close as possible to our observed data.
After we estimated the parameters,
the next step is, and I already mentioned it,
to compare the model implied co-variance matrix
with the empirical co-variance matrix,
with the real data.
Well, to be honest, in our case, this is made-up data.
It's just to understand the principles.
But on top, you see in the model implied co-variance matrix
the results of our restrictions.
We said that the two latent factors don't correlate,
so the variables 1 and 2 correlate with each other,
but they don't correlate with the variables 3 and 4,
because those don't load on the same factor.
If we subtract the model implied co-variance matrix
from the empirical co-variance matrix,
we get the matrix of residuals,
which tells us how huge the difference is
between the two matrices.
In fact, it's better to use standardized residuals
because of the metrics,
but for our example, to understand the principle,
it's okay to take a look at this.
What we see is that the difference
between our model and the real data
in general isn't that big, so this is okay.
But there is one, well, let's say outlier,
because almost all residuals are almost equal to zero,
but the co-variance of item 2 and item 4
is much higher than our model predicted,
so this is something which needs further observation.
Maybe there is a good explanation, maybe not,
so we might have to adjust our model.
But looking at the residuals,
or as I said, it's better to look at standardized residuals,
is only one method to examine the accuracy of our model.
In fact, quality measures can relate
to detailed aspects of our model,
but it can also relate to the overall model.
Then we might, for example, use
the standardized root-mean-square residual,
or the g-square test,
or another often used method
is the root-mean-square error of approximation.
But it's also possible to compare two different models,
so you might compare the one-factor model
with just one latent factor, for example, extraversion,
with a two-factor model or a three-factor model,
or you can change other aspects of the model,
and then you can run a likelihood ratio difference test,
or information-theoretic measures like AIC or BIC,
or you can run the incremental fit indices.
As I already said, I won't go any deeper into this.
These methods are well described in books on this topic.
Here you see a typical output of model fit statistics,
we have two degrees of freedom,
and all the values are quite acceptable,
so this would mean we have a quite good model fit.
For all the statistics of model fit,
there are cut-off values when to accept a model,
or when to speak of a good model fit,
or when to speak of a bad model fit.
And what can you do if the model fit is quite disappointing,
it just didn't work,
all the values or some of the values
are not the way you want them to be.
Well, you can think about model modification,
you can think about re-specification.
For example, if you observe any low loadings,
you might say, if from a theoretical point of view it makes sense,
you might say, okay, this loading is zero,
and then you can run the model fit statistics again.
But you could also evaluate the error variances,
are those too small or negative?
You could think about, as we did in our example,
an additional factor.
In our example we chose openness.
And of course you evaluate the measurements,
you look at the standardized residuals,
you look at the modification indices,
and you should also watch out for correlated error terms.
But you should keep in mind that model re-specification
is not the best idea.
Ideally you have an a priori model which works,
which really fits the data.
If you have to run many re-specifications,
you should always plan a replication.
Because otherwise chance is having a too big impact.
That's it for today.
I hope you can use this episode as an introduction
to confirmatory factor analysis.
Of course, if you want to go deeper,
you will have to spend some more time.
You will have to take a closer look
at the statistics and calculations.
But I think as an introduction,
this episode should give you some guidelines.
So have a good time and see you next episode.