Reliability - How exact is the test? (Psychology) - Part 2

www.LearningPsychology.net
87 followers
1 Video View·Jan 29, 2025
Reliability is one of the most important quality aspects of measurement instruments. It can be fatal if measurement instruments are not reliable.
But how can you determine the reliability of a psychological test or
questionnaire?
To estimate reliability several methods can be used which
have pros and cons:
a) test-retest reliability method
b) parallel-forms method
c) split-half method
d) internal consistency
Author: Eskil Burck (Degreed Psychologist)
www.learningpsychology.net
One obvious weakness of the retest method is that people remember what they answered
in the first test. So in case of a personality questionnaire which is presented at test point
one and then maybe two weeks later, some people tend to give the same answer like they did two
weeks ago because they want to present a consistent image of themselves. We could also say
they want to be reliable. So even though they might feel quite different from how they felt
two weeks ago, they might have the tendency to answer the questions in the same way they did
during test one. And in case of ability tests like an intelligence test, it can happen that
some people who are very conscientious begin to search for similar tasks on the internet because
they want to improve their results in the second intelligence testing. So this would be an
unsystematic change which leads to an underestimation of the reliability of the test.
So these are some weaknesses that can be overcome by the so-called method of parallel test forms.
The idea is you don't present one and the same test but you present two slightly different
parallel forms of one and the same test. The difficulty about the parallel forms method
is to create the parallel form. Especially in personality questionnaires, maybe a questionnaire
on charisma, you already tried really hard to find the right items, to find the right questions,
and now you are supposed to find even more items that are slightly different but measure exactly
the same like your original charisma questionnaire. And this sometimes can be really difficult. So
the parallel forms method is not that frequently used. In case of ability tests, it's usually a
bit easier because, for example, in case of a maths test, you just have to exchange a number.
Let's suppose the original item was what's the result of 3 plus 7. And so the parallel item could
be what's the result of 2 plus 7. And participants probably won't have much difficulty in solving
both items. So there's going to be a high correlation between solving the items from
test form A and solving the items from the parallel form B. Another method that is more
frequently used is the so-called split half reliability. All you do is to split your test,
your questionnaire into two halves and correlate them. Because in order to be reliable,
both halves should be measuring the same. So in case of an intelligence test, you can use the
so-called odd even method, which means for the one half you choose the odd items like 1, 3, 5,
7 and so on. And for the other half you are going to correlate with test half 1. You choose
the even items like 2, 4, 6, 8. This usually works quite well. And one really big advantage of this
method is that you don't need a retest. You can present one and the same intelligence test
at one time and later on you split the test into two halves and then you correlate the two halves.
This is not only more economic, because sometimes it's difficult to get the participants to the
laboratory for two times. Another advantage is that there are not going to be unsystematic
changes due to the time interval between the two tests. Because if you chose a retest method,
it can happen that one group of people had to solve intelligence tests for another occasion.
And so this group is now going to score higher, because they are now better used to solving
intelligence tests. And this obviously would decrease the estimation of reliability.
The last method we are going to talk about is also very often used, because it's very practical.
It's the so-called method of internal consistency. Like in case of the split half reliability,
which in fact is a method of internal consistency, we try to find out whether the items
of our test all measure one and the same thing. So in the split half method we correlate the two
halves. But you can also correlate little parcels of items or single items with the rest of the
test. So if all items of an intelligence test or of a personality questionnaire correlate with each
other very high, this is an indicator for a high internal consistency. And the items reliably
measure the same thing. Usually in test manuals the so-called Kronbach's alpha is reported in
respect to internal consistency. So that's been the episode on reliability. Another really important
quality aspect of measurement instruments that needs attention if you are developing a new test
or if you are struggling about which test to use in your scientific, school, psychological or
clinical context.
