Week 12_Measurement_Scaling, Reliability, Validity

Scaling, Reliability, and
Validity in Research
Research Methods for Business Students
Authors: Mark N.K. Saunders, Philip Lewis, Adrian Thornhill
Introduction to Scaling,
Reliability, and Validity
Understanding scaling, reliability, and validity is

crucial to ensuring research quality and consistency.
These concepts allow researchers to measure
variables accurately and produce replicable findings.
Scaling in Research
Scaling involves assigning numerical values to qualitative data,

enabling quantitative analysis of attitudes, perceptions, and
opinions.
Example: Customer satisfaction measured on a 1-5 scale, where 1

indicates 'Very Dissatisfied' and 5 'Very Satisfied'.
An illustration of scale scores
Strongly Agree Un Disagree Strongly
Agree decided Disagree
a. If you start trying to change things

very much you usually make them 4 3 2 1 0
worse.
b. If something grows up over a long

time period, there will always be
much wisdom in it.
c. It is better to stick by what you
have, than to try new things you do
not really know about.
d. We must respect the work of our
forebears and not think that we
know better than they did.
e. A person does not really have much
wisdom until they are well along in
years.
Continue….
Person a b c d e Scale Interpretation
Score
1 4 4 4 4 4 20 Very highly traditionalist
2 4 2 1 0 3 10 Moderate
3 2 1 2 1 1 7 Moderate
4 0 0 0 0 0 0 Very Non-Traditionalist
Types of Scales
Common Types:
– Nominal: Categories without any order (e.g., gender).
– Ordinal: Ordered categories (e.g., rank positions).
– Interval: Ordered with equal intervals but no true zero (e.g.,
temperature).
– Ratio: Ordered with equal intervals and a true zero (e.g.,
income).
Importance of Scaling
Scaling transforms qualitative attributes into

quantitative measures, allowing statistical analysis.
Benefits:
•Facilitates comparisons
•Standardization
•Simplifies complex data.
Likert Scales
The Likert format provides five response
alternatives which gives more flexibility. It
provides a measure of intensity, extremity
and direction. If needed you can later
collapse five categories down to two or
three
Stages:
1.constructing a rough scale;
2.selecting the best items;
3.creating the final scale variable
Example of Likert Scale
A popular scale for measuring attitudes, where

respondents indicate their level of agreement.
Example: A statement like 'I am satisfied with the

company’s customer service' with responses ranging
from 1 (Strongly Disagree) to 5 (Strongly Agree).
Strongly Agree Agree Un- decided Disagree Strongly Disagree
1 Eat, drink and be merry for tomorrow we die. 1 2 3 4 5
2 I am a fairly ambitious person. (+) 1 2 3 4 5
3 If people thought ahead more we would not be 1 2 3 4 5

facing many of today’s problems. (+)
4 I tend not to think about the future very much. 1 2 3 4 5
5 Conserving our resources for the future is one 1 2 3 4 5

of the most important issues facing the world
today. (+)
6 I try to save money for a rainy day rather than 1 2 3 4 5

spend it as soon as I get it.
7 I prefer to enjoy the present rather than to plan 1 2 3 4 5

ahead.
8 The trouble with many people is that they are 1 2 3 4 5

not prepared to wait for things. (+)
9 It’s all very well to plan for the future but the 1 2 3 4 5
first priority should be to deal with present
problems
10 I mainly take life as it comes rather than always 1 2 3 4 5

planning ahead
11 Zero population growth is an important goal 1 2 3 4 5

given the likely overpopulation in the future. (+)
12 There is so much to worry about in the present 1 2 3 4 5

that I do not give the future that much thought.
13 It is important to make plans for the future and 1 2 3 4 5

not just accept what comes. (+)
Reliability in Research
Reliability refers to the consistency of a measurement tool; it should

produce similar results under consistent conditions.
“A reliable measurement is one where we obtain the same result on

repeated occasions. If people answer a question the same way on
repeated occasions then it is reliable”. ………….deVaus, D.A. (2014)
Types:
•Internal Reliability: Consistency within the test itself.
•External Reliability: Consistency across different instances.
Internal Reliability Example
Example: Using a questionnaire to measure job

satisfaction; if respondents consistently agree on similar
questions, the questionnaire is internally reliable.
Sources of Un-reliability
• A question may be unreliable due to bad wording: a
person may understand the question differently on
different occasions.
• Different interviewers can elicit different answers
from the respondent: the gender, ethnic background
and dress of the interviewer can influence responses.
• Even well-developed questions will be subject to
unreliability problems. For example, studies of the
same respondents over time show that they give
different answers to questions on different occasions,
even though there should have been no change
Reference: deVaus, D.A. (2014) Surveys in Social Research (6th edn). Abingdon: Routledge. Useful for survey strategy
Assessing Internal Reliability
Methods:
•Cronbach’s Alpha: Statistic used to measure the consistency of
responses across a set of questions (scale items) designed together to
measure a particular concept (scale). It consists of an alpha coefficient
with a value between 0 and 1.
• Values above 0.7 indicate acceptable internal consistency.
•Split-half Method: Splitting test items into two halves to check
consistency. Ask the same people the same questions at intervals of two
to four weeks and calculate the correlation between the answers on
both occasions.
• If the correlation is high (a rule of thumb is 0.8 or above) then we
assume that the question is reliable.
• It is often difficult to give the same test to the same sample twice.
External Reliability Example
External reliability refers to whether your data

collection procedures and analysis techniques would
produce consistent findings if you repeated them on
another occasion, or they were replicated by another
researcher.
Example: Repeating the same job satisfaction survey

in different companies. If results are consistent,
external reliability is strong.
Validity in Research
Internal validity is the extent your findings can be attributed

to the intervention you are researching rather than to flaws in
your research design.
Example: In an experiment, internal validity would be
established where an intervention can be shown statistically
to lead to an outcome rather than this having been caused by
some other confounding variable acting at the same time.
In a survey strategy using questionnaires, criterion validity,
is whether the questions are actually measuring what they are
intended to measure, thereby allowing accurate statistical
predictions to be made.
Validity ensures the research measures what it intends to
measure, indicating the accuracy of a method
Types of Validity
Internal Validity: Control over the research design to
ensure results are due to the variable studied. extent your
findings can be attributed to the intervention you are
researching rather than to flaws in your research design.
Example: In an experiment, internal validity would be

established where an intervention can be shown statistically
to lead to an outcome rather than this having been caused by
some other confounding variable acting at the same time
• Criterion Validity: Compare how people answered our

new measure of a concept, with existing, well-accepted
measures of the concept. If their answers on both the
new and the established measure are highly correlated
this is taken to mean that the new measure is valid.
Types of Validity
External Validity: Generalizability of findings
beyond the immediate study.
Extent research findings from a particular study
are generalizable to other relevant contexts.
Example: A corporate manager may ask, ‘Can

the findings from the research study in one
organization in our corporation also be used to
inform policy and practice in other
organizations in the group

Week 12_Measurement_Scaling, Reliability, Validity

Uploaded by

Copyright:

Available Formats

Week 12_Measurement_Scaling, Reliability, Validity

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Week 12_Measurement_Scaling, Reliability, Validity

Uploaded by

Copyright:

Available Formats

Scaling, Reliability, and

Understanding scaling, reliability, and validity is

Scaling involves assigning numerical values to qualitative data,

Example: Customer satisfaction measured on a 1-5 scale, where 1

a. If you start trying to change things

b. If something grows up over a long

Scaling transforms qualitative attributes into

A popular scale for measuring attitudes, where

Example: A statement like 'I am satisfied with the

1 Eat, drink and be merry for tomorrow we die. 1 2 3 4 5

2 I am a fairly ambitious person. (+) 1 2 3 4 5

3 If people thought ahead more we would not be 1 2 3 4 5

4 I tend not to think about the future very much. 1 2 3 4 5

5 Conserving our resources for the future is one 1 2 3 4 5

6 I try to save money for a rainy day rather than 1 2 3 4 5

7 I prefer to enjoy the present rather than to plan 1 2 3 4 5

8 The trouble with many people is that they are 1 2 3 4 5

10 I mainly take life as it comes rather than always 1 2 3 4 5

11 Zero population growth is an important goal 1 2 3 4 5

12 There is so much to worry about in the present 1 2 3 4 5

13 It is important to make plans for the future and 1 2 3 4 5

Reliability refers to the consistency of a measurement tool; it should

“A reliable measurement is one where we obtain the same result on

Example: Using a questionnaire to measure job

External reliability refers to whether your data

Example: Repeating the same job satisfaction survey

Internal validity is the extent your findings can be attributed

Example: In an experiment, internal validity would be

• Criterion Validity: Compare how people answered our

Example: A corporate manager may ask, ‘Can

You might also like