D A N I E L M U I J S

DOING

quantitative
research
IN EDUCATION
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 1

■■■ Chapter 1

Introduction to quantitative
research

■ ■ ■ What is quantitative research?

Research methods in education (and the other social sciences) are often
divided into two main types: quantitative and qualitative methods. This
book will discuss one of these two main strands: quantitative methods. In
this chapter we will have a look at what is meant by the term quantitative
methods, and what distinguishes quantitative from qualitative methods.
When you think of quantitative methods, you will probably have spe-
cific things in mind. You will probably be thinking of statistics, numbers
– many of you may be feeling somewhat apprehensive because you think
quantitative methods are difficult. Apart from the last, all these thoughts
capture some of the essence of quantitative methods.
The following definition, taken from Aliaga and Gunderson (2002),
describes what we mean by quantitative research methods very well:

Quantitative research is ‘Explaining phenomena by collecting numeri-

cal data that are analysed using mathematically based methods (in
particular statistics).’

Let’s go through this definition step by step. The first element is explain-
ing phenomena. This is a key element of all research, be it quantitative or
qualitative. When we set out do some research, we are always looking to
explain something. In education this could be questions like ‘why do
teachers leave teaching?’, ‘what factors influence pupil achievement?’
and so on.
The specificity of quantitative research lies in the next part of the defini-
tion. In quantitative research we collect numerical data. This is closely
connected to the final part of the definition: analysis using mathematically

1
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 2

2 ■ Doing Quantitative Research in Education

based methods. In order to be able to use mathematically based methods our

data have to be in numerical form. This is not the case for qualitative
research. Qualitative data are not necessarily or usually numerical, and
therefore cannot be analysed using statistics.
Therefore, because quantitative research is essentially about collecting
numerical data to explain a particular phenomenon, particular questions
seem immediately suited to being answered using quantitative methods:

■ How many males get a first-class degree at university compared

to females?
■ What percentage of teachers and school leaders belong to ethnic
minority groups?
■ Has pupil achievement in English improved in our school district
over time?

These are all questions we can look at quantitatively, as the data we need
to collect are already available to us in numerical form. However, does
this not severely limit the usefulness of quantitative research? There are
many phenomena we might want to look at, but which don’t seem to
produce any quantitative data. In fact, relatively few phenomena in edu-
cation actually occur in the form of ‘naturally’ quantitative data.
Luckily, we are far less limited than might appear from the above.
Many data that do not naturally appear in quantitative form can be col-
lected in a quantitative way. We do this by designing research
instruments aimed specifically at converting phenomena that don’t nat-
urally exist in quantitative form into quantitative data, which we can
analyse statistically. Examples of this are attitudes and beliefs. We might
want to collect data on pupils’ attitudes to their school and their teach-
ers. These attitudes obviously do not naturally exist in quantitative form
(we don’t form our attitudes in the shape of numerical scales!). Yet we
can develop a questionnaire that asks pupils to rate a number of state-
ments (for example, ‘I think school is boring’) as either agree strongly,
agree, disagree or disagree strongly, and give the answers a number (e.g.
1 for disagree strongly, 4 for agree strongly). Now we have quantitative
data on pupil attitudes to school. In the same way, we can collect data
on a wide number of phenomena, and make them quantitative through
data collection instruments like questionnaires or tests. In the next three
chapters we will look at how we can develop instruments to do just that.
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 3

Introduction to quantitative research ■ 3

The number of phenomena we can study in this way is almost unlim-

ited, making quantitative research quite flexible. However, not all
phenomena are best studied using quantitative methods. As we will see,
while quantitative methods have some notable advantages, they also
have disadvantages, which means that some phenomena are better stud-
ied using different (qualitative) methods.
The last part of the definition refers to the use of mathematically based
methods, in particular statistics, to analyse the data. This is what people
usually think about when they think of quantitative research, and is
often seen as the most important part of quantitative studies. This is a bit
of a misconception. While it is important to use the right data analysis
tools, it is even more important to use the right research design and data
collection instruments. However, the use of statistics to analyse the data
is the element that puts a lot of people off doing quantitative research,
because the mathematics underlying the methods seem complicated and
frightening. Nevertheless, as we will see later on in this book, most
researchers do not really have to be particularly expert in the mathemat-
ics underlying the methods, because computer software allows us to do
the analyses quickly and (relatively) easily.

■ ■ ■ Foundations of quantitative research

methods
Realism, subjectivism and the ‘paradigm wars’

Now we have defined quantitative research, let’s compare it with qualita-

tive research, against which it is usually contrasted. While quantitative
research is based on numerical data analysed statistically, qualitative
research uses non-numerical data. Qualitative research is actually an
umbrella term encompassing a wide range of methods, such as inter-
views, case studies, ethnographic research and discourse analysis, to
name just a few examples.
The difference between quantitative and qualitative research is often
seen as quite fundamental, leading people to talk about ‘paradigm wars’
in which quantitative and qualitative research are seen as belligerent and
incompatible factions (a bit like capitalism and communism). Many
researchers define themselves as either quantitative or qualitative. Where
does this idea come from?
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 4

4 ■ Doing Quantitative Research in Education

This idea is linked to what are seen as the different underlying

philosophies and worldviews of researchers in the two ‘paradigms’ (also
called ‘epistemologies’). According to this view, two fundamentally dif-
ferent worldviews underlie quantitative and qualitative research. The
quantitative view is described as being ‘realist’ or sometimes ‘posi-
tivist’, while the worldview underlying qualitative research is viewed as
being ‘subjectivist’.
What does this mean? Realists take the view that what research does is
uncover an existing reality. ‘The truth is out there’ and it is the job of the
researcher to use objective research methods to uncover that truth. This
means that the researcher needs to be as detached from the research as
possible, and use methods that maximise objectivity and minimise the
involvement of the researcher in the research. This is best done using
methods taken largely from the natural sciences (e.g. biology, physics,
etc.), which are then transposed to social research settings (like educa-
tion). Positivism is the most extreme form of this worldview. According to
positivism, the world works according to fixed laws of cause and effect.
Scientific thinking is used to test theories about these laws, and either
reject or provisionally accept them. In this way, we will finally get to
understand the truth about how the world works. By developing reliable
measurement instruments, we can objectively study the physical world.
However, this view, that there is a true reality out there that we can meas-
ure completely objectively, is problematic. We are all part of the world we
are observing, and cannot completely detach ourselves from what we are
researching. Historical research has shown that what is studied and what
findings are produced are influenced by the beliefs of the people doing the
research and the political/social climate at the time the research is done.
If one looks at research from a quantitative versus qualitative perspec-
tive, qualitative researchers are subjectivists. In contrast to the realist view
that the truth is out there and can be objectively measured and found
through research, subjectivists point to the role of human subjectivity in
the process of research. Reality is not ‘out there’ to be objectively and dis-
passionately observed by us, but is at least in part constructed by us and
by our observations. There is no pre-existing objective reality that can be
observed. The process of our observing reality changes and transforms it,
and therefore subjectivists are relativistic. All truth can only be relative
and is never definitive as the positivists claim. The extreme relativist posi-
tion is obviously as problematic as the extreme positivistic one, because,
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 5

Introduction to quantitative research ■ 5

for example, it would in theory deny that anything more than social con-
sensus and power distinguishes witchcraft and modern science.
If you look at the extreme forms of the two views we have set out here,
it would seem that quantitative and qualitative research methods are
pretty incompatible. These extremes are, however, a gross simplification
of the views of both quantitative and qualitative researchers, and very
few people in either ‘camp’ subscribe to them. I have included them here
because they are frequently presented in only slightly less extreme forms
as straw men with which critics of one method (qualitative for example)
may attack users of different methods (for example quantitative).
Qualitative methods is an umbrella term for a large number of different
research methods (such as participant observation, interviews, case stud-
ies, ethnographic research) which are quite different. They are used by
researchers with quite different worldviews, some of which clearly lie
towards the realistic end of the spectrum. To ascribe radical subjectivist
views to all qualitative researchers is a fallacy.
To label all quantitative researchers positivists is equally inaccurate.
Quantitative researchers have taken up many criticisms of positivist
views, and there are now a variety of epistemologies underlying theory
and practice in quantitative research. I think it is true to say that very
few quantitative researchers nowadays are radical positivists.

Post-positivism, experiential realism and pragmatism

Post-positivists accept the critique of traditional positivism that has been

presented by the subjectivists, without going so far as to reject any
notion of realism. Post-positivists accept that we cannot observe the
world we are part of as totally objective and disinterested outsiders, and
accept that the natural sciences do not provide the model for all social
research. However, they do believe in the possibility of an objective real-
ity. While we will never be able to totally uncover that reality through
our research, post-positivists believe that we should try and approximate
that reality as best we can, all the while realising that our own subjectiv-
ity is shaping that reality. Rather than finding the truth, the
post-positivist will try and represent reality as best he or she can.
In contrast to positivists, post-positivists believe that research can
never be certain. Rather than focusing on certainty and absolute truth,
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 6

6 ■ Doing Quantitative Research in Education

post-positivist social science focuses on confidence – how much can we

rely on our findings? how well do they predict certain outcomes?
A second worldview or epistemology that underlies the work of some
quantitative researchers is called experiential realism. Experiential realism
claims, as do anti-positivist positions, that we cannot observe the world
in a purely objective way, because our perception itself influences what
we see and measure. In contrast to subjectivist positions, however, expe-
riential realists believe that there is a limit to subjectivity. Humans are
limited in their subjectivity by the fact that we use a limited number of
schemas to formulate our views of the world. This is because our percep-
tion is ‘embodied’. We don’t observe passively, but actively interact with
the world through our bodies.
Experiential realists see the use of metaphor as crucial to the way we
make sense of the world around us. We use metaphors to understand our
world. One of the main metaphors we use to do this is the subject/object
schema, which divides the world up into objects (things) and subjects
(people). This metaphor has its origins in the fact that in our dealings
with the world we find that there is a distinction between an external
world consisting of edges, surfaces and textures that are not us, and
those things that are us, the actor. As we move around our world, the
objects remain invariant. Science, according to this view, is an activity
that is based on this subject/object schema (Mulaik, 1995).
A lot of researchers, both quantitative and qualitative (the author
included), take a pragmatist approach to research, using different methods
depending on the research question they are trying to answer. In some cases
this will lead them to quantitative research, for example when they need to
give a quantitative answer to a question or generalise findings to a popula-
tion, or are looking to test a theory mathematically; in other cases they will
employ qualitative methods. Sometimes a mixed methods approach com-
bining quantitative and qualitative methods will be the most appropriate.
Philosophers like Peirce, Dewey and James developed pragmatism as a
philosophy in the USA. One of the main contentions of this school of
philosophy is that the meaning and the truth of any idea is a function of
its practical outcome(s). Pragmatists strongly oppose the absolutism they
see as a key part of most other philosophical beliefs, and put themselves
in opposition to other philosophies (think of the positivist/subjectivist
debate) which are totally rejected.
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 7

Introduction to quantitative research ■ 7

As for the subjectivists, there is no definite truth in pragmatic philoso-

phy. Truth is constantly changing and being updated through the
process of human problem-solving. The key question for pragmatists is
not ‘is it true?’ or ‘is it right?’ but ‘does it work?’

■ ■ ■ When do we use quantitative methods?

If we take a pragmatic approach to research methods, first of all we need
to find out what kinds of questions are best answered using quantitative
as opposed to qualitative methods.
There are four main types of research question that quantitative
research is particularly suited to find an answer to:

1. The first is when we want a quantitative answer. Examples are: ‘How

many students choose to study education?’ or ‘How many mathe-
matics teachers do we need and how many have we got in our school
district?’ That we need to use quantitative research to answer this
kind of question is obvious. Qualitative, non-numerical methods will
obviously not provide us with the (numerical) answer we want.
2. Numerical change can likewise only accurately be studied using
quantitative methods. Are the numbers of students in our univer-
sity rising or falling? Is achievement going up or down? We would
need to do a quantitative study to find out.
3. As well as wanting to find out about the state of something, we
often want to explain phenomena. What factors predict the recruit-
ment of mathematics teachers? What factors are related to changes
in student achievement over time? As we will see later in this book,
this kind of question can also be studied successfully using quanti-
tative methods, and many statistical techniques have been
developed that allow us to predict scores on one factor or variable
(e.g. teacher recruitment) from scores on one or more other factors
or variables (e.g. unemployment rates, pay, conditions).
4. The final activity for which quantitative research is especially
suited is the testing of hypotheses. We might want to explain some-
thing, for example whether there is a relationship between a
pupil’s achievement and their self-esteem and social background.
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 8

8 ■ Doing Quantitative Research in Education

We could look at the theory and come up with the hypothesis that
lower social class background leads to low self-esteem, which
would in turn be related to low achievement. Using quantitative
research we can try and test this kind of model.

■ ■ ■ Units and variables

When we collect data in quantitative educational research, we have to

collect them from someone or something. The people or things (e.g.
schools) we collect data on or from are known as units or cases.

The data that we are collecting from these units are known as variables.
Variables are any characteristic of the unit we are interested in and want
to collect (e.g. gender, age, self-esteem).

The name variable refers to the fact that this data will differ between
units. For example, achievement will differ between pupils and schools,
gender will differ between pupils, and so on. If there are no differences
at all between units we want to study we probably aren’t going to be
able to do any interesting research (for example, studying whether
pupils are human would not yield interesting findings).

■ ■ ■ What is a hypothesis?

A hypothesis is a tentative explanation that accounts for a set of facts

and can be tested by further investigation.

For example, one hypothesis we might want to test could be that poverty
causes low achievement, or that there is a relationship between pupils’
self-esteem and the amount of time they spend watching television.

Quantitative researchers will design studies that allow us to test these

hypotheses. We will collect the relevant data (for example, parental
income and school achievement) and use statistical techniques to decide
whether or not to reject or provisionally accept the hypothesis.

Accepting a hypothesis is always provisional, as new data may emerge

that causes it to be rejected later on.
9079 Chapter 01 (1-12) 24/2/04 12:10 pm Page 9

Introduction to quantitative research ■ 9

The types of problem outlined in 1 and 2 opposite are called ‘descriptive’ –

we are merely trying to describe a situation – while those in 3 and 4 are
‘inferential’ – we are trying to explain something rather than just describe it.
As mentioned above, while quantitative methods are good at answer-
ing these four types of questions, there are other types of question that
are not well suited to quantitative methods:

1. The first situation where quantitative research will fail is when we

want to explore a problem in depth. Quantitative research is good
at providing information in breadth from a large number of units.
But when we want to explore a problem or concept in depth quan-
titative methods are too shallow. To really get under the skin of a
phenomenon, we will need to go for ethnographic methods, inter-
views, in-depth case studies and other qualitative techniques.
2. We saw above that quantitative research is well-suited for the test-
ing of theories and hypotheses. What quantitative methods cannot
do very well is develop hypotheses and theories. The hypotheses to
be tested may come from a review of the literature or theory, but
can also be developed using exploratory qualitative research.
3. If issues to be studied are particularly complex, an in-depth quali-
tative study (a case study, for example) is more likely to pick up on
this than a quantitative study. This is partly because there is a limit
to how many variables can be looked at in any one quantitative
study, and partly because in quantitative research it is the
researcher who defines the variables to be studied. In qualitative
research unexpected variables may emerge.
4. Finally, while quantitative methods are better at looking at cause and
effect (causality, as it is known), qualitative methods are more suited
to looking at the meaning of particular events or circumstances.

What then do we do if we want to look at both breadth and depth, or at

both causality and meaning? In these situations, it is best to use a so-
called mixed methods design in which we use both quantitative (for
example, a questionnaire) and qualitative (for example, a number of case
studies) methods. Mixed methods research is a flexible approach where
the research design is determined by what we want to find out rather
than by any predetermined epistemological position. In mixed methods
research, qualitative or quantitative components can predominate or
both can have equal status.