MIxed Method Design PDF
MIxed Method Design PDF
MIxed Method Design PDF
CHAPTER 4
CHOOSING A MIXED
METHODS DESIGN
58
04-Creswell (Designing)-45025.qxd 5/16/2006 8:35 PM Page 59
(Continued)
04-Creswell (Designing)-45025.qxd 5/16/2006 8:35 PM Page 62
SOURCE: Adapted from Creswell, Plano Clark, et al. (2003, pp. 216-217, Table 8.1).
QUAN QUAL
Interpretation based on
QUAN + QUAL results
QUAN QUAN
QUAN
data data
results
collection analysis
Compare
Interpretation
and
QUAN + QUAL
contrast
QUAL QUAL
QUAL
data data
results
collection analysis
(c) Triangulation Design: Data Transformation Model (Transforming QUAL data into QUAN)
QUAN
data QUAN data analysis
collection
Compare and
Interpretation
interrelate two
QUAN + QUAL
QUAN data sets
Validate QUAN
Interpretation
results with
QUAN + qual
qual results
qual
qual
data collection: qual
data
Open-ended results
analysis
survey items
Level 1:
QUAN
data collection, analysis, results
Level 2:
QUAL Overall
data collection, analysis, results interpretation
Level 3:
QUAN
data collection, analysis, results
methods during the same timeframe and with equal weight (see Figure 4.1a).
The single-phase timing of this design is the reason it has also been referred
to as the concurrent triangulation design (Creswell, Plano Clark, et al.,
2003). It generally involves the concurrent, but separate, collection and
analysis of quantitative and qualitative data so that the researcher may best
understand the research problem. The researcher attempts to merge the two
data sets, typically by bringing the separate results together in the interpre-
tation or by transforming data to facilitate integrating the two data types dur-
ing the analysis. Jenkins (2001) single-phase study (appendix A) of rural
adolescent perceptions of alcohol and other drug resistance is an example of
a Triangulation Design. She collected and analyzed quantitative and qualita-
tive data and merged the two data sets into one overall interpretation, in
which she related the quantitative results to the qualitative findings.
Variants of the Triangulation Design. The four variants are the convergence
model, the data transformation model, the validating quantitative data
model, and the multilevel model. The first two models differ in terms of how
the researcher attempts to merge the two data types (either during interpre-
tation or during analysis), the third model is used to enhance findings from
a survey, and the fourth is used to investigate different levels of analysis.
The convergence model (Figure 4.1b) represents the traditional model
of a mixed methods triangulation design (Creswell, 1999). In this model, the
researcher collects and analyzes quantitative and qualitative data separately
on the same phenomenon and then the different results are converged (by
comparing and contrasting the different results) during the interpretation.
Researchers use this model when they want to compare results or to validate,
04-Creswell (Designing)-45025.qxd 5/16/2006 8:35 PM Page 65
The Embedded Design is a mixed methods design in which one data set
provides a supportive, secondary role in a study based primarily on the other
data type (see Figure 4.2a) (Creswell, Plano Clark, et al., 2003). The premises
of this design are that a single data set is not sufficient, that different ques-
tions need to be answered, and that each type of question requires different
types of data. Researchers use this design when they need to include quali-
tative or quantitative data to answer a research question within a largely
quantitative or qualitative study. This design is particularly useful when a
researcher needs to embed a qualitative component within a quantitative
design, as in the case of an experimental or correlational design. In the exper-
imental example, the investigator includes qualitative data for several rea-
sons, such as to develop a treatment, to examine the process of an intervention
or the mechanisms that relate variables, or to follow up on the results of an
experiment.
qual qual
QUAN Intervention QUAN
before after
premeasure postmeasure
intervention intervention
Interpretation
based on
QUAN(qual)
results
qual
during
intervention
QUAN
Predictors
QUAN QUAN
Predictors Outcome Interpretation
based on
QUAN QUAN(qual)
Predictors results
qual
process
data are used to answer different research questions within the study
(Hanson et al., 2005). For example, Rogers et al. (2003) (appendix B) embed-
ded qualitative data within their experimental design in two different ways:
before the intervention, to inform the development of the treatment, and
after the intervention, to explain the treatment results.
It can be a challenge to differentiate between a study using an Embedded
Design and a study using one of the other mixed methods designs. The key
question is whether the secondary data type is playing a supplemental role
within a design based on the other data type. Consider the question: Would
the results of the secondary data type be useful or meaningful if they were
not embedded within the other data set? For example, Rogers et al.s (2003)
qualitative explanation of the treatment results would not make much sense
or have value if there had not been outcomes measured from an experimen-
tal study using those treatments.
2003). For example, this design is well suited to a study in which a researcher
needs qualitative data to explain significant (or nonsignificant) results, outlier
results, or surprising results (Morse, 1991). This design can also be used
when a researcher wants to form groups based on quantitative results and
follow up with the groups through subsequent qualitative research (Morgan,
1998; Tashakkori & Teddlie, 1998) or to use quantitative participant charac-
teristics to guide purposeful sampling for a qualitative phase (Creswell, Plano
Clark, et al., 2003).
Variants of the Explanatory Design. There are two variants of the Explanatory
Design: the follow-up explanations model and the participant selection model.
Although both models have an initial quantitative phase followed by a quali-
tative phase, they differ in the connection of the two phases, with one focus-
ing on results to be examined in more detail and the other on the appropriate
participants to be selected (see center boxes of Figures 4.3b and 4.3c). They
also differ in the relative emphasis often placed on the two phases.
The follow-up explanations model (Figure 4.3b) is used when a
researcher needs qualitative data to explain or expand on quantitative results
(Creswell, Plano Clark, et al., 2003). In this model, the researcher identifies
specific quantitative findings that need additional explanation, such as statis-
tical differences among groups, individuals who scored at extreme levels, or
unexpected results. The researcher then collects qualitative data from partic-
ipants who can best help explain these findings. In this model, the primary
emphasis is usually on the quantitative aspects. Ivankovas (2004) disserta-
tion study of doctoral students persistence in an online learning environ-
ment is an example of this variant. In the initial quantitative phase, she
collected quantitative survey data to identify factors predictive of students
(a) Explanatory Design
04-Creswell (Designing)-45025.qxd
Interpretation based on
QUAN qual QUAN qual
results
5/16/2006
73
04-Creswell (Designing)-45025.qxd 5/16/2006 8:35 PM Page 74
persistence. In the second phase, she used a qualitative multiple case study
approach to help explain why certain factors identified in the first phase were
significant predictors of student persistence in the program.
The participant selection model (Figure 4.3c) is used when a researcher
needs quantitative information to identify and purposefully select participants
for a follow-up, in-depth, qualitative study. In this model, the emphasis of the
study is usually on the second, qualitative phase. For example, May and Etkina
(2002) collected quantitative data to identify physics students with consis-
tently high and low conceptual learning gains. They then completed an in-
depth qualitative comparison study of these students perceptions of learning.
Interpretation based on
04-Creswell (Designing)-45025.qxd
Develop
QUAL QUAL quan quan
QUAL taxonomy quan Interpretation
data data data data
results or theory results QUAL qual
collection analysis collection analysis
for testing
Variants of the Exploratory Design. This design has two common variants:
the instrument development model and the taxonomy development model.
Each of these models begins with an initial qualitative phase and ends with a
quantitative phase. They differ in the way the researcher connects the two
phases (see center boxes of Figures 4.4b and 4.4c) and in the relative empha-
sis of the two methods.
Researchers use the instrument development model (see Figure 4.4b)
when they need to develop and implement a quantitative instrument based on
qualitative findings. In this design, the researcher first qualitatively explores the
research topic with a few participants. The qualitative findings then guide
the development of items and scales for a quantitative survey instrument.
In the second data collection phase, the researcher implements and validates
this instrument quantitatively. In this design, the qualitative and quantitative
methods are connected through the development of the instrument items.
Researchers using this variant often emphasize the quantitative aspect of the
study. Using this model, Mak and Marshall (2004) initially qualitatively explored
young adults perceptions about the significance of the self to others in roman-
tic relationships (that is, how they perceive that they matter to someone else).
Based on their qualitative results, they developed an instrument and then
implemented it during a second quantitative phase in their study.
The taxonomy development model (see Figure 4.4c) occurs when the ini-
tial qualitative phase is conducted to identify important variables, develop a
taxonomy or classification system, or develop an emergent theory, and the sec-
ondary, quantitative phase tests or studies these results in more detail (Morgan,
1998; Tashakkori & Teddlie, 1998). In this model, the initial qualitative phase
04-Creswell (Designing)-45025.qxd 5/16/2006 8:35 PM Page 78
Strengths of the Exploratory Design. Due to its two-phase structure and the
fact that only one type of data is collected at a time, the Exploratory Design
shares many of the same advantages as the Explanatory Design. Its advan-
tages include the following:
The researcher needs to decide which data to use from the qualitative
phase to build the quantitative instrument and how to use these data
to generate quantitative measures. In Chapter 7, we will discuss pro-
cedures for using qualitative quotes, codes, and themes to generate
aspects of quantitative instruments.
Procedures should be undertaken to ensure that the scores developed
on the instrument are valid and reliable. In Chapter 6, we will review
rigorous steps of instrument and scale development for this process.
(a) What will the timing of the quantitative and qualitative methods be?
Concurrent Sequential
timing timing
Quantitative Qualitative
first first
(b) What will the weighting of the quantitative and qualitative methods be?
Equal Unequal
weight weight
Quantitative Qualitative
emphasis emphasis
Figure 4.5 Decision Tree for Mixed Methods Design Criteria for Timing, Weighting, and Mixing
SOURCE: Based on Creswell, Plano Clark, et al. (2003); Hanson et al. (2005); and Plano Clark (2005).
assistants. The expectations of audiences for the research can also influence
the design choice, particularly if the audience values one type of evidence
over the other type.
In addition to these factors, the choice of a research design relates to
three decisions: the timing of the use of collected data (i.e., the order in
which the data are used in a study), the relative weight of the quantitative and
qualitative approaches (i.e., the emphasis given to each), and the approach
04-Creswell (Designing)-45025.qxd 5/16/2006 8:35 PM Page 81
to mixing the two datasets (i.e., how the two datasets will be related or con-
nected). A decision tree, shown in Figure 4.5, can help identify choices for
each of these three decisions.
More space being devoted to one method in the article or the empha-
sis of one method within the abstract
More sophisticated and complex procedures used for one method
compared to the other
Merging Data Sets. The data are merged when the researcher takes the two
data sets and explicitly brings them together or integrates them. Researchers
can merge the two data sets during the interpretation (by analyzing them
separately in a results section and then merging the two sets of results
together during the interpretation or discussion phase) or during the analy-
sis of the data (by transforming one data type into the other type or consoli-
dating the data into new variables).
Embedding Data at the Design Level. The researcher could decide to embed
data of one type within a design of the other type. This is an example of mix-
ing at the design level, not just at the level of data. A researcher may choose
to embed qualitative data within a larger quantitative (e.g., experimental)
design or to embed quantitative data within a larger qualitative (e.g., phe-
nomenology) design. One form of data can be embedded in a concurrent
data collection with the other dataset; alternatively, the embedded data may
be collected sequentially before or after the other dataset. Researchers may
make interpretations from using the secondary, embedded dataset by bring-
ing the two datasets together in the concurrent approach and keeping them
separate in the sequential approach.
analysis of one type of data leads to (and thereby connects to) the need for
the other type of data. This can occur in one of two ways. A researcher may
obtain quantitative results that lead to the subsequent collection and analysis
of qualitative data. A researcher can also start with qualitative results that
build to the subsequent collection and analysis of quantitative data. The mix-
ing occurs in the way that the two data types are connected. This connection
can occur in different ways, such as in specifying research questions, select-
ing participants, or developing an instrument or other materials.
If there is a single phase, both types of data are given equal emphasis, the
two sets of results are converged during the interpretation, and the intent
is to draw valid conclusions about a research problem, then the choice of
design is the Triangulation Designconvergence model.
If there is a single phase, both types of data are given equal emphasis,
one type of data is transformed into the other type, and the intent is
to interrelate different data types about a research problem, then the
choice of design is the Triangulation Designdata transformation model.
If both types of data are collected at the same time from a survey and
the intent is to use qualitative information to validate the quantitative
results, then the choice of design is the Triangulation Designvalidat-
ing quantitative data model.
If different types of data are collected to represent different levels of
analysis within a system, with the intent of forming an overall inter-
pretation of the system, then the choice of design is the Triangulation
Designmultilevel model.
If quantitative data are used to answer the primary question in an exper-
imental design and qualitative data are embedded within the experi-
mental design (before, during, or after the intervention) with the intent
of answering a secondary question related to the experiment, then the
choice of design is the Embedded Designexperimental model.
04-Creswell (Designing)-45025.qxd 5/16/2006 8:35 PM Page 85
the results of the first phase, and the intent is to purposefully select
participants to best address the qualitative research question, then
the choice of design is the Explanatory Designparticipant selection
model.
If one phase is followed by another phase, the first phase is qualita-
tive, the two phases are connected by the development of an instru-
ment based on the results of the first phase, and the intent is to
develop and implement an instrument on the topic of interest, then
the choice of design is the Exploratory Designinstrument develop-
ment model.
If one phase is followed by another phase, the first phase is qualitative
and results in a taxonomy or emergent theory, the two phases are con-
nected by quantitative testing of the results of the first qualitative
phase, the qualitative phase is emphasized, and the intent is to quan-
titatively generalize the qualitative results, then the choice of design is
the Exploratory Designtaxonomy development model.
WRITING A PARAGRAPH TO
IDENTIFY A STUDYS DESIGN
Because many researchers and reviewers are currently unfamiliar with the
different types of mixed methods designs, it is important to include an
overview paragraph that introduces the design when writing about a study in
proposals or research reports. This overview paragraph generally is placed at
the start of the methods discussion and should address four topics. First,
identify the type of mixed methods design and variant model, if appropriate.
Next, give the defining characteristics of this design, including its timing,
weighting, and mixing decisions. Third, state the overall purpose or rationale
for using this design for the study. Finally, include references to the mixed
methods literature on this design. An example of an overview paragraph is
included in Figure 4.6, along with comments that will assist in identifying
these features within the paragraph.
Summary
Names design
Mixed Methods Sequential Explanatory Design
Activities
1. Which of the four major design types will you use in your study? Write
a one-paragraph overview that identifies this design; defines its
timing, weighting, and mixing; and states your rationale for choosing
it for your study.
2. What challenges are associated with your design choice? Write a para-
graph that discusses the challenges that you anticipate occurring with
your design and how you might address them.
04-Creswell (Designing)-45025.qxd 5/16/2006 8:35 PM Page 88
3. Draw a diagram of the procedures you will use, following one of the
major variants of the four types of designs advanced in this chapter.
Use the depictions of the different variants in the figures as examples
to follow.
For additional information on the major mixed methods design types, consult
Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed
methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of
mixed methods in social and behavioral research (pp. 209240). Thousand
Oaks, CA: Sage.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework
for mixed-method evaluation designs. Educational Evaluation and Policy
Analysis, 11(3), 255274.
CHAPTER 5
INTRODUCING A MIXED
METHODS STUDY
89