Han 251 Book

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health
Sciences
RESEARCH METHODS IN
THE HEALTH SCIENCES

Sciences
RESEARCH METHODS IN
THE HEALTH SCIENCES
Deborah Zelizer, Kathleen McGoldrick, and Deborah Firestone
First Edition

Sciences
Bassim Hamadeh, CEO and Publisher
John Remington, Senior Field Acquisitions Editor
Michelle Piehl, Senior Project Editor
Alia Bales, Production Editor
Emely Villavicencio, Senior Graphic Designer
Stephanie Kohl, Licensing Coordinator
Natalie Piccotti, Director of Marketing
Kassie Graves, Vice President of Editorial
Jamie Giganti, Director of Academic Publishing
Copyright © 2018 by Deborah Zelizer, Kathleen McGoldrick and Deborah Firestone. All rights
reserved. No part of this publication may be reprinted, reproduced, transmitted, or utilized in any
form or by any electronic, mechanical, or other means, now known or hereafter invented, including
photocopying, microfilming, and recording, or in any information retrieval system without the writ-
ten permission of Cognella, Inc. For inquiries regarding permissions, translations, foreign rights,
audio rights, and any other forms of reproduction, please contact the Cognella Licensing Depart-
ment at [email protected].
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
Cover image copyright © 2017 iStockphoto LP/Filograph.
Printed in the United States of America.
ISBN: 978-1-5165-1866-1 (pbk) / 978-1-5165-1867-8 (br)

Sciences
For our students.
Your commitment to making the world a better place as the next generation of clinical and
non-clinical healthcare providers is a constant source of inspiration.

Sciences
BRIEF CONTENTS
ACKNOWLEDGEMENTS............................................................................................................................................. x iii
INTRODUCTION�� xv
CHAPTER 1 Generating New Knowledge: Conceptualizing

and Planning Research. . ............................................................................................................................... 1
CHAPTER 2 Ethical Issues When Conducting Research.......................................................................... 25
CHAPTER 3 Understanding the Role of Literature in the Research Process:

Reviewing, Writing, and Critiquing the Scholarly Article. . ...................................... 43
CHAPTER 4 Developing the Research Problem Statement, Purpose Statement,

and Research Questions.. ....................................................................................................................... 75
CHAPTER 5 Selecting Methods to Increase Rigor: Planning a Study

with Validity and Reliability or Trustworthiness............................................................. 91
CHAPTER 6 Qualitative Research: Qualitative Designs......................................................................... 125
CHAPTER 7 Understanding Quantitative Research: Experimental/

Quasi-Experimental Designs. . ........................................................................................................ 139
CHAPTER 8 Understanding Quantitative Research:

Non-Experimental Designs.. ............................................................................................................. 161
CHAPTER 9 Understanding Quantitative Research:

Survey Research. . ....................................................................................................................................... 187
CHAPTER 10 Understanding Mixed Methods Research:

Mixed Methods Designs.. ................................................................................................................ 209
APPENDIX A ................................................................................................................................................................................ 223
INDEX ................................................................................................................................................................................................ 229
vii

Sciences
DETAILED CONTENTS
ACKNOWLEDGEMENTS............................................................................................................................................. x iii
INTRODUCTION. . ...................................................................................................................................................................... x v
VARK................................................................................................................................................................................................ x v i
References................................................................................................................................................................................... x v iii
CHAPTER 1 Generating New Knowledge: Conceptualizing

and Planning Research. . ............................................................................................................................... 1
Chapter Goals:.. ................................................................................................................................................................................... 1
Learning Objectives:. . ....................................................................................................................................................................... 1
Introduction......................................................................................................................................................................................... 1
Health Science—Defined.. ......................................................................................................................................................... 1
Scientific Method and Evidence-Based Practice—Defined............................................................................... 2
Research—Defined........................................................................................................................................................................ 3
Different Types of Research................................................................................................................................................... 8
Paradigms and Methodology................................................................................................................................................. 9
Deductive Versus Inductive Reasoning. . ....................................................................................................................... 13
Methods and Methodology.................................................................................................................................................. 14
Summary of Main Ideas and Concepts............................................................................................................................... 17
Study Practice.................................................................................................................................................................................. 17
Practice Multiple-Choice Questions..................................................................................................................................... 18
Study Activities................................................................................................................................................................................ 21
References.......................................................................................................................................................................................... 22
CHAPTER 2 Ethical Issues When Conducting Research.......................................................................... 25

Chapter Goals:.. ............................................................................................................................................................................... 25
Learning Objectives:. . ................................................................................................................................................................... 25
Introduction..................................................................................................................................................................................... 25
Ethical Research—Defined.................................................................................................................................................. 25
Early Research Practices and the Creation of Research Standards.......................................................... 25
Common Rule.. .............................................................................................................................................................................. 30
Institutional Review Boards................................................................................................................................................. 31
Composition of the Institutional Review Board..................................................................................................... 33
Research with Human Beings Requires Informed Consent........................................................................... 33
Summary of Main Ideas and Concepts.............................................................................................................................. 36
ix

Sciences
x | Detailed Contents
Study Practice................................................................................................................................................................................. 36
Practice Multiple-Choice Questions.................................................................................................................................... 37
Study Activities............................................................................................................................................................................... 39
Case Study 1. . .................................................................................................................................................................................... 39
Case Study 2..................................................................................................................................................................................... 39
References......................................................................................................................................................................................... 40
CHAPTER 3 Understanding the Role of Literature in the Research Process:

Reviewing, Writing, and Critiquing the Scholarly Article. . ...................................... 43
Chapter Goals:.. ............................................................................................................................................................................... 43
Introduction..................................................................................................................................................................................... 43
Systematic Review of the Literature—Defined. . ................................................................................................... 4 4
Conducting the Systematic Review of the Literature........................................................................................ 45
Literature Review—Defined.. ............................................................................................................................................... 53
Writing the Literature Review........................................................................................................................................... 54
Critiquing a Journal Article—Defined......................................................................................................................... 56
Quantitative Research Articles. . ........................................................................................................................................ 59
Qualitative Research Articles............................................................................................................................................. 65
Mixed Methods Research Articles. . ................................................................................................................................. 68
Summary of Main Ideas and Concepts............................................................................................................................. 69
Study Practice................................................................................................................................................................................ 69
References.......................................................................................................................................................................................... 73
CHAPTER 4 Developing the Research Problem Statement, Purpose Statement,

and Research Questions.. ....................................................................................................................... 75
Chapter Goals:.. ............................................................................................................................................................................... 75
Introduction..................................................................................................................................................................................... 75
The Research Problem Statement, Purpose Statement, and Research
Questions—Defined.. ................................................................................................................................................................ 75
Problem Statement.................................................................................................................................................................... 75
Purpose Statement.. ................................................................................................................................................................... 76
The Research Question........................................................................................................................................................... 78
Summary of Main Ideas and Concepts.............................................................................................................................. 84
Study Practice................................................................................................................................................................................. 85
References.......................................................................................................................................................................................... 89

Sciences
Detailed Contents | xi
CHAPTER 5 Selecting Methods to Increase Rigor: Planning a Study with Validity

and Reliability or Trustworthiness.. .............................................................................................. 91
Chapter Goals:.. ................................................................................................................................................................................ 91
Learning Objectives:. . .................................................................................................................................................................... 91
Introduction...................................................................................................................................................................................... 91
Validity and Reliability—Defined..................................................................................................................................... 91
Trustworthiness—Defined.................................................................................................................................................. 94
Methods: Sampling................................................................................................................................................................... 96
Methods: Data Collection.. ................................................................................................................................................. 105
Methods: Data Analysis....................................................................................................................................................... 108
Summary of Main Ideas and Concepts............................................................................................................................ 115
Study Practice............................................................................................................................................................................... 118
Practice Multiple-Choice Questions.................................................................................................................................. 119
Study Activities............................................................................................................................................................................. 121
References........................................................................................................................................................................................ 123
CHAPTER 6 Qualitative Research: Qualitative Designs......................................................................... 125

Chapter Goals:.. ............................................................................................................................................................................ 125
Learning Objectives:. . ................................................................................................................................................................ 125
Introduction.................................................................................................................................................................................. 125
Qualitative Research Designs—Defined.................................................................................................................. 126
Study Practice.............................................................................................................................................................................. 13 4
Practice Multiple-Choice Questions................................................................................................................................. 13 4
References........................................................................................................................................................................................ 137
CHAPTER 7 Understanding Quantitative Research: Experimental/

Quasi-Experimental Designs. . ........................................................................................................ 139
Chapter Goals:.. ............................................................................................................................................................................ 139
Introduction.................................................................................................................................................................................. 139
Experimental and Quasi-Experimental Designs—Defined. . ...................................................................... 139
Experimental Research. . ..................................................................................................................................................... 140
Experimental Research Designs.................................................................................................................................... 143
Quasi-Experimental Research Designs.. ................................................................................................................... 148
Methods.. ......................................................................................................................................................................................... 152
Study Practice............................................................................................................................................................................... 155
Practice Multiple-Choice Questions.................................................................................................................................. 155
References....................................................................................................................................................................................... 159

Sciences
xii | Detailed Contents
CHAPTER 8 Understanding Quantitative Research: Non-Experimental Designs........... 161

Chapter Goals:.. ............................................................................................................................................................................. 161
Learning Objectives:. . ................................................................................................................................................................. 161
Introduction................................................................................................................................................................................... 161
Non-Experimental Research—Defined..................................................................................................................... 161
Non-Experimental Research Designs........................................................................................................................ 162
Methods: Non-Experimental Research. . ................................................................................................................... 176
Threats to Validity in Non-Experimental Research......................................................................................... 179
Summary of Main Ideas and Concepts........................................................................................................................... 180
Study Practice.............................................................................................................................................................................. 182
Practice Multiple-Choice Questions................................................................................................................................. 182
Study Activities............................................................................................................................................................................ 185
References....................................................................................................................................................................................... 186
CHAPTER 9 Understanding Quantitative Research: Survey Research..................................... 187

Chapter Goals:.. ............................................................................................................................................................................ 187
Introduction.................................................................................................................................................................................. 187
Survey Research—Defined. . .............................................................................................................................................. 187
Writing Questions for a Survey Instrument. . ........................................................................................................ 190
Methods: Survey Research.. .............................................................................................................................................. 200
Summary of Main Ideas and Concepts.......................................................................................................................... 203
Study Practice............................................................................................................................................................................. 204
Practice Multiple-Choice Questions................................................................................................................................ 204
Study Activities........................................................................................................................................................................... 206
References...................................................................................................................................................................................... 207
CHAPTER 10 Understanding Mixed Methods Research: Mixed Methods Designs...... 209

Chapter Goals:.. ........................................................................................................................................................................... 209
Learning Objectives:. . ............................................................................................................................................................... 209
Introduction................................................................................................................................................................................. 209
Mixed Methods Research—Defined........................................................................................................................... 210
Mixed Methods Research—Designs............................................................................................................................ 211
Mixed Methods Research—Purpose Statement and Research Questions....................................... 214
Mixed Methods Research—Selecting the Qualitative and Quantitative
Designs and Methods. . .......................................................................................................................................................... 214
Mixed Methods Research: Strengths and Challenges.. ................................................................................... 216
Summary of Main Ideas and Concepts........................................................................................................................... 217
Study Practice.............................................................................................................................................................................. 217
Practice Multiple-Choice Questions Combined with Study Activities........................................................... 218
References...................................................................................................................................................................................... 220
APPENDIX A ................................................................................................................................................................................ 223
INDEX ................................................................................................................................................................................................ 229

Sciences
ACKNOWLEDGEMENTS
We would like to extend our gratitude to our colleague, Sharon Cuff, her support and feed-
back of the initial versions of our chapters was invaluable.
We would like to extend our gratitude to the following Health Science students for
providing the authors with substantive and detailed feedback throughout the editing and
revising process while working on the preliminary edition of the text book.
Angela Halloran
Victoria Logan
Theodora Panagos
Francesca Prainito
Michael Rizzo
Jackey Wu
With an extra special thank you to Francesca Prainito for spearheading the students’
review of the book.
xiii

Sciences
INTRODUCTION
The authors of this text are faculty who teach a required 200-level research methods course
in an undergraduate major, the Bachelor of Science in Health Science, in the School of Health
Technology and Management, at Stony Brook University. We have taught this course for
several years and during this time have reviewed numerous books, but we have not found
one that is written at the appropriate undergraduate level and adequately provides a com-
prehensive review of the designs and methods most frequently utilized in health science
research. As such, the creation of this text is informed by our experience teaching research
methods to over a thousand health science students.
The uniqueness of this textbook is twofold: its purpose/scope and structure. It is writ-
ten for clinical and non-clinical health science/health professions students. The purpose/
scope of this textbook is not to prepare students to conduct original research; rather it is
to develop a broad working knowledge of research processes across methodologies. Some
professional accrediting bodies (e.g., dosimetry) have required, and others are beginning
to discuss requiring, research methods in the curriculum. As such, it is our intention that
this text will be useful to other two-year and four-year colleges that offer professional allied
health degree programs. Feedback from our clinical students revealed that this working
knowledge prepared them to understand physicians and staff at rotation sites where research
was conducted. Our non-clinical students, who were admitted to masters’ and doctoral level
health career programs, shared that this introductory course prepared them to be successful
in graduate-level research courses or evidence-based practice courses.
This brings us to our second point, structure. This text blends broad content (at the under-
graduate level) with tips and exercises to help students with multiple learning styles effectively
engage with the material. Simply put, health professions students need to engage with course
material beyond memorization and the rote recitation of facts and utilize more complex think-
ing levels and skills (e.g., application of knowledge to novel situations, synthesis of material).
To that end, each chapter includes field-tested tips on how to study effectively based on stu-
dents’ learning style(s) and self-directed learning activities students can use to help develop
higher-order thinking skills. Anecdotally, we have had numerous students express gratitude
that the explicitly embedded learning style–based study tips in our research course yielded
more effective study skills that were transferable to other courses. Below are two examples that
are representative of the feedback we have received from our students on course evaluations:
• The [VARK] evaluations she did at the beginning, it really helped me learn about how to
study better. I learned how I best learn and study … made me excited to learn.
• I found it most valuable that [the professors] introduced how people learn differently. I
was able to identify my learning method and truly thrive in this class by configuring my
notes and strategies in studying.
xv

Sciences
xvi | Introduction
Additionally, we had a group of senior-year students from different learning styles vol-
unteer to review the textbook (see the acknowledgment page); they provided substantive
improvement feedback for the first edition of this book. All stated without exception:
• I really like this textbook I wish I had this when I took the class.
• This textbook would have made research methods so much easier to understand.
VARK
This text will utilize Dr. Neil Fleming’s VARK model to conceptualize how people learn. Dr.
Fleming (1995) categorized four learning styles/modes: visual (V); aural (A); read/write (R);
and kinesthetic (K). His example of what a student’s class notes would look like illustrates
the differences between each. A visual (V) student’s notes might look more like a colorful art
project than typical formatted class notes; the lecture notes of a student with a preference
for reading and writing (R) might resemble a verbatim transcript of the lecture that includes
word and letter patterns (mnemonics) of important concepts. The aural (A) student’s notes are
dismal; the student is so engrossed in listening to the professor that they forget to take notes,
while the kinesthetic (K) student’s notes feature the real examples and stories the professor
has told to illustrate points (Fleming, 1995, pp. 308–313). Dr. Fleming has an on-line test that
will help students identify their learning style: http://vark-learn.com/the-vark-questionnaire/
Dr. Fleming’s website [http://vark-learn.com/] includes a more detailed review of each
learning style and numerous mode-specific suggestions on how to study. A summary of the
characteristics of each learning mode is found in Table I.1 (Fleming, 2017).
TABLE I.1 Characteristics of the VARK Learning Styles
Characteristics of the learning styles/modes
Learning = image based (e.g., material has pictures, graphs, tables, charts).
V Studying = needs to use symbolic tools, such as arrows, color-coding, flowcharts, graphs, models, mind maps, and
hierarchies to represent course material.
Learning = verbally based (e.g., listening to lecturer and small/large group discussion).
Studying = needs to hear the course material to understand the course material; should ask to borrow an R stu-
A
dent’s notes, should read out loud to self while studying privately and join a study group to talk out the concepts or
teach someone the concepts.
Learning = textual based (e.g., books, handouts, notes, lists).

R Studying = needs repetition in reading and rewriting notes to learn, requires supplemental reading material, prefers
attending professor’s office hours to joining a study group.
Learning = tactile, sensory based (e.g., lab activities, role-play, case studies).
K Studying = needs practical applications and interaction with material, needs to move while studying, especially
while reading.
Adapted from R. Whittemore and G. D’Eramo Melkus, “Designing a research study,” The Diabetes Educator, vol. 34, no. 2, p. 202.
Copyright © 2008 by SAGE Publications.

Sciences
Introduction | xvii
While VARK technically is not a learning style theory, it does include assessment of other
factors that influence learning (e.g., motivation; social, physical, and environmental elements).
There are hundreds of research studies on the applicability of VARK in a variety of student
populations. Research conducted in the United States and internationally with health science/
health professions students has documented that a significant number are multimodal, mean-
ing the students have two or more predominant learning styles (Balasubramaniam & Indhu,
2016; Gebru, Nasrabadi, Nigussie, & Kahsay, 2016; James, D’Amore, & Thomas, 2011; Lujan
& Dicarlo, 2005; Mon, Fatini, Ye, Barakat, Jen, & Lin, 2014; Prithishkumar & Michael, 2014;
Meehan-Andrews, 2009). Practically, this means that students must utilize each mode-specific
strategy to learn. The research also suggests that simple awareness of learning styles, without
embedding learning style–specific teaching strategies, has not yielded enhanced student learn-
ing outcomes (Liew, Sidhu, & Barua, 2015), while embedded teaching strategies intentionally
developed across learning styles (Alkhasawneh, Mrayyan, Docherty, Alashram, & Yousef, 2008;
Fleming, 1995; Meehan-Andrews, 2009) or coaching on study methods matched to learning
style(s) (Kumar & Chacko, 2012), was associated with increased student learning outcomes.
Learning preference studies conducted with health care patients have also found that
a significant number of patients are multimodal learners. For example, in additional to
auditory methods, “incorporating kinesthetic methods of learning, such as role plays and
problem-solving case scenarios, into standardized asthma education curricula may be bene-
ficial to patients and families in terms of understanding and using their regimen” (Dinakar,
Adams, Brimer, & Silva, 2005, p. 683). A randomized controlled study demonstrated that
patients’ retention of diabetes education increased significantly when educational materials
were targeted to both health literacy levels and learning style preferences (Koonce, Giuse,
Kusnoor, Hurley, & Fei, 2015). Another randomized control study found a statistically sig-
nificant improvement in self-care behaviors in patients’ living with type 2 diabetes when
the patient education was targeted specifically to learning style (Moghadam, Araghi, Bazzi,
Voshani, & Moonaghi, 2017).
Additionally, it has become more common on intake forms to ask patients how they
would like to receive information on a diagnosis or receive patient education. For example,
patients are asked to list their preference(s) with regard to how they would prefer their
physician communicate with them.
Would they prefer the physician:
• draw a diagram;
• verbally explain it to them;
• provide handouts and brochures to read; or
• use a model that they can touch?
Once the physician, nurse practitioner, or physician assistant explains the diagnosis or
treatment plan, it is often other members of the health care team who are called upon to
reinforce the information with the patient or the patient’s family, underscoring the impor-
tance of exposing students to the concept of learning styles.
It is our goal that the activities found at the end of each chapter will supplement instruc-
tors’ in-class teaching and activities and provide students with ample (self-directed and
learning style–specific) exercises to foster a deep connection to the material.

Sciences
xviii | Introduction
REFERENCES
Alkhasawneh, I. M., Mrayyan, M. T., Docherty, C., Alashram, S., & Yousef, H. Y. (2008). Problem-based
learning (PBL): Assessing students’ learning preferences using VARK. Nurse Education Today, 28,
572–579. doi:10.1016/j.nedt.2007.09.012
Balasubramaniam, G., & Indhu, K. (2016). A study of learning style preferences among first year
undergraduate medical students using VARK model. Education in Medicine Journal, 8(4), 15–21.
doi:10.5959/eimj.v8i4.440
Dinakar, C., Adams, C., Brimer, A., & Silva, M. D. (2005). Learning preferences of caregivers of asth-
matic children. Journal of Asthma, 42, 683–687. doi:10.1080/02770900500265157
Fleming, N. D. (1995). I’m different; not dumb. Modes of presentation (VARK) in the tertiary classroom. In
A. Zelmer (Ed.), Research and development in higher education, proceedings of the 1995 Annual
Conference of the Higher Education and Research Development Society of Australasia (HERDSA),
Volume 18 (pp. 308–313).
Fleming, N. D. (2017). VARK: A guide to learning styles. Retrieved from http://vark-learn.com/
Gebru, A. A., Nasrabadi, A. N., Nigussie, A. W., & Kahsay, W. G. (2016). Assessment of postgraduate
international students’ learning preferences at Tehran University of Medical Sciences, Tehran,
Iran. Journal of Medical & Allied Sciences, 6(1), 14–22. doi:10.5455/jmas.210557
James, S., D’Amore, A., & Thomas, T. (2011). Learning preferences of first year nursing and midwifery
students: Utilizing VARK. Nurse Education Today, 31(4), 417–423. doi:10.1016/j.nedt.2010.08.008
Koonce, T. Y., Giuse, N. B., Kusnoor, S. V., Hurley, S., & Fei, Y. (2015). A personalized approach to
deliver health care information to diabetic patients in community care clinics. Journal of the Medical
Library Association, 103(3), 123–130. doi:10.3163/1536-5050.103.3.004
Kumar, L. R., & Chacko, T. V. (2012). Using appreciative inquiry to help students identify strategies
to overcome handicaps of their learning styles. Education for Health: Change in Learning & Practice,
25(3), 160–164. doi:10.4103/1357-6283.109794
Liew, S., Sidhu, J., & Barua, A. (2015). The relationship between learning preferences (styles and
approaches) and learning outcomes among pre-clinical undergraduate medical students. BMC
Medical Education, 15(44), 1–7. doi:10.1186/s12909-015-0327-0
Lujan, H. L., & Dicarlo, S. E. (2005). First year medical students prefer multiple learning styles.
Advances in Physiology Education, 30, 13–16.
Meehan-Andrews, T. A. (2009). Teaching mode efficiency and learning preferences of first year nursing
students. Nurse Education Today, 29, 24–32. doi:10.1016/j.nedt.2008.06.007
Moghadam. S., Araghi, F. H., Bazzi, A., Voshani, H. B., & Moonaghi, H. K. (2017). Examining the
effects of training according to learning styles on self-care among patients with Type 2 Diabetes.
Evidence Based Care Journal, 6(4), 47–56. doi:10.22038/ebcj.2016.19599.1462
Mon, A. A., Fatini, A., Ye, C. W., Barakat, M. A., Jen, P. L., & Lin, T. K. (2014). Learning style pref-
erences among pre-clinical medical students. Journal of Medical & Allied Sciences, 4(1), 22–27.
Prithishkumar, I. J., & Michael, S. A. (2014). Understanding your student: Using the VARK model.
Journal of Postgraduate Medicine, 60(2), 183–186. doi:10.4103/0022-3859.13233

Sciences
GENERATING NEW KNOWLEDGE:
CONCEPTUALIZING AND
PLANNING RESEARCH
INTRODUCTION
1
CHAPTER GOALS:
It is important for students studying in the field of health science or a
related health care profession field to have a basic understanding of the
THE GOALS OF THIS CHAPTER ARE:
research process. Research is the driving force behind improving patient
care and advancing health care knowledge and practice. The purpose • To provide working definitions of the
of this chapter is not to prepare students to conduct original research; terms research, systematic investiga-
rather it is to develop a broad working knowledge of research processes tion, scientific merit, paradigm, research
across methodologies. methodology, and research methods.
The word research is used in a variety of ways in higher education. • To introduce foundational research
For example, a professor requires students to write a research paper; the concepts and practice that will
students work with a research librarian to optimize search terms, or the be explored in greater detail in
students use their phones to research the hours the library is open. Simi- subsequent chapters.
larly, the term health science is used in numerous contexts; for example, a
health science professional, a health science major, or the health science
building. Therefore, an operational definition of how both terms will be
used within this book is required. LEARNING OBJECTIVES:
BY THE COMPLETION OF THIS CHAPTER,

HEALTH SCIENCE—DEFINED THE STUDENT WILL BE ABLE TO:
The terms health science professionals, health care professionals, and health • Define at least 30 terms related to
care providers have been used both interchangeably and distinctly to the research process;
describe health professionals who fall under the term allied health pro-
• Compare basic and applied/
fessionals. According to the Association of Schools of Allied Health
clinical research;
Professions (ASAHP), allied health professions are non-nurse and
non-physician health care providers that constitute over 60 percent of • Differentiate among
the health care workforce. The association states that this workforce is research methodologies;
required to (ASAHP, 2016, para. 1): • Identify the philosophical assump-
tions among the four research
use scientific principles and evidence-based practice for the diagnosis, evalua- paradigms; and
tion and treatment of acute and chronic diseases; promote disease prevention • Select the research methods
and wellness for optimum health and apply administration and management best suited to a study’s purpose
skills to support health care systems in a variety of settings. and methodology.
According to the ASAHP (2016, para. 4), there are over 200 allied
health careers. These include but are not limited to:

Sciences
2 | Research Methods in the Health Sciences
health administration personnel;

audiologists, speech language pathologists;
physical therapists, occupational therapists;
respiratory therapists;
diagnostic medical personnel (medical laboratory scientists, cytogenetic technol-
ogists, diagnostic molecular scientists, histotechnologists, and pathologists’
assistants);
imaging specialists (radiographers, nuclear medicine technologists, etc.);
specialists in cancer diagnosis and treatment (medical physicists, medical dosimetrists,
and radiation therapists);
nutritionists and dietitians;
physician assistants;
dental personnel (dental hygienists and dental assistants);
emergency medical personnel (EMTs, paramedics);
exercise science professionals (athletic trainers, exercise physiologists, kinesiotherapists);
health information technologists;
health educators (asthma educators, diabetes educators);
counselors (genetic counselors, mental health counselors, family therapists);
pharmacy personnel (pharmacy technicians and assistants); and
other healthcare providers and support personnel, such as medical assistants.
SCIENTIFIC METHOD AND EVIDENCE-BASED PRACTICE—DEFINED

There are two terms within the ASAHP definition that warrant further exploration: sci-
entific principles and evidence-based practice (EBP). Scientific principle is a global term
used to describe the utilization of scientific procedures; this term is closely related to an
important concept in this book: the scientific method. Scientific method, as defined by
Merriam-Webster’s (2017) medical dictionary, is the “principles and procedures for the
systematic pursuit of knowledge involving the recognition and formulation of a prob-
lem, the collection of data through observation and experiment, and the formulation
and testing of hypotheses.” The concept of the scientific method will be revisited later
in this chapter.
SIDE BOX 1.1 Important terms are bolded and italicized the first time they appear, so you may keep a
list of these terms to study.
The definition of some terms/concepts will be expanded upon in subsequent chapters.
Sackett, Rosenberg, Gray, Haynes, and Richardson (1996, p. 71) define the term
evidence-based medicine (EBM) as the highest level of care a physician can provide patients.
They conceptualized EBM as the

Sciences
Chapter 1 Generating New Knowledge: Conceptualizing and Planning Research | 3
conscientious, explicit, and judicious use of current best evidence in making decisions about the
care of individual patients … integrating individual clinical expertise with the best available external
clinical evidence from systematic research. By individual clinical expertise we mean the proficiency
and judgment that individual clinicians acquire through clinical experience and clinical practice.
In more recent years a new term, EBP, has evolved from EBM. Evidence-based practice
includes a wider scope of health care professionals and includes patients as partners in health
care delivery, as evidenced by Samonte and Vallente’s (2016, para. 1) definition.
Evidence-based practice is the
amalgamation of research evidence, experience and expertise, and patient preferences in the pro-
cess of clinical patient care. Clinicians bring collected experience, while patients describe personal
values and past encounters. These two aspects are combined with clinically relevant research that
has been proven with comprehensive methodology to develop the most optimal outcome in the
patient’s care process and overall quality of life. EBP can be incorporated into various fields such
as medicine, nursing, psychology, and allied health.
In both definitions, the authors make clear distinctions between EBM/systematic research
and EBP/research evidence. Systematic research and research evidence are different ways
of saying the same thing: all health care professionals (allied health professionals, nurses,
and physicians) make individualized patient treatment/care decisions rooted in the results
of research studies. Inherent in the definitions of EBM/EBP is the health care professional’s
ability to understand the research process so they are able to read, evaluate, interpret, and
synthesize the results of research studies.
For the purpose of this book, the definition of health science will be used as an umbrella
term to include any health care field where professionals use the scientific method, research
results, and EBM/EBP in the delivery of health care services. Therefore, the research examples
in this book will span numerous fields in health science and will examine research conducted
for the purpose of “diagnosis, evaluation and treatment of acute and chronic diseases, disease
prevention, health promotion, patient and health professions education … [or] best practices in
the application of healthcare administration and management skills” (ASAHP, 2016, para. 1).
RESEARCH—DEFINED
The Office for Human Research Protections (OHRP), which is located in the Department of Health
and Human Services (HHS), defines research as “a systematic investigation, including research
development, testing and evaluation, designed to develop or contribute to generalizable knowl-
edge” (OHRP, 2008, p. 2). Utilizing this definition, the term research transforms from describing
simple fact finding to describing a systematic process designed to generate new knowledge.
With this distinction made, it becomes clear that the purpose of research is not to increase
an individual’s knowledge of a topic; rather its purpose is to increase knowledge for every-
one in the field of health science. The OHRP definition also uses the word generalizable to
modify the word knowledge. This will be discussed later in the chapter; for now, simply put,
research = generation of new knowledge. The combination of the terms research and health
science implies that the pursuit of new knowledge will be health related and might involve

Sciences
clients/patients. This necessitates another definition, the definition of human subjects. The
OHRP (2008) regulations define human subjects (also known as research participants or
simply as participants) as “a living individual about whom an investigator (whether profes-
sional or student) conducting research obtains (1) data through intervention or interaction
with the individual, or (2) identifiable private information” (p. 2). Regulations regarding
conducting research with participants will be more fully explored in Chapter 2.
What Constitutes New Knowledge?

When a researcher has an idea for a research study, the researcher must determine if the idea
is indeed research by asking: If this idea is systematically defined and studied, does it have the
potential to generate new knowledge? The researcher answers this question by reviewing the
scientific literature; this process is called conducting a systematic review of the literature.
This is such a vital concept in the determination of whether an idea will meet the definition
of research and a critical component in the development of the research study that there is
an entire chapter (Chapter 3) devoted to this concept/action step of research.
For now, consider the examples below to understand the steps and thought processes
researchers undertake to determine whether an idea meets the definition of research.
Idea: A researcher is walking through the parking lot and someone blows smoke in their
face. They start to gag and their eyes water. From this experience the researcher
has an idea for a research study: What are the effects of exhaled smoke from an
individual smoking a cigarette on the health of others exposed to that smoke? The
researcher thinks this could be a very important study.
Literature review: The researcher conducts a review of the scientific literature by going
to the library and searching for scientific articles (previous research studies) related
to cigarette smoke and health. While reading the journal articles, it is realized that
this potential research question has already been answered. The exhaled smoke from
a cigarette has already been studied; it is labeled secondhand smoke. Secondhand
smoke has been determined to have negative health consequences to those exposed
to it. This research topic has progressed beyond secondhand smoke to the study of
the negative health impact of third-hand smoke (the toxic chemicals that are left
behind when the secondhand smoke dissipates).
Decision: Subsequent to the systematic review of the literature, the researcher realizes
that they have increased their personal knowledge, but that conducting this research
study will not add new knowledge to the health science scientific community. Thus,
the idea fails as a potential research topic. It will not generate new knowledge; it is
not research and would not be appropriate for a research study.
Implied in the example above, new knowledge builds on previous knowledge; answer-
ing one research question often leads the researcher to new unanswered questions. Thus,
research is iterative; it can be seen as an ever-expanding cycle of answered questions that
lead to emerging unanswered questions. Hundreds of questions must be asked and answered
to develop an expansive and detailed understanding of a research topic. For example, the
2014 U.S. Department of Health and Human Services’ (USDHHS) Surgeon General’s Report:

Sciences
The Health Consequences of Smoking synthesized 50 years of scientific research on the topic.
The number of questions that need to be asked and answered is indeed expansive in the
pursuit of new knowledge. Below are only a few summary examples of how the answer of
one group of questions leads to the generation of another avenue of inquiry.
• What are the health implications of smoking cigarettes on the lungs?

■■ Hundreds of studies building upon the findings of one another lead to the
knowledge that people who smoke are “15 to 30 times more likely to get
lung cancer or die from lung cancer than people who do not smoke” (Cen-
ters for Disease Control and Prevention, 2018, para. 2).
■■ New knowledge has been generated, but this answer leads to other unan-
swered research questions, like …
• What are the health implications, if any, of cigarette smoking on other organs in
the human body?
■■ Decades of research studies demonstrated that “smoking leads to disease
and disability and harms nearly every organ of the body … diseases caused
by smoking, including such common diseases as diabetes mellitus, rheu-
matoid arthritis, and colorectal cancer” (USDHHS, 2014, p. 7).
ɝɝ New knowledge has been generated, but this answer leads to other
unanswered research questions, like …
• To what extent is the smoke expelled from someone’s cigarette harmful to
human health?
■■ Numerous iterations of research were conducted to answer this question.
Exposure to “secondhand tobacco smoke has been causally linked to
cancer, respiratory, and cardiovascular diseases, and to adverse effects on
the health of infants and children” (USDHHS, 2014, p. 7).
• How effective are tobacco control policies in reducing smoking rates?
■■ Various policies were enacted and subsequently researched to answer
this question. The evidence “is sufficient to conclude that mass media
campaigns, comprehensive community programs, and comprehensive
statewide tobacco control programs prevent initiation of tobacco use and
reduce the prevalence of tobacco use among youth and adults” (USDHHS,
2014, p. 12).
• How effective are smoking cessation interventions?
■■ A multitude of interventions were researched to identify the most effec-
tive cessation strategies across diverse populations. This cumulative
evidence revealed that “tobacco cessation treatments are effective across
a wide population of smokers, including those with significant mental and
physical comorbidity” (USDHHS, 2014, p. 12).

Sciences
Over the past 50 years the research has and continues to explore the health consequences
of cigarette smoking. As illustrated in the examples above, research has expanded into areas
of inquiry on other tobacco products/nicotine delivery methods, exploring the impact of
tobacco control policies, and behavioral interventions to reducing tobacco use. Hopefully,
the point has clearly been made, research is iterative. Rigorous research not only answers
questions that generate new knowledge, but simultaneously generates new unanswered
questions to open new avenues of inquiry that can be systematically investigated.
What Are the Components of a Systematic Investigation?

To generate new knowledge, the OHRP definition also states that the researcher must use
systematic investigation. Systematic investigation is comprised of numerous components
that will be covered in greater detail throughout this book. In this chapter, the focus is
on introducing the terms, concepts, and components of a systematic investigation. A sys-
tematic investigation includes utilizing approved and predefined sets of procedures, also
referred to as design and methods, to conduct a research study. The design of the study is
the framework, or roadmap, of how the study will be conducted; this framework includes
the procedures (methods) used to conduct the study.
Whittemore and D’Eramo Melkus (2008) described research as a five-stage process:
Conceptual, design, empirical, analytic, and dissemination. Table 1.1 is an adaption of
these five stages, including the action steps a researcher undertakes during each stage of
the research process.
TABLE 1.1 Research Stages and Corresponding Action Steps
Stages Action Steps
Conceptual (Thinking) Having an idea (research problem), systematically reviewing the literature to verify the problem
has the potential to generate new knowledge, writing a problem statement, a research purpose
statement, and research question(s)
Design (Planning) Selecting the best research design and research methods (sampling, data collection, data analy-
sis) that align with research question(s), submitting the research study proposal for IRB review
Empirical (Doing) Obtaining IRB approval to conduct the study, recruiting/selecting the sample, obtaining informed
consent from participants, collecting the data
Analytic (Analyzing) Utilizing the best analysis strategies to yield meaningful results from data (answering the
research questions and simultaneously uncovering new avenues of inquiry)
Dissemination (Sharing) Writing a journal article to share results/finding (new knowledge) with the scientific community
Adapted from “Designing a Research Study,” Diabetes Educator, Vol. 34, no. 2
Methods
As previously stated, every research study utilizes research design and methods. A com-
prehensive review of research designs can be found in Chapters 6–10. In this chapter, the

Sciences
focus will be on defining the three research methods. Every research study employs three
research methods: sampling, data collection, and data analysis.
Definition of methods:
1. Sampling is the way a researcher recruits or selects individuals from a population
to be participants in the study.
2. Data Collection is the type of data that will be collected and the procedures/ pro-
cesses a researcher uses to collect data.
3. Data Analysis is how the researcher performs the analysis on the data that has
been collected.
SIDE BOX 1.2 Examples of each method, as well as introductory information on the alignment of
methods with methodology, are provided in the Methods and Methodology section of this chapter.
Scientific Merit
A research study must be conducted by a qualified researcher, meaning that the researcher
must have expertise in the research topic area and have the skills to extensively plan and
implement the research study. The ultimate purpose of conducting research is to dissemi-
nate the results/finding (new knowledge) to the larger scientific community. The scientific
merit of a study is the combination of the terms research and systematic investigation.
First, the researcher must ensure that the proposed research study meets the definition of
research. Second, the researcher must ensure that the proposed research study has been
constructed in such a way that once the study has been concluded, new knowledge can be
gleaned from the results of the study.
All research studies are judged for scientific merit. During the conceptualization stage
the researcher reviews all the previous research studies done on a specific topic to identify
and create the problem statement, which leads to the development of a research purpose
statement and research question(s). These three in combination outline, define, and direct
the study construction. These terms will be covered in greater detail in Chapter 4. Once the
researcher has conceptualized the proposed study, the next step is the design or planning
process. This involves selecting the best approved and predefined sets of procedures (design
and methods: sampling, data collection, data analysis) used to conduct the study.
Said another way, the study must clearly and concisely identify a problem that is worthy
of being investigated and will generate new knowledge (problem statement), have a clear
purpose of what the study will accomplish in relation to the problem (purpose statement),
and have questions that when answered have the potential to generate new knowledge
(research questions). The questions must be investigated by a qualified researcher in a
manner (design and methods) that will make it possible to adequately answer the questions.
Remember, it is the unbiased answers to the research questions that increase/generate
new knowledge.

Sciences
Review of Scientific Merit

All proposed research studies must be submitted to the scrutiny of an impartial Institutional
Review Board (IRB). The primary purpose of the IRB review is to ensure that in a research-
er’s zeal for new knowledge, human subjects are being treated ethically. If the researcher
is part of a university, typically there is an IRB on campus; if, however, the researcher is
not affiliated with an organization that has established an IRB, then the researcher must
pay a private IRB company for the review. This is such an important concept and practice
that there is an entire chapter (Chapter 2) devoted to ethical research, the protection of
human subjects, and the role of the IRB in that protection. Included in the IRB’s review of
the proposed research study is the determination of the study’s scientific merit. This review
includes an assessment of the researcher’s qualifications (knowledge, expertise, and skills),
design of the research study, and implementation plan. One IRB company (Solutions IRB,
2017, para. 9) offers the following questions to assist researchers in the planning phase of
their research study. (Note: Some questions have been altered for brevity).
1. Are the procedures consistent with sound research design?

2. Is the study design appropriate given the hypothesis being tested? […]
3. Is the study designed in a manner likely to answer the research questions?
4. Are the research questions aligned with the proposed data collection and analysis?
5. Does the study have the potential to contribute to the field or add to the existing
body of knowledge?
6. Will the knowledge to be gained [be] sufficiently important to justify the risks involved?
7. Are the risks minimized and benefits optimized to the extent possible?
8. Does the proposed sample contain [enough] participants […] to result in reliable
[and] replicable [results]?
DIFFERENT TYPES OF RESEARCH

Although different types of research (basic and applied/clinical) are conducted for different pur-
poses (exploratory, descriptive, evaluative, and explanatory), all are evaluated for scientific merit.
One type of research is not superior to the other; each simply has different goals and purposes.
The two types of research are basic and applied/clinical. Basic research is a type of
research that seeks to enhance overall knowledge about the “physical, biological, psycholog-
ical, or social world or to shed light on historical, cultural, or aesthetic phenomena” (Leedy
& Ormrod, 2013, p. 27). Applied research is a type of research that seeks to study issues
that have “immediate relevance to current practices, procedures, and policies” (Leedy &
Ormrod, 2013, p. 27). For the purpose of this book, applied and clinical research will share
a definition with the only distinction being the setting. Clinical research is applied research
conducted in the health care arena with the primary focus of finding practical solutions to
improve patient care/clinical outcomes.
For example, a basic research study would explore how the human immunodeficiency virus
(HIV) replicates in a human cell. This type of study is vital to creating an understanding
of the life cycle of HIV, despite the fact it has no direct application to real-world practice.

Sciences
Clinical research, often building on basic research studies, yields a practical, real-world out-
come. For example, is drug X better than drug Y at reducing circulating virus in people living
with HIV? The outcome of this study would have direct applicability to improving patient
care. Physicians, nurse practitioners, and physician assistants could use the results of this
study (EBM/EBP) to determine which drug to prescribe to their patients living with HIV.
There are several global purposes a research study can have; the purpose of the study
is directly linked to how a researcher will construct the research question. Chapter 4 will
provide guidance on the wording of research question development and examples of research
questions written for each purpose; for now, a simple definition of each will be sufficient
(Lanier, Ford, Reid, & Strickland, 2014).
• Exploratory: Explore or investigate to determine scope of issue or to understand

a problem that has not been clearly defined.
• Descriptive: Describe the problem (who, what, where, and when; how many?)
• Evaluative: How well is this working?
• Explanatory: Determine a cause-and-effect relationship.
Simply stated, for a study, regardless of type or purpose, to have scientific merit, the
totality of the proposed research study (problem statement, purpose statement, research
questions, design, and methods) must be in alignment so it is possible to generate new
knowledge. There will be more detailed information on this throughout the book; scientific
merit is a concept that is woven through every chapter.
How Does One Choose the Correct Research Type,

Purpose, Research Design, and Methods?
The answer is simple: a researcher chooses the type, purpose, design, and methods that
can best answer the proposed research question(s). Prior to a discussion about research
questions, an exploration of a researcher’s philosophical thought processes is warranted.
Researchers can have varying philosophical assumptions about the world. These assump-
tions are foundational to how a researcher will plan and implement the research study.
This segment of the chapter discusses these varying philosophical assumptions and links
assumptions to research practices.
PARADIGMS AND METHODOLOGY
SIDE BOX 1.3 As you read, reflect on which paradigm and associated research methodology most
closely expresses how you see the world. For example:
Was one of your favorite courses a literature class that required the analysis of symbolism to reveal the
underlying truths about the human condition? If yes, you most likely will be drawn to Qualitative research.
Conversely, was your favorite class a chemistry lab where you conducted tightly regulated experiments
that had one correct outcome? If yes, you most likely will be drawn to Quantitative research.
If you are thinking, “I have no favorite; I’d use analysis of symbolism or a lab experiment based on the
purpose of the project,” then you are mostly likely drawn to Mixed Methods research.

Sciences
A researcher’s background and position will affect what they choose to investigate, the angle of
investigation, the methods judged most adequate for this purpose, the findings considered most
appropriate, and the framing and communication of conclusions (Malterud, 2001, pp. 483–484).
The quote from Malterud (2001) suggests that how a researcher sees the world informs
the type of research ideas and resulting research questions a researcher will have. These
basic assumptions or the worldview a researcher operates under are conceptualized as
research paradigms.
Guba (1990) suggests that the determination of one’s paradigm can be found in how one
answers the following three questions (p. 18):
1. Ontological: What is the nature of the “knowable”? Or, what is the nature of “reality”?
2. Epistemological: What is the relationship between the knower (the inquirer) and
the known (or knowable)?
3. Methodological: How should the inquirer go about finding out knowledge?
The answers to questions about truth/reality (ontology), knowledge (epistemology),

and the actions taken in the pursuit of new knowledge (methodology) shed light on the
researcher’s paradigm. The researcher’s paradigm is linked to the research methodology a
researcher will employ. Methodology, in combination with the type of research question, will
delineate the appropriate research design, sampling method(s), data collection method(s),
and data analysis method(s) utilized in the systematic investigation.
There are other research paradigms; however, for the purpose of this book, only posi-
tivism, post-positivism, constructivism, and pragmatism will be explored. A summary of
the answers to the questions Guba (1990) posed is presented by paradigm in the following
chart. Source material used to create Table 1.2 is an amalgamation of several authors’ inter-
pretation of the paradigms (Creswell, 2009; Guba, 1990, pp. 18–27; Guba & Lincoln, 1994;
Mack, n.d.; Reichardt & Rallis, 1994; Teddlie & Tashakkori, 2009, pp. 4–15).
The positivism and post-positivism paradigms are very similar: both use a quantitative
methodology to conduct research. The differences between the two are subtle. The positivist
researcher seeks to prove the existence of relationships that are rooted in the immutable
physical laws of nature. The post-positivist researcher entertains the possibility that units of
measurement are fallible and that it might not be possible to control for all variables; there-
fore, they hesitate to say (like the positivist) that they have absolutely, undeniably proved
something. Rather, the post-positivist researcher sees truth as constructs open to retesting
and revision if enough scientific evidence is found. The post-positivist paradigm is the one
that governs most health science quantitative research. This stance is very different from
the constructivist researcher, whose beliefs espouse that accounts of an event can be equally
true or false; the researcher does not need to find one truth but the whole truth gleaned from
participants’ experiences. These health science researchers value inductive explorations,
believing the most significant evidence cannot be quantified in numerical representations.
Constructivists find meaning in the analysis of their participants’ perspectives, which leads
to rich, comprehensive, and detailed stories of the central phenomenon (Creswell, 2009).
Finally, the hallmark of the pragmatist researcher is to do what works, rejecting the need to

Sciences
TABLE 1.2 Paradigms: Assumptions and Methodologies
Ontological Epistemological Methodological Research

Paradigm Assumptions Assumptions Assumptions Methodology
Positivism Cause-and-effect rela- Objective: the researcher Deductive reasoning: empirical study Quantitative
tionship can be found, can control for bias and (often conducted in a lab) that con-
natural laws predict and must remain objective so trols variables and uses rigorous sci-
control events, research the results of the research entific methods to test hypothesis.
should seek absolute are free from value and
truths. bias.
Post-Positivism Truth exists as a result Objective: control for bias Deductive reasoning: empirical study Quantitative
of natural laws, but per- and remaining objective that controls variables and uses
haps not all the laws are is the benchmark, but ab- rigorous scientific methods to test
completely understood, solute control for subjec- hypothesis but believes research can
therefore one should tivity and bias may not be be conducted outside the lab.
not say there is absolute achievable.
truth.
Constructivism Reality and truth are Subjective: since truth and Inductive reasoning: ontology and Qualitative
dynamic and fluid, there reality are within individ- epistemology become one; con-
is not one truth; rather uals, it is the role of the structs are revealed through an
there are multiple truths researcher to work directly iterative process and are expressed
and realities which are with individuals to uncover in rich textual format. The goal is
rooted in individual meaning of the phenome- to generate key findings about the
and social constructs, non; the research findings phenomenon which contributes to
meaning what is true for are the result of the in- advancing knowledge.
one can be untrue for depth interaction between
another. researcher and participant.
Pragmatism Truth is both governed Objective and subjective: Deductive and inductive reasoning: Mixed Methods
by natural law and sub- rejects either/or choices; both philosophical assumptions
jective reality. rather the researcher are compatible; one can seam-
should focus on what lessly go between multiple types of
works. evidence and analysis to answer a
research question. There is no loy-
alty to either paradigm: Positivism/
Post-Positivism or Constructivism.
Sources: Creswell (2009), Guba (1990), Guba and Lincoln (1994), Mack (n.d.), Reichardt and Rallis (1994), Teddlie and Tashakkori (2009).
follow only one paradigm. The pragmatist health science researcher views the paradigms
and research methodologies as compatible and will use what works by combining methods
of data collection and analysis to “generate deeper and broader insights, to develop import-
ant knowledge claims that respect a wider range of interest and perspectives” (Greene &
Caracelli, 1997, p. 7). Pragmatism allows the health science researcher the flexibility to
select designs, techniques, procedures, and data collection strategies that provide practical
results for problem-centered research questions (Creswell, 2009; Greene & Caracelli, 1997;
Reichardt & Rallis, 1994; Tashakkori & Teddlie, 1998).

Sciences
FIGURE 1.1 The health science research designs that are associated

with each methodology.
Researchers can become entrenched
in their paradigm, believing that their
assumptions about research are the only
Quantitative Designs Qualitative Designs correct way of conceptualizing and con-
Experimental ducting research. It must be stated that the
Focused Ethnography
Grounded Theory philosophical assumption underlying the
Experimental Quasi-experimental Phenomenological construction of this book is pragmatism.
Solomon 4 group Simple time series Case Study While the authors of this book personally
Pretest/Posttest Pretest/Posttest
Posttest only
operate under different worldviews (para-
Posttest only
Within subject Within subject digms), all agree that one paradigm is not
Control group: time Control group: time superior to another.
series series
etc. etc. Also, as shown in Table 1.2,
paradigms are linked to research method-
Non-experimental/Descriptive Mixed Methods Designs
Survey Convergent
ologies. Researchers who are positivist or
Developmental Explanatory post-positivist use the quantitative meth-
Observational Exploratory odology; constructivists use the qualitative
Cohort
Case-control methodology; and pragmatists use mixed
Correlational methods (a combination of quantitative and
qualitative methodologies). A summary of
the characteristics of the three methodol-
ogies are presented below (Creswell, 2009;
Creswell & Plano Clark, 2011; Merriam,
2009; Teddlie & Tashakkori, 2009).
SIDE BOX 1.4 Figure 1.1 is simply illustrative of which designs will be covered in subsequent chapters.
For now, focus on building a strong understanding of the characteristics of each methodology. Your
understanding will be sufficient if, upon the review of the methods used in a research study, you can imme-
diately identify the methodology of the study.
The understanding of methodologies you develop now is vital to understanding the material in
Chapters 4–10.
Quantitative Methodology: Research involves the use of deductive reasoning in the

collection and analysis of numerical data with the goal of proving, explaining, pre-
dicting, testing, describing, or comparing. Based on the purpose of the research, the
researcher can use the scientific method and will control, compare, or manipulate
variables. Paradigm: Positivism or Post-Positivism.
Qualitative Methodology: Research involves the use of inductive reasoning. The
researcher drives the data collection, analysis, and interpretation of comprehen-
sive verbal, narrative, and/or visual data in order to gain insights into a particular
phenomenon of interest. Paradigm: Constructivism.
Mixed Methods: Research asks questions that cross quantitative and qualitative meth-
odologies, often conducting two (qualitative/quantitative or quantitative/qualitative)

Sciences
studies to gain a more nuanced understanding of the topic. The combination and
order of quantitative and qualitative studies are directly related to the purpose of
the study and its resulting research questions. Paradigm: Pragmatism.
As previously stated, during the planning stage of a research study, the researcher selects
a research design and methods. Certain designs are exclusive to certain methodologies
(Figure 1.1), so when a researcher plans a study and selects a design, they are announcing
to the scientific community which methodology (paradigm) the researcher is working from.
DEDUCTIVE VERSUS INDUCTIVE REASONING

Health care providers use the process of deductive reasoning in providing patient care. For
example, a physician assistant (PA) is working with a patient who has a very high fever. The
patient just returned from a trip to a tropical region known for having very high rates of
malaria. Infection with malaria can cause high fevers. The PA would test the patient’s blood
for malaria. If the parasite that causes malaria is present, then the presumed diagnosis of
malaria is supported. If, however, there is no evidence of the parasite, the PA would begin
the process of ruling out other tropical diseases that cause high fever. If all tropical diseases
are ruled out, the PA would begin to test for other diseases that cause high fevers until
an answer is found. In research, quantitative researchers also use deductive reasoning by
employing the scientific method. They begin with a theory, collect data to test the theory,
and the results of the data analysis “either supports or refutes the theory” (Creswell, 2009,
p. 7). This process is also known as hypothesis testing. When the researcher is able to sta-
tistically disprove one hypothesis, it is rejected, and the researcher forms new hypotheses
to test until an answer has been found.
Qualitative researchers use inductive reasoning; they apply the findings from a few
observations to build a general/global understanding of an issue. Inductive reasoning and/
or qualitative research is a common practice when there is a lack of preexisting theory, or in
an area that has very little previous research (Creswell, 2009; Merriam, 2009). The findings
from these observations are not tested; rather they are offered as an explanation of what is
occurring or why something is occurring. It is the reader of the research who determines
whether the inductive conclusions apply to their situation.
At times, even quantitative clinical research must utilize inductive reasoning when the
full use of the scientific method (experimentation) on human subjects would be impossible
or unethical. The following research examples illustrate this point. Silverman, Masland,
Saunders, and Schwab’s (1970) work demonstrated how a few observations, when combined,
can provide a generalization applied to all cases (Figure 1.2). For example, the observations
from this study showed that none of the individuals reported below (n = 2,650) regained
brain function after 24 hours of brain inactivity. An electroencephalogram (EEG) is a med-
ical test that measures brain activity. This type of data collection method would fall under
quantitative research, yet the following examples reveal an inductive process to interpret
and apply the results of a few cases to a larger population.
Green and Lauber’s (1972) research found that only two observations had the ability
to expand the understanding of how long a child can have brain inactivity, unrelated to
hypothermia or drug use, and still recover brain activity for a period of time (Figure 1.3). It

Sciences
FIGURE 1.2 An illustration of inductive reasoning.

Adapted from David Silverman, et al., “Irreversible Coma Associated with Electrocerebral Silence,”
Neurology, vol. 20, no. 6. Copyright © 1970 by American Academy of Neurology.
Conclusion:
Combined total of Irreversible coma is
2,650 reported adult
cases from various Inductive reasoning associated with 24 hours
hospitals. of electrocerebral silence
(no brain activity).
FIGURE 1.3 Another illustration of inductive reasoning.

Adapted from: Joseph B. Green and Andre Lauber, “Return of Eeg Activity after Electrocerebral Silence: Two Case Re-
ports,” Journal of Neurology, Neurosurgery, and Psychiatry, vol. 35, no. 1. Copyright © 1972 by BMJ Publishing Group.
Case 1–
5 year old boy, coma, brain
Conclusion:
activity resumed after 24
“Until further experience with the
hours of inactivity.
determination of brain death in
Inductive reasoning children is acquired, the 24 hour
Case 2– criteria accepted for adults should
6 week old boy, seizures, not be presumed to be valid” for
brain activity resumed after children (p. 107).
24 hours of inactivity.
is clear that the researchers are not stating they proved that all children will recover brain
activity after 24 hours of brain inactivity, but inductively these two cases had the power
to inform EMB/EBP until more evidence is available to make a definitive determination.
METHODS AND METHODOLOGY

As previously stated, a systematic investigation requires the use of the correct methods based
on methodology, design, and research question. This concept will be discussed in finer detail
in Chapters 4–10, but for now, a thorough understanding of the fundamental differences
between methods based on methodology will suffice. Table 1.3 provides the characteristics
of each method (sampling, data collection, and data analysis) by quantitative and qualitative
methodology. The table also provides a few examples of the different kinds of data that can
be collected from the two methodologies. Mixed methods research would systematically
use both qualitative and quantitative methods in some combination.
Sometimes the methodology of a research study is not explicitly stated; it is assumed by
how the research question is written and by simply stating the research design and methods
used in the study that the reader will immediately understand the researcher’s paradigm
and the research methodology of the study.

Sciences
TABLE 1.3. Methods by Methodology: Sampling, Data Collection, and Data Analysis
Method: Method: Method:

Methodology Sampling Data Collection Examples Data Analysis
Quantitative Study is conducted Collects data on pre- Weight, blood pressure, Utilizes descriptive and
on a sample that determined variables, EEG results, results of inferential statistical
represents a larger includes a variety of a survey or depression procedures on various types of
population. The size measurements that can screening, structured ob- numerical data. Data analysis
of the sample is based be turned into a numer- servations (i.e., gait, etc.) is conducted at conclusion
on the purpose of the ical value, measurement of study; if the correct test is
study. Studies can instruments must be selected, analysis is quick and
have large numbers of tested for validity and straightforward; the results of
participants. reliability. analysis can be presented in
tables and charts.
Qualitative Requires only a small Data includes verbal, Unstructured interviews, Iterative process of breaking
group of purposefully narrative, and/or visual unstructured observa- data into small constructs
selected research data; data collection tions, participant diaries, (codes) to find patterns
participants. evolves as the study art, artifacts, etc. (themes) that reveal the
progresses. essence of meaning. Data
analysis begins while data is
being collected; the process is
lengthy, and the presentation
of findings includes rich textual
descriptions, direct quotes,
and/or images.
Sources: Creswell (2009), Guba (1990), Guba and Lincoln (1994), Mack (n.d.), Reichardt and Rallis (1994), Teddlie and Tashakkori (2009).
As a rule, quantitative research typically asks very narrowly defined questions that are
extensions of previous scientific studies. Conversely, qualitative research typically asks
very broad questions to explore a previously uncharted avenue of inquiry. This will be
explored in more detail in Chapters 3 and 4. Based on what is known about methodology
and methods, let’s revisit the research questions introduced earlier in the chapter and apply
the appropriate methods to each.
1. What is the meaning of successfully completing a smoking cessation program, tran-
sitioning from a state of tobacco dependence to a state of being tobacco free, for
10–20 individuals who have recently been diagnosed with smoking-related cancer?
Adapted from Vangeli & West (2012).
2. What are the effects of maternal tobacco use during pregnancy on fetal development?
Adapted from USDHHS (2014).
Immediately it becomes clear by the way the research question is written that the first
research question is from a qualitative study, while the other research question is from a
quantitative study. The first question seeks to uncover the individuals’ personal experiences

Sciences
(subjective reality) related to participating in a smoking cessation program (uncover meaning

of the phenomenon). This aligns with the qualitative assumptions about how one seeks truth
and knowledge. The researcher would use inductive logic working directly with a carefully
selected group of individuals (sampling), the researcher might interview participants over
the course of several months and ask participants to keep diaries (data collection). The
researcher would read and reread transcripts of hundreds of hours of verbal interview data
and read hundreds of pages of participant diaries to uncover the common themes of meaning
and experience across the 10–20 individuals (data analysis).
The second question clearly needs to use deductive logic. This aligns with the quan-
titative assumptions about how one seeks truth and knowledge. The researcher would
select a large group of participants to include pregnant women who used tobacco prod-
ucts and similar women who did not use tobacco products during pregnancy (sampling).
The researcher would have to control for numerous variables (e.g., length of maternal
tobacco use, type of tobacco product used, maternal age, diet, frequencies of prenatal
visits). The researcher might ask permission to review the results of medical tests con-
ducted during prenatal care; for example, blood tests, amniotic fluid tests, ultrasound
tests (data collection) and use several different types of statistical analyses on the data
to determine if there were differences in fetal development (data analysis) associated
with tobacco use.
Rigor and Methodology

While the quantitative and qualitative researcher view the world differently and seek truth
in very divergent manners, both types of research approaches are scientific and seek to
systemically generate new knowledge. Despite the vast differences in how the study is
conducted (e.g., sampling, data collection, and data analysis), each methodology should
be viewed as having the ability to conduct rigorous (high-quality) research. Although the
concept of rigor is shared by both methodologies, the practices utilized by each to ensure
it are very different. Chapter 5 will provide greater detail on the intersecting and analogous
practices and procedures researchers of quantitative and qualitative methodologies utilize
to ensure rigor.
Now that the concept of methodology has been presented, there is an important point
that needs to be revisited: the term generalizable results in the OHRP definition of research.
Generalizable results are a comment on how sampling was conducted and to what extent the
study results are “likely to apply, generally or specifically, in other study settings” (Kukull
& Ganguli, 2012, p. 1886). Generalizability is a concept that is only relevant to quantitative
research. This is why the definition of research presented earlier—research = generation
of new knowledge—is a much more appropriate definition of research. Removing the word
generalizable, the definition becomes inclusive of research conducted under either quantita-
tive or qualitative methodologies. In addition, not all quantitative research studies achieve
generalizable results, meaning the results of the study might only be applied to one group
and might not be valid for other groups or study settings. Even within quantitative research,
there are levels of rigor. Rigor can be determined by an evaluation of the study’s internal
and external validity, which will be explained in Chapter 5.

Sciences
SUMMARY OF MAIN IDEAS AND CONCEPTS

Below is an outline of how all the key concepts presented in this chapter are related. During
the conceptualization and planning phase, all researchers ask themselves: Has this study
been planned in such a way that the potential to generate new knowledge exists?
1. Research idea: Is it research?

a. Literature review is used to determine if the research idea, when systematically
studied, can lead to creation of new knowledge.
b. There must be a researchable problem (something that is unknown).
c. Purpose of research must be in alignment with the research problem.
d. The research question(s) or hypothesis must align with the research purpose.
2. Planning the research study: Are all components in alignment?
a. Methodology is selected
i. Methodology is based on the researcher’s paradigm and the nature of the
research question(s) being asked.
ɝɝ Quantitative
ɝɝ Qualitative
ɝɝ Mixed methods
b. Design is selected
i. Design must be appropriate for the methodology and research question(s).
c. Methods are selected
i. Methods must be appropriate for the design and research question(s).
ɝɝ sampling – correct procedures for participant selection/recruitment.
ɝɝ data collection – correct type of data collected in the correct way.
ɝɝ data analysis – correct data analysis procedures/tools are selected.
STUDY PRACTICE
Study tips for all VARK styles: Reducing notes from 3:1 is a recommendation that cuts
across all learning styles. Therefore, students from any of the VARK learning styles should
start making flash cards for every bolded/italicized term. The important terms/concepts
presented in this chapter include:
ɠɠ scientific principles ɠɠ systematic investigation
ɠɠ evidence-based practice ɠɠ design
ɠɠ scientific method ɠɠ methods:
ɠɠ research ɝɝ sampling
ɠɠ generalizable knowledge ɝɝ data collection
ɠɠ human subjects ɝɝ data analysis
ɠɠ scientific literature ɠɠ scientific merit
ɠɠ literature review ɠɠ problem statement

Sciences
ɠɠ purpose statement ɠɠ paradigms

ɠɠ research question(s) ɠɠ methodology
ɠɠ Institutional Review Board (IRB) ɠɠ positivism
ɠɠ basic research ɠɠ post-positivism
ɠɠ applied/clinical research ɠɠ constructivism
ɠɠ exploratory ɠɠ pragmatism
ɠɠ descriptive ɠɠ quantitative methodology
ɠɠ evaluative ɠɠ qualitative methodology
ɠɠ explanatory ɠɠ mixed methods
• Some of the concepts in Chapter 1 are expanded upon in later chapters; your
definitions of important terms and concepts might need to be amended. Don’t
let that hold you back from making cards now. As the definitions or concepts
become more complex, your basic understanding of the terms at this point will
be a required foundation for the new knowledge to land on.
■■ Learning, just like research, is an iterative process!
• Don’t use premade flash cards as they will not help you learn! The process of
making the cards yourself by paraphrasing (putting the definitions into your own
words) the information found in the chapter will reinforce learning.
■■ For example, students have shared with us, “I understood the informa-
tion while I was reading it but realized I didn’t understand when I took
the test.” When you reduce your notes 3:1 by paraphrasing you can easily
identify if you understand the material. Simply said, if you understand the
content you will be able to paraphrase. If you cannot paraphrase a con-
cept, you have identified content you don’t understand. It is better to find
areas that you need clarification on now versus when you sit down to take
the exam.
Other study tips:
• V: start making a flowchart of how various components are related. Color code
your flash cards based on methodology or how terms relate to one another.
• A: talk out loud to yourself as you create the flash cards and when you use flash
cards to study.
• R: many of the concepts presented in the book have been simplified. If you need
more detail about the topic, look up the resource material in the reference sec-
tion. A simple web search might yield conflicting results and confuse you.
• K: make sure you move around while reading; the simple practice of using your
finger or pen to trace the words as you read really helps.

Sciences
PRACTICE MULTIPLE-CHOICE QUESTIONS

1. Which of the following is the correct way for a researcher to think or act? A
researcher should:
a. use the scientific method to test hypotheses
b. realize that truth is relative; meaning an account can be both true and false at
the same time
c. be detached, neutral, and objective in order to strictly control all variables
d. be active and engaged in the research, realizing that all research is value laden
e. attempt to prove a cause-and-effect relationship
f. seek a rich understanding of a phenomenon
g. Any of the above
The purpose of this study was to explore the experiences women who self-identify as les-
bian had with the health care system as their partners prepared to give birth. The researcher
purposefully selected 20 women with whom to conduct in-depth interviews. The researcher
read and reread transcripts of the verbal interview data to identify common themes among
the experiences. The researcher found that being acknowledged as a co-mother to the new-
born was the most important finding (adapted from Dahl, Fylkesnes, Sorlie, & Malterud,
2013). Answer questions 2–4.
2. Which methodology did this study use?
a. Mixed Methods
b. Applied
c. Quantitative
d. Qualitative
3. The in-depth interviews the researcher conducted is how the researcher ?
a. sampled
b. collected data
c. analyzed the data
d. achieved scientific merit
4. Reading and rereading the data looking for themes is how the researcher ?
a. sampled
b. collected data
c. analyzed the data
d. achieved scientific merit
Studies have documented that Pre-Exposure Prophylaxis (PrEP) can reduce the risk of
HIV infection in people who are at high risk by more than 90% (CDC, 2016). As such, this
drug treatment is strongly recommended for people who are HIV-negative and at very high
risk for HIV infection. However, insufficient studies have been conducted that explore and
describe the experiences of those taking PrEP daily. Further exploration must be conducted
to understand the meaning this drug has in the lives of those who are at ongoing risk of
HIV infection. Answer question 5.

Sciences
5. Which methodology would be used in this study?

a. Mixed Methods
b. Applied
c. Quantitative
d. Qualitative
The purpose of this study was to create an integrated care manual for health care providers
using the perspectives of patients who are homeless and living with chronic illnesses. The
manual was developed from both the results of quantitative survey data and the analysis
of in-depth interviews. The combined results identified 14 areas that can enhance provider
competency when working with individuals that are homeless. This manual will assist the
health care team in providing higher-quality health care to people who are homeless (adapted
from the Colorado Coalition for the Homeless, 2013). Answer questions 6 and 7.
6. What type of research was performed?
a. Basic
b. Applied/clinical
7. Which methodology was used in this study?
a. Mixed Methods
b. Applied
c. Quantitative
d. Qualitative
The purpose of this study was to measure the physiological stress responses of radiologic
technology students during their initial clinical rotations. Specifically, the first rotation where
the students took images (X-rays) of patients in a familiar setting (radiology department
where they had their lab classes) versus an unfamiliar setting (first time taking portable
X-rays in the emergency department). Cortisol is a hormone that is related to stress. In this
study stress responses were measured using a lab test that measured the levels of cortisol
in the students’ saliva (adapted from Pottier et al., 2011). Answer question 8.
8. Which methodology would be used in this study?
a. Mixed Methods
b. Applied
c. Quantitative
d. Qualitative
9. Once a researcher has an idea for a new study, the first thing they should
do is:
a. Conduct a systematic review of the literature to see if the idea rises to the level
of research.
b. Collect and analyze data, as this is the only way to find out if the idea rises to
the level of research.
c. Consider whether the idea is from the positivism, constructivism, or
pragmatism perspective.

Sciences
d. Submit the idea to the Institutional Review Board (IRB) to see if the idea is ethical.
10. Why do researchers ask themselves this question: Has this study been planned
in such a way that the potential to generate new knowledge exists?
a. This question is used to keep the concept of scientific merit in the researcher’s
mind as they conceptualize, plan, design, and conduct the study.
b. This question is used solely to guide the researchers through the literature
review process.
c. This question is used to keep the concept of scientific method in the researcher’s
mind as they disseminate the findings (write the journal article).
d. This question is used solely to guide the researcher through the process of
identifying the paradigm the researcher holds.
STUDY ACTIVITIES
You now know enough about quantitative and qualitative methodologies to be able to take
the terms presented in Chapter 1 and determine which methodology they belong to.
a. Take a piece of paper and make two columns, Quantitative and Qualitative. Now
take your flash cards, and ask yourself: Does the word on this flash card apply
to both methodologies or is it limited to one?
i. If the card applies to both quantitative and qualitative, place it above the
paper. If the card is limited to one of the methodologies, then place it in the
appropriate column.
ii. Once you have placed all the cards at the top of the page or in one of the
columns, make a hierarchal outline with the cards based on how they are
related to each other. This activity will be very helpful to students of all
learning styles (reducing notes to 3:1).
b. As a variation on the flash card activity, make a list of terms and words found
in this chapter. Now, on a separate piece of paper, write each word/term under
one of the headings below.
Match the list of words, terms, and concepts with the methodologies. For exam-
ple, under which column would the following words/terms go?
Scientific merit; sampling; compare two groups; understand; impact;
research; paradigm; variables; scientific method; data collection; predict-
ing; rigor; and inductive reasoning.
Quantitative only Common to both QUAN/QUAL Qualitative only
c. Advanced activity: Without the use of flash cards

i. From memory, write the terms in the corresponding columns.
Quantitative only Common to both QUAN/QUAL Qualitative only
ii. From memory, write examples of methods for each methodology.

Sciences
Quantitative Qualitative
REFERENCES
Association of Schools of Allied Health Professions. (2016). Definition of allied health. Retrieved from
http://www.asahp.org/about-us/what-is-allied-health/definition-of-allied-health/
Centers for Disease Control and Prevention. (2016, September 19). HIV/AIDS: Pre-exposure prophylaxis
(PrEP). Retrieved from https://www.cdc.gov/hiv/risk/prep/
Centers for Disease Control and Prevention. (2018, July 19). What Are the Risk Factors for Lung Cancer?
Retrieved from https://www.cdc.gov/cancer/lung/basic_info/risk_factors.htm
Colorado Coalition for the Homeless. (2013, October). Developing an integrated health care model for
homeless and other vulnerable populations in Colorado. A report from the Colorado Coalition for the
Homeless. Retrieved from http://www.coloradocoalition.org/!userfiles/library/cch.sim.2013.pdf
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches
(3rd ed.). Thousand Oaks, CA: SAGE Publications.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.).
Thousand Oaks, CA: SAGE Publications.
Dahl, B., Fylkesnes, A., Sorlie, V., & Malterud, K. (2013). Lesbian women’s experiences with healthcare
providers in the birthing context: A meta-ethnography. Midwifery, 29(6), 674–681.
Green, J. B., & Lauber, A. (1972). Return of EEG activity after electrocerebral silence: Two case reports.
Journal of Neurology, Neurosurgery, and Psychiatry, 35(1), 103–107.
Greene, J. C., & Caracelli, V. J. (1997). Advances in mixed methods evaluation: The challenges and benefits
of integrating diverse paradigms. New Directions for Program Evaluation, no. 74. San Francisco,
CA: Jossey-Bass.
Guba, E. G. (1990). The paradigm dialog. London, UK: SAGE Publications.
Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In N. K. Denzin &
Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 105–117). Thousand Oaks, CA: SAGE
Publications.
Kukull, W. A., & Ganguli, M. (2012). Generalizability: The trees, the forest, and the low-hanging fruit.
Neurology, 78(23), 1886–1891. doi:10.1212/WNL.0b013e318258f812
Lanier, M. M., Ford, C. A., Reid, J. C., & Strickland, K. M. (2014). Advanced research methods for the
social sciences. San Diego, CA: Cognella Academic Publishing.
Leedy, P. D., & Ormrod, J. E. (2013). Practical research: Planning and design (10th ed.). Upper Saddle
River, NJ: Pearson Education.
Mack, L. (n.d.). The philosophical underpinnings of educational research. Retrieved from http://www.
apu.ac.jp/rcaps/uploads/fckeditor/publications/polyglossia/Polyglossia_V19_Lindsay.pdf
Malterud, K. (2001). Qualitative research: Standards, challenges and guidelines. Lancet, 358, 483–488.
Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. San Francisco, CA:
John Wiley & Sons.
Merriam-Webster. (2017). Medical Dictionary: Scientific method. Retrieved from https://www.
merriam-webster.com/dictionary/scientific%20method
Office for Human Research Protections. (2008, October 16). Guidance on engagement of institutions
in human subjects research. Retrieved from Department of Health and Human Services at http://
www.hhs.gov/ohrp/sites/default/files/ohrp/policy/cdebiol.pdf
Pottier, P., Hardouin, J., Dejoie, T., Bonnaud, A., Le Loupp, A., Planchon, B., & LeBlanc, V. (2011).
Stress responses in medical students in ambulatory and in-hospital patient consultations. Medical
Education, 45(7), 678–687. doi:10.1111/j.1365-2923.2011.03935.x

Sciences
Reichardt, C. S., & Rallis, S. F. (1994). The qualitative-quantitative debate: New perspectives. New Direc-
tions for Program Evaluation, no. 61. San Francisco, CA: Jossey-Bass.
Sackett, D. L., Rosenberg, W. M., Gray, J. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence
based medicine: What it is and what it isn’t. British Medical Journal, 312(7023), 71–72.
Samonte, P. V., & Vallente, R. P. (2016). Evidence-based practice (EBP). Salem Press Encyclopedia.
Silverman, D., Masland, R. L., Saunders, M. G., & Schwab, R. S. (1970). Irreversible coma associated
with electrocerebral silence. Neurology, 20(6), 525–533.
Solutions IRB. (2017). Why is scientific merit important? Retrieved from http://www.solutionsirb.com/
scientific-merit/
Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative
approaches. Applied Social Research Methods Series, Volume 46. Thousand Oaks, CA: SAGE
Publication, Inc.
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research. Thousand Oaks, CA:
SAGE Publications.
U.S. Department of Health and Human Services. (2014). 2014 Surgeon General’s report: The health
consequences of smoking—50 years of progress. Retrieved from https://www.cdc.gov/tobacco/data_
statistics/sgr/50th-anniversary/index.htm
Vangeli, E., & West, R. (2012). Transition towards a ‘non-smoker’ identity following smoking cessation:
An interpretative phenomenological analysis. British Journal of Health Psychology, 17(1), 171–184.
Whittemore, R., & D’Eramo Melkus, G. (2008). Designing a research study. Diabetes Educator, 34(2),
201–216. doi:10.1177/0145721708315678

Sciences
ETHICAL ISSUES WHEN
CONDUCTING RESEARCH
INTRODUCTION
2
CHAPTER GOALS:
This chapter discusses the role of ethics in research and the creation of
standards to ensure that research involving human participants is ethical.
What is ethics in research, and why is it important? In order to answer
this question, it’s important to provide a summary of some of the early • To provide working definitions of the
studies conducted under the guise of research, which then served as terms beneficence, justice, respect for
an impetus for the development of standards for conducting ethical persons, and informed consent.
research with human participants. Figure 2.1 illustrates the time line of • To explore the relationship
unethical research practices that led to current research standards in between early research practices
the United States. and the creation of research stan-
In terms of the research process, the highest level of ethical behavior dards in response to these early
is mandatory during each of the five stages of research. The primary research practices.
focus of this chapter will be to explain the policies and practices that a • To furnish students with an under-
researcher engages in to ensure that research participants (also known standing of the Institutional Review
as human subjects) are protected during the design and empirical stages Board process and its implications
of research (Table 2.1). for ethical research.
ETHICAL RESEARCH—DEFINED
Ethical behavior, as defined by Merriam-Webster’s (2017) dictionary, is LEARNING OBJECTIVES:
“conforming to accepted standards of conduct.” Research, as defined by
the Office for Human Research Protections (OHRP) in Chapter 1, is “a BY THE COMPLETION OF THIS CHAPTER,
systematic investigation, including research development, testing and THE STUDENT WILL BE ABLE TO:
evaluation, designed to develop or contribute to generalizable knowledge”
• Define the terms: assent, autonomous,
(OHRP, 2008, p. 2). Ethical research, therefore, is a systematic investi-
Belmont Report, beneficence, coercion,
gation designed to develop or contribute to generalizable knowledge that
ethical research, informed consent, Insti-
conforms to accepted standards of conduct.
tutional Review Board, justice, respect for
persons, undue influence, and vulnerable;
EARLY RESEARCH PRACTICES AND THE • Explain the process and elements of
CREATION OF RESEARCH STANDARDS informed consent;
• Identify ethical issues in clinical
The Tuskegee Study of Untreated Syphilis research trials that led to the imple-
In 1932, the Tuskegee Institute, in conjunction with the U.S. Public Health mentation of Institutional Review
Service, began the Tuskegee Study of Untreated Syphilis (also known Boards; and
as the Tuskegee Syphilis Study or Tuskegee Syphilis Experiment) to • Apply the information learned
regarding informed consent and the
Institutional Review Board to iden-
25 tify ethical issues in case studies.

Sciences
FIGURE 2.1 The time line of events leading to current research standards.
1974
1947 National 1979 1991
Nuremberg Research Belmont Common
Code Act Report Rule
1932 1939‐1945 1955 1962

Tuskegee study Nazi War Willowbrook Jewish Chronic
of untreated Crimes Hepatitis Study Disease Hospital
syphilis
document the natural disease progression of syphilis in black men. Six hundred black men
from Macon County, Alabama, most of whom were living in poverty, were enrolled in the
study. Of these 600 men, 399 had syphilis; 201 did not (Centers for Disease Control and
Prevention [CDC], 2013; Rothman, 1982).
Men who participated in the study were not told they were in an experiment; they were
told they were being treated for “bad blood,” which for some involved painful spinal taps.
Participants were given, “free medical exams and treatment,” meals, and, upon death, a
burial stipend was paid to their survivors. In 1945, penicillin was approved by the U.S. Public
Health Service (USPHS) to treat syphilis, but the Tuskegee Study continued without the
men being treated (CDC, 2013; Rothman, 1982).
Rothman (1982) explains that the study continued “through the 1960s, untouched by the
civil rights agitation, and unaffected by the code of research ethics adopted by the USPHS
itself. It ended only in 1972, when an account of the experiment in the Washington Star
sparked a furor” (p. 5).
Nazi War Crimes

Between 1939 and 1945, medical researchers in Nazi Germany conducted a wide range of
heinous and often fatal experiments on prisoners of concentration camps, without their
consent. Other prisoners were euthanized, solely because they were deemed “unworthy of
life” (Nazi Medical Experiments, n. d.).
TABLE 2.1 Stages of Research: Action Steps Highlighted in This Chapter
Stages Action Steps
Design (Planning) Selecting the best research design and research methods (sampling, data collection, data analysis) that
align with research question(s), submitting the research study proposal for IRB review
Empirical (Doing) Obtaining IRB approval to conduct the study, recruiting/selecting the sample, obtaining informed
consent from participants, collecting the data

Sciences
Chapter 2 Ethical Issues When Conducting Research | 27
In 1946, an American military court held criminal proceedings against 23 German

physicians for their participation in war crimes and crimes against humanity. These
proceedings, known as the Nuremberg Doctors’ Trial, lasted 140 days. Eighty-five wit-
nesses testified, and 1,471 documents were introduced. Although 16 of the physicians were
found guilty, they could not be found guilty of violating the rights of human participants
because, at that time, there were no accepted standards (Nazi Medical Experiments,
n. d.; Shuster, 1997).
The Nuremberg Code

The Nuremberg Code was framed by American judges sitting in judgment of the Nazi
doctors accused of committing heinous medical experiments in concentration camps. This
code combined Hippocratic ethics and the protection of human participants into a single
document and has been called the most important document in the history of ethics in
medical research. The code focuses on the human rights of research participants and is
comprised of ten standards that physician-investigators must adhere to when conducting
experiments on human participants. Included in this code was the requirement of voluntary
informed consent; the recognition that risks must be weighed against anticipated benefits;
and the caveat that experiments should only be conducted by scientifically qualified persons
(Fischer, 2006; Shuster, 1997).
Willowbrook Hepatitis Study

The Willowbrook Hepatitis Study was initiated in the mid-1950s at the Willowbrook
State School, an institution for cognitively disabled children on Staten Island, New York.
Hepatitis was widespread at the institution, and researchers were interested in studying
the natural course of viral hepatitis as well as the efficacy of gamma globulin (Krugman,
1986).
Although parents gave consent for their children to be inoculated with a mild form of
hepatitis, they were not fully informed of the possible hazards involved in the study. There
is evidence that indicates parents might have been under the impression that if they did
not give consent, their children would not be cared for. Researchers defended their actions
under the guise that because hepatitis was widespread at the institution, the children most
likely would have been infected with the disease within their first year at the institution
(Krugman, 1986; Rothman, 1982).
Jewish Chronic Disease Hospital Study

Another example of an egregious study that took place in an institution in which people were
to have been cared for is the Jewish Chronic Disease Hospital Study. In 1963 two doctors
injected live cancer cells into hospitalized patients with chronic diseases. The premise of
the study was to see if patients who were debilitated with a chronic disease rejected cancer
cells, albeit at a slower rate, than healthy patients (McNeil, 1993, p. 57).
It is important to note that this study was so egregious that it was brought to the attention
of the Board of Regents of the State University of New York who, upon review, found that
not only had the research protocol not been presented to the hospital’s review committee,
but the patients’ physicians were unaware of their patients’ involvement in this study. As a
result of their blatant disregard for the welfare of the patients who, unknowingly, had been

Sciences
injected with live cancer cells, the doctors were found guilty of “fraud, deceit and unpro-
fessional conduct in the practice of medicine” (Mulford, 1967, p. 99).
National Research Act

The development of the regulatory process governing the ethical conduct of researchers
began with the signing of the National Research Act into law in 1974, thereby creating the
National Commission for the Protection of Human Subjects of Biomedical and Behavioral
Research. The commission was charged with identifying the key components of ethical
research involving human participants and developing guidelines to ensure that human
research is conducted in accordance with those principles (OHRP, 1979).
Belmont Report
The Belmont Report was drafted in 1979 by the National Commission for the Protection of
Human Subjects of Biomedical and Behavioral Research. The Belmont Report is important
to the content of this chapter as it identified three important basic principles: respect for
persons, beneficence, and justice, to be followed in the ethical conduct of research on humans.
Respect for Persons

“Respect for persons incorporates at least two ethical convictions: first, individuals should
be treated as autonomous agents, and second, that persons with diminished autonomy are
entitled to protection” (OHRP, 1979, para. 3). According to the Belmont Report, Part B:
Basic Ethical Principles “An autonomous person is an individual capable of deliberation
and personal goals and of acting under the direction of such deliberation” (OHRP, 1979,
para. 3). Demonstrating respect for the decisions made by an autonomous person would
involve respecting their decisions and opinions, unless said decisions and opinions would be
harmful to others. Showing a lack of respect to an autonomous person could be manifested
in a number of ways. One could show a lack of respect by rejecting, or interfering with, a
person’s ability to carry out or act on their opinions and choices. Another example would be
by withholding information, for no compelling reason, for the purpose of interfering with an
individual’s ability to make a decision. Two hallmarks of an autonomous individual would
be that they have the ability to both understand and process information and, should they
choose to participate in a research study, they are free to do so without being coerced or
influenced by others. “In research involving human subjects, respect for persons implies
that, when given adequate information about the research project, that subjects voluntarily
decide to participate” (OHRP, 1979, para. 3).
Not every person has the capacity to act as an autonomous agent, whether due to illness,
a mental disability, or circumstances that severely restrict their freedom and therefore may
require protection while incapacitated. The term diminished autonomy is used to describe
an individual who is not able to act as an autonomous agent and therefore is not “… capable
of deliberation and personal goals and of acting under the direction of such deliberation”
(OHRP, 1979, para. 3). This diminished autonomy may be manifested as having limitations
when it comes to giving thoughtful consideration to or carrying out their personal goals.
A vulnerable population can be described as “the disadvantaged sub-segment of the
community …” (Shivayogi, 2013, p. 53). When a person has limitations on either their capacity
or voluntariness, they are considered vulnerable. “The vulnerable individuals’ freedom and

Sciences
capacity to protect one-self from intended or inherent risks is variably abbreviated, from
decreased freewill to inability to make informed choices” (Shivayogi, 2013, p. 53).
Examples of participants who lack capacity or are unable to make their own choices and
decisions are children, those with intellectual disability, prisoners, students in hierarchical
organizations, institutionalized individuals, the elderly, and individuals who are education-
ally and economically disadvantaged (Shivayogi, 2013).
Coercion
Under Part C, section 1 of the Belmont Report, coercion “occurs when an overt threat of
harm is intentionally presented by one person to another in order to obtain compliance”
(OHRP, 1979, para. 11). Consider the following example: an elderly woman who is a resident
in a nursing home is forced to choose between participating in a research study or leaving
the nursing home. The elderly woman lacks the ability to make a decision based on her own
free will. She is being forced to choose one of two options; participate in the research study
and stay in the nursing home or don’t participate in the research study and leave the nursing
home. The participant in this case is being threatened in order to obtain compliance, the
threat that she will not be able to stay in the nursing home. Her ability to make a decision
based on her own free will has been taken away.
Another example of coercion in research would be where a physician threatens to stop
providing care to his patient unless the patient joins a clinical trial. The physician is making
an overt threat of harm: “You can no longer be my patient” in order to coerce or force the
patient to participate in the study.
Undue Influence
In contrast, Part C, section 1 of the Belmont Report defines undue influence as influence
that “occurs through an offer of an excessive, unwarranted, inappropriate or improper
reward or other overture in order to obtain compliance” (OHRP, 1979, para. 10). Consider
the example of an investigator promising students in her psychology class that they will
receive extra credit if they participate in her research project. If students are presented with
only this one way to earn extra credit, then the investigator is unduly influencing potential
study participants. If, however, students who did not want to participate in the research
project were given non-research opportunities to earn extra credit, then the possibility of
undue influence is decreased.
Another example of undue influence would be if a researcher offered a large sum of money
(for instance, a month’s salary) to participants for taking part in a one-day study to test the
effects of a drug with potentially serious side effects that is under investigation. Because the
sum of money offered could persuade potential participants to engage in the study against
their better judgment, this offer could present undue influence.
Beneficence
The principle of beneficence requires that persons are treated in an ethical manner by
(1) protecting them from harm; and (2) maximizing possible benefits and minimizing
possible risks of harm. It is the obligation of researchers to maximize benefits for the
individual participant and/or society while minimizing the risk of harm to the individual
participant. This doesn’t mean that there are not any risks involved to participants. It means

Sciences
that thoughtful consideration has been given to both the possible benefits and harms, and
a decision is then made as to when it is justifiable to seek certain benefits in spite of the
risks involved and when the risks involved outweigh the potential benefits. Should the risks
outweigh the benefits, consideration should be given to determining if there is another
way to conduct a study in which the same knowledge could be obtained with lower risks
to participants (OHRP, 1979).
Justice
The principle of justice in Part B raises the question: “Who ought to receive the benefits of
research and bear its burdens?” (OHRP, 1979, para. 11). An injustice occurs when a person
who is entitled to a benefit is denied said benefit, without good reason, or when a burden is
unduly imposed (OHRP, 1979). The selection of research participants must be fair, avoid-
ing participants from a population (i.e., educationally or economically disadvantaged) or
selecting participants from a certain population only for the experiment group. Research
conducted in the United States in the early-to-mid-20th century illustrates the violation of
the principle of justice. For example, participants in the Tuskegee Study of Untreated Syph-
ilis were disadvantaged rural black men who were denied treatment, even though penicillin
was available to treat syphilis, so that the study could be continued.
COMMON RULE
Using the Belmont Report as an ethical guideline, currently what governs the protection
of human subjects in the United States is the Federal Policy for the Protection of Human
Subjects, also known as the Common Rule. Said another way, the Common Rule operation-
alized the principles in the Belmont report by setting the rules and procedures researchers
must follow when conducting human subject research. The rules and procedures include,
but are not limited to, developing an Institutional Review Board (IRB), setting standards
on what information must be included in the consent forms, and the level of review stud-
ies must go through. There is more information on this in later sections of the chapter.
This 1991 federal policy requires compliance across 15 different federal departments and
agencies. Each department/agency was required to develop a set of policies that complied
with this federal regulation (i.e., Department of Education, 34 CFR Part 97; Department of
Justice, 28 CFR Part 46; National Science Foundation, 45 CFR Part 690; etc.). The policies
for human subjects’ protection in health science research can be found under the Depart-
ment of Health and Human Services regulations, 45 CFR Part 46. This regulation has four
subparts which include: “subpart A, also known as the Federal Policy or the ‘Common
Rule’; subpart B, additional protections for pregnant women, human fetuses, and neonates;
subpart C, additional protections for prisoners; and subpart D, additional protections for
children” (OHRP, 2016, para 2).
Proposed updates to the Common Rule are being worked on; it is anticipated that these
changes will go into effect in 2019. Some of the proposed changes being discussed include
revisions to the consent forms that will help people be better informed when making decisions
about whether to participate in a research study as well as inclusion of initiatives to help
streamline the review processes for low-risk research studies and the paperwork required
annually to renew low-risk research studies (OHRP, 2018).

Sciences
INSTITUTIONAL REVIEW BOARDS

The Common Rule mandates that an Institutional Review Board (IRB) be formed to
review research studies. The IRB is a group that has been assembled for the primary
purpose of reviewing research proposals to ensure that the rights and welfare of human
subjects participating in research studies are protected. The OHRP (2008) human sub-
jects regulations (45 CFR 46) define human subjects as “a living individual about whom
an investigator (whether professional or student) conducting research obtains (1) data
through intervention or interaction with the individual, or (2) identifiable private infor-
mation” (para 6).
Private information includes information about behavior that occurs in a context in which an
individual can reasonably expect that no observation or recording is taking place, and the individual
can also reasonably expect that information that has been provided for specific purposes will not
be made public (e.g., a medical record). Private information must be individually identifiable (i.e.,
the identity of the subject is or may readily be ascertained by the investigator or associated with
the information) in order for obtaining the information to constitute research involving human
subjects (OHRP, 2008, para. 7).
Research involving human subjects funded or regulated by the federal government is

required to be reviewed and approved by an IRB. Requirements for human research are the
same for most research institutions, professional organizations, and scholarly journals. Each
institution is responsible for establishing their own IRB committee and selecting members
to serve on the committee (Sherchand, 2017). Even though federal regulations refer to
IRBs, one may find that an institution chose to assign a different name to a committee that
performs this function.
Institutional Review Boards review research proposals to ensure there is no harm to
participants, that appropriate procedures will be followed to obtain informed consent, and
that the privacy and anonymity of participants will be respected. They have the right to
ask that modifications be made to a proposal, to monitor the manner in which the research
study is conducted and, if necessary, “… suspend or stop any health-related research that
violates any ethical issues” (Sherchand, 2017, p. 1). Tasks carried out by the IRB committee
should be implemented without bias and influence from the government, political parties,
professional organizations and bodies, etc. (Sherchand, 2017). It is the responsibility of the
IRB and institution to “… ensure that the conduct of all health-related research approved
by the IRB be monitored and supervised by procedures and/or by using existing appropriate
mechanisms within the institution” (Sherchand, 2017, p. 2).
Prior to the Belmont Report, as in the case of the Tuskegee Study of Untreated Syph-
ilis, numerous scientific journal articles on the experiments’ findings were published in
well-respected medical journals. Since the adoption of the Belmont Report, it is an agreed-
upon standard in reputable journals to only publish human participants’ research that has
received IRB approval.
Fischer (2006) explains that a physician named Dr. Beecher was instrumental in raising
awareness regarding ethical research practices. Dr. Beecher wrote the seminal paper on the
ethics of research in 1966, exposing research studies that put the generation of new knowledge
above the ethical treatment of human subjects. His passionate lecturing and advocacy for

Sciences
the protection of human subjects was a motivating factor in changing publication standards.
Fischer (2006, p. 71) wrote that
Beecher concludes by saying that, just as U.S. courts reject evidence obtained unconstitutionally—
even if it is useful in the pursuit of justice—journal editors should reject papers with information
obtained unethically. The idea that medical journal editors are a final common pathway in the
evaluation of a study’s ethics was later adopted by the International Committee of Medical Journal
Editors—known as the Vancouver Group.
Subpart A of 45 CFR 46 sets the standards for the IRB, including the board composi-
tion, type of review required for a proposed research project, and required elements in
an informed consent (OHRP, 2009). The following sections on the IRB are cited from
this source.
There are three levels of review: The research project may be exempt from review; have
an expedited review; or be brought before the board for a full review.
Exempt from Review

The Common Rule allows research activities to be exempt from IRB full review when they
are considered low risk and the involvement of human participants is within one of the
categories defined by the Department of Health and Human Services. Two examples that
fall under this category are research conducted in commonly accepted educational settings
and research that uses educational tests. Data used in these studies must be collected and
presented in a manner in which research participants cannot be identified. While consent
is always required, often in this type of research a signed consent form is not required (i.e.,
a statement on an anonymous survey that explains completing the survey is considered
giving consent). Examples of research activities that are exempt from IRB approval include
surveying educators regarding the use of a new curriculum, evaluating the use of a revised
standardized test, and analyzing data from an existing database which are recorded without
identifiers (OHRP, 2009).
Expedited Review
Additionally, the Common Rule allows, “research activities that (1) present no more
than minimal risk to human subjects, and (2) involve only procedures listed in one or
more of the following categories, may be reviewed by the IRB through the expedited
review procedure…” (OHRP, 1998b, para. 1). Research activities in this category include
the collection of human data (i.e., height, weight), imaging tests (i.e., electrocardio-
grams and magnetic resonance imaging), and blood and bodily fluids (OHRP, 2009).
The review takes place with the staff of the IRB and typically a few experts but not the
full IRB board.
Full Review
Research on participants or any protected participant population (i.e., fetuses, pregnant
women, prisoners, children, the elderly, and psychiatric patients) that involves more than
minimal risk needs to be brought before the IRB board for a full review.

Sciences
COMPOSITION OF THE INSTITUTIONAL REVIEW BOARD

According to Section 46.107 of the Code of Federal Regulations, the composition of the
IRB should be at least five members whose backgrounds are varied enough that they can
completely and adequately review proposals put forth by the institution. Consideration
should be given to the diversity of members with regard to race, gender, and cultural and
professional backgrounds, as well as any other issues that would be relevant to the research
interests of the institution. In addition, the IRB committee shall include at least one member
who is primarily involved in a scientific area, one who is not primarily involved in a scien-
tific area, and one who is not affiliated with the institution. If the IRB regulations are not
followed, consequences could include, but are not limited to: suspension or termination of
the research project; inability to use data or publish results; inability to receive federal grant
funding; additional monitoring and oversight by the IRB and/or a third party; termination
of employment; and termination of all research at the institution (OHRP, 2009).
RESEARCH WITH HUMAN BEINGS REQUIRES INFORMED CONSENT

Informed consent is a process that includes giving all the information to a potential research
participant in a way they can understand so they are able to make an informed decision on
whether to volunteer for the study. In other words, “… relevant information is provided to
a person who is competent to make a decision, and who is situated to do so voluntarily”
(Appelbaum, Lidz, & Klitzman, 2009, p. 30).
The informed consent process is comprised of three elements: information, compre-
hension, and voluntariness. It is imperative that during the informed consent process the
researcher put the participant’s rights, welfare, and safety above all other concerns, whether
they be personal or scientific (OHRP, 1979).
Information
It is vitally important that potential research participants be given sufficient information
in order to ascertain whether or not they want to participate in the research process. This
information may include, but not be limited to: a description of the purpose of the research
procedure as well as the procedures involved; potential risks and anticipated benefits; any
available alternative procedures (when therapy is involved); and a statement informing the
potential research participant of their ability to ask questions as well as withdraw from the
research study at any time (OHRP, 1979).
Comprehension
Many factors may impact a participant’s ability to comprehend the information presented
as part of the informed consent process, and it is necessary to adapt the presentation of the
material to the participant’s capacities. If language is a barrier, it is important to present
information in a language that is understandable to the participant or his or her represen-
tation. If English is not the participant’s primary language, consideration should be given
to providing non-English-speaking participants a translated informed consent document
(OHRP, 1979).
Should comprehension be severely limited due to immaturity or mental disability, it is
important to seek the permission of a third party in order to protect the participant from

Sciences
harm. The individual selected to be the third party should be someone who understands
the participant’s situation and will act in their best interest (OHRP, 1979).
As part of the informed consent process, it is important to provide an opportunity
for individuals to ask questions about any of the information presented. In addition, it is
important to consider what, if any barriers may be present that might prevent an individ-
ual from asking questions. For instance, in some cultures, it may be considered rude to
ask questions of the investigator/researcher presenting the information, which may result
in the participant not fully understanding the information presented. In these situations,
the question of who is presenting the information to the participant, how it is explained,
and establishing an atmosphere in which the participant is comfortable asking questions
becomes extremely important (OHRP, 1979).
Voluntariness
An agreement to participate in research is considered a valid consent only if it is given
voluntarily. Once the individual has received and comprehended the required information,
it is important to give sufficient time for the individual to think about the research before
giving consent to participate in the study. This element of the informed consent process
requires that the conditions surrounding the consent are free of coercion (i.e., inappropri-
ate financial or other rewards) and/or undue influence (refer to the section above on undue
influence for a review of this information) (OHRP, 1979).
Appelbaum, Lidz, and Klitzman (2009) use the term “potentially impaired voluntari-
ness” (p. 31) when describing situations that may preclude an individual from being able to
voluntarily give a valid consent. For instance, a substantial amount of money or compen-
sation offered in exchange for participating in a research study may potentially impair an
individual’s ability to give thoughtful consideration as to whether they want to voluntarily
give consent for the study. The same could be said for patients who are ill with a particular
medical condition and do not have access to health care. Should this patient be presented
with an opportunity to participate in a research study that will provide a possible treatment
for their condition, this opportunity may potentially impair their ability to give a voluntary
valid consent.
Informed Consent Checklist

Below are the basic elements of the informed consent process from section 46.116, Office
for Human Research Protections. An Institutional Review Board in a particular setting (i.e.,
research university) may require additional elements in the informed consent process, but
the following eight elements listed below are required.
1. A statement that the study involves research, an explanation of the purposes of
the research and the expected duration of the subject’s participation, a descrip-
tion of the procedures to be followed, and identification of any procedures which
are experimental;
2. A description of any reasonably foreseeable risks or discomforts to the subject;
3. A description of any benefits to the subject or to others which may reasonably be
expected from the research;

Sciences
4. A disclosure of appropriate alternative procedures or courses of treatment, if any,

that might be advantageous to the subject;
5. A statement describing the extent, if any, to which confidentiality of records identi-
fying the subject will be maintained;
6. For research involving more than minimal risk, an explanation as to whether any
compensation and an explanation as to whether any medical treatments are avail-
able if injury occurs and, if so, what they consist of, or where further information
may be obtained;
7. An explanation of whom to contact for answers to pertinent questions about the
research and research subjects’ rights, and whom to contact in the event of a
research-related injury to the subject; and
8. A statement that participation is voluntary, refusal to participate will involve no
penalty or loss of benefits to which the subject is otherwise entitled, and the subject
may discontinue participation at any time without penalty or loss of benefits to which
the subject is otherwise entitled. (OHRP, 1998a)
Research Involving Children

According to 45 CFR Part 46, subpart D, “Children are persons who have not attained the
legal age for consent to treatments or procedures involved in the research, under the appli-
cable law of the jurisdiction in which the research will be conducted” (OHRP, 2009, subpart
D, para. 2). Yet, in consideration of the evolving maturity and independence of a child and,
consistent with federal regulations, investigators should engage children, when appropriate,
in discussion about research and their assent (Field & Behrman, 2004). “Assent means a
child’s affirmative agreement to participate in research. Mere failure to object should not,
absent affirmative agreement, be construed as assent” (OHRP, 2009, subpart D, para. 2).
This means that although the legal guardian is the one giving informed consent for the
child to participate in the research study, the researcher has explained the study to the
child, the child has agreed to take part in the study, and the researcher has documented
they have the child’s assent. When determining if a child is capable of providing assent,
their age, maturity, and psychological state are important factors to consider. Permission
from parent(s) or guardian(s) must also be obtained prior to enrolling a child in research.
If a child does not assent to participate in research, even if the parents or legal guardian
grant permission, the child’s decision prevails.
For example, a researcher wants to study the impact of certain television programs on the
decision-making capacity of adolescents. In this case, the assent form should be simple enough
for the adolescent to understand what he or she is agreeing to do and explain the following; that
they are being asked to participate in the study, the purpose of the study, an estimation of how
much time would be involved, what their involvement will be if they agree to participate, conceiv-
able risks/discomforts and benefits, that they have the ability to ask their parents/guardians or the
researcher any questions they have, that it is their decision as to whether they participate in the
research study and, if they do, that they can stop at any time and that their parent/guardian is aware
that they are being asked to participate in the project. The adolescent will then be asked to assent
to participate in the study and sign their name (Rochester Institute of Technology, n.d., para. 1–3).

Sciences

This chapter introduced the history of early research practices, which led to the devel-
opment of the regulatory process in the United States. This began with the signing of
the National Research Act in 1974, followed by the creation of the Belmont Report. The
Belmont Report identified three important ethical principles to be considered when con-
ducting research on human participants: respect for persons, beneficence, and justice. An
important concept included in the discussion on respect for persons was that individuals
should be treated as autonomous agents and that individuals with diminished autonomy
were to be protected from undue influence and coercion. The Common Rule required
the establishment of Institutional Review Boards (IRBs) to help protect the rights and
welfare of human research participants by developing a process to review and approve
research proposals involving human participants. After review of a research proposal,
the committee makes a decision as to whether the process may be exempt from review,
have an expedited review, or be brought before the IRB committee for a full review.
Research involving human beings requires informed consent, and included in this chap-
ter are the basic elements of the informed consent process, the role that information,
comprehension, and voluntariness plays in the process, and the role of assent in research
involving children.
Research on human beings is essential to the advancement of scientific and medical
knowledge. History has taught us the importance of respecting the rights of human partic-
ipants in research. The importance of informed consent of research for human participants
(with assent for young children) and peer review of research proposals by an IRB before
research can begin must never be taken for granted.
STUDY PRACTICE
At this point, you should be able to apply the principles of the Belmont Report to the early
research practices presented at the beginning of this chapter to determine what ethical
principles were breached. You should also recognize the role of the IRB in the practice of
research as well as the components of the informed consent process.
Study tips for all VARK styles: Since the reducing notes from 3:1 recommendation cuts
across all learning styles, the first suggestion for studying is to start making flash cards
for every bolded/italicized term. The important terms/concepts presented in this chapter
include:
ɠɠ Ethical research ɠɠ full review
ɠɠ Tuskegee Study of Untreated Syphilis ɠɠ Composition of the IRB board
ɠɠ Nuremberg Code ɠɠ Belmont Report
ɠɠ Willowbrook Hepatitis Study ɠɠ Common Rule
ɠɠ Jewish Chronic Disease Hospital Study ɠɠ Respect for persons
ɠɠ National Research Act ɝɝ autonomous person
ɠɠ Institutional Review Board ɝɝ diminished autonomy
ɝɝ private information ɓɓ vulnerable population
ɠɠ exempt from review ɐɐ coercion
ɠɠ expedited review ɐɐ undue influence

Sciences
ɠɠ Beneficence ɝɝ Comprehension
ɠɠ Justice ɝɝ Voluntariness
ɠɠ Informed consent ɠɠ Assent
ɝɝ Information

1. An investigator planning to study behavioral changes during alcohol intoxication
will pay participants $600 for six hours of testing that includes drinking a moderate
level of alcohol and completing several written questionnaires. He plans to recruit
people age 21 years and older (college students taking his course and people who
are homeless in the local area).
Considering this scenario, which of the following is the most important issue that
the researcher needs to address before submitting the protocol to the university’s
Institutional Review Board?
a. The reading level comprehension of the participants
b. How to advertise this study to recruit participants
c. The monetary compensation may be considered undue influence
d. The most effective way to pay participants who are homeless
2. Which of the following is acceptable when conducting research involving
human participants?
a. Saliva and blood samples are taken from a group of high school students compar-
ing DNA and giftedness. Volunteers gave proper informed consent for the study.
b. A group of people in a cancer support group are asked to participate in a study
in which painful tissue biopsies (10 times over a two-year time period) must be
collected to serve as a control group in a national cancer study. Participation
in the study is voluntary, and the participants give proper informed consent.
c. A professor makes an announcement in a psychology class that student volun-
teers are being sought for a behavioral psychology study. The class is informed
that extra credit will be given if the students participate. Students who don’t
participate can receive extra credit by doing an alternative assignment.
d. All are acceptable.
3. An investigator is studying a new drug. There might be some foreseeable risks to
taking the new drug, but he makes this clearly known to all participants he consents.
He recruits participants into the control group (no drug) from the local college. He
recruits participants from homeless shelters into the experimental group (drug)
because he knows if the drug works that group will benefit most.
Which of the following ethical principles did this study violate?
a. Justice
b. No ethical violation
c. Beneficence
d. Respect for persons

Sciences
4. A surgeon keeps a patient under anesthesia an extra 3 minutes to collect tissue
samples after clinically required surgery. She only keeps demographic information
such as age, race, and gender and removes all other identifying information from
her data collection. Since there is no identifying information linked to the samples,
she does not consent the participants.
This is an example of:
a. Lack of justice
b. Lack of respect for autonomy
c. Acceptable research procedures
d. Coercion of participants
5. An individual with diminished autonomy is entitled to protection. This state-
ment demonstrates the principle of:
a. Respect for persons
b. Beneficence
c. Justice
d. Voluntariness
6. Which of the following need(s) to be obtained when doing research
with children?
a. Informed consent from the parent or guardian
b. Assent from the child if he or she is capable
c. Informed consent from the child
d. both a and b
7. Which of the following is not acceptable when conducting research with humans?
a. Having the participant give informed consent
b. Telling participants they must remain in the study until it is completed
c. Keeping the identity of the participants confidential
d. Informing participants they are free to withdraw from the study at any time
8. IRB is an acronym for:
a. Internal Review Board
b. Institutional Review Board
c. Internal Revision Board
d. Institutional Responsibility Board
9. IRB members are responsible for all of the following EXCEPT:
a. determining if research findings are generalizable
b. assessing whether participants are giving informed consent
c. determining the risks and benefits to participants
d. reviewing the manner in which data will be secured
10. A researcher obtains IRB approval to conduct a research study:
a. before a plan is developed
b. before data is collected

Sciences
c. after data is collected

d. after data is analyzed
STUDY ACTIVITIES
You now know enough about ethical terms, concepts, and practices to be able to read a case
study and identify the relevant ethical issues.
Revise your notes/flash cards with this new information.
V: add this information to your diagrams and flow charts.

A: make sure you read out loud to yourself when studying and writing flash cards.
R: turn the charts in the chapter into paragraphs.
K: take your revised flash cards and on the floor or a large table, make outlines with the
flash card … see how things are related to one another.
Without using your notes, carefully read both of the case studies below and identify as
many ethical issues as you can find. While reading each case study, use the study technique(s)
you identified that best work for you (e.g., highlighting, underlining, making notes) to make
the important information in each case study stand out.
CASE STUDY 1
“Dan Markingson was acutely psychotic” (Lemmens & Richards, 2014, p. 11) when psychi-
atrists at the University of Minnesota enrolled him in a controversial clinical trial which
involved the study of three new neuroleptic drugs. Dan had previously, on multiple occasions,
been found to be incompetent with regard to making decisions regarding his treatment,
the most recent of which was two days before enrolling in the trial. After making delu-
sional threats to his mother, he was involuntarily committed to a psychiatric ward. Yet,
he was found to be competent to consent to participate in the trial. Shortly after the trial
began, he was moved to a halfway house. His mother was increasingly concerned about
his deteriorating condition; she attempted to remove her son from the study and warned
the research team that she felt her son was in danger of killing himself. Unfortunately, her
warnings were ignored, and in May 2004, he violently killed himself. Adapted from Stone
(2012); Lemmens and Richards (2014).
What is/are the ethical issue(s) in this case study?
CASE STUDY 2
Fifteen hundred infants born between 25 and 28 weeks of gestation were enrolled in a study
designed to compare ranges of oxygen saturation. An external source of oxygen is given
to premature infants, as their lungs are often not fully developed and therefore not able
to nourish the brain as well as other organs. It must be noted that severe eye damage and
blindness can result from the administration of too much oxygen, while the administration
of too little oxygen can result in brain damage and death. Parents were required to sign a

Sciences
consent form prior to their infants participating in this IRB approved study. The infants
were randomly divided into two groups; one group received oxygen at the high end of the
target range, while the other group received oxygen at the lower end of the target range. The
consent form did not state the risk of death may be directly related to whether the amount
of oxygen the infant received was at the lower or higher end of the target range. Adapted
from Waldemar et al. (2010).
REFERENCES
Appelbaum, P. S., Lidz, C. W., & Klitzman, R. (2009). Voluntariness of consent to research: A con-
ceptual model. Hastings Center Report, 39(1), 30–39. doi:10.1353/hcr.0.0103
Centers for Disease Control and Prevention. (2013, December). The Tuskegee timeline. In U.S. Public
Health Service Syphilis Study at Tuskegee. Retrieved from https://www.cdc.gov/tuskegee/time-
line.htm
Field, M. J., & Behrman, R. E. (2004). Ethical conduct of clinical research involving children. Institute of
Medicine (US) Committee on Clinical Research Involving Children. Washington, DC: National
Academies Press (US). doi:10.17226/10958
Fischer, B. A. (2006). A summary of important documents in the field of research ethics. Schizophrenia
Bulletin, 32(1), 69–80. doi:10.1093/schbul/sbj005
Krugman, S. (1986). The Willowbrook hepatitis studies revisited: Ethical aspects. Clinical Infectious
Diseases, 8(1), 157–162.
Lemmens, T., & Richards, B. J. (2014). Investigating research and accessing reproductive material.
Bioethical Inquiry, 11(1), 11–19.
Merriam-Webster. (2017). Ethical: Ethical behavior. Retrieved from https://www.merriam-webster.
com/dictionary/ethical
McNeil, P. M. (1993). The ethics and politics of human experimentation. New York, NY: Cambridge
University Press.
Mulford, R. D. (1967). Experimentation on human beings. Stanford Law Review, 20(1), 99–100.
Nazi Medical Experiments. (n.d.). Holocaust Encyclopedia. Retrieved from https://www.ushmm.org/
wlc/en/article.php?ModuleId=10005168
Office for Human Research Protections. (1979, April 18). The Belmont Report: Ethical principles and
guidelines for the protection of human subjects of research. Retrieved from https://www.hhs.gov/ohrp/
regulations-and-policy/belmont-report/index.html
Office for Human Research Protections. (1998a). Informed consent checklist. Retrieved from https://
www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html#46.116
Office for Human Research Protections. (1998b). OHRP expedited review categories. Retrieved from
https://www.hhs.gov/ohrp/regulations-and-policy/guidance/categories-of-research-expedit-
ed-review-procedure-1998/index.html
Office for Human Research Protections. (2008). Engagement of institutions in human research.
Retrieved from https://www.hhs.gov/ohrp/regulations-and-policy/guidance/guidance-on-en-
gagement-of-institutions/index.html
Office for Human Research Protections. (2009). Basic HHS policy for protection of human research
subjects. Retrieved from https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/
index.html#46.402

Sciences
Office for Human Research Protections. (2016, March 18). Federal policy for the protection of human
subjects (“Common Rule”). Retrieved from https://www.hhs.gov/ohrp/regulations-and-policy/
regulations/common-rule/index.html
Office for Human Research Protections. (2018, June 18). HHS and 16 Other Federal Departments
and Agencies Issue a Final Rule to Delay for an Additional 6 Months the General Compliance Date of
Revisions to the Common Rule While Allowing the Use of Three Burden-Reducing Provisions during
the Delay Period. Retrieved from https://www.hhs.gov/ohrp/final-rule-delaying-general-complia
nce-revised-common-rule.html
Rochester Institute of Technology. (n.d.). Ethical and safe research: Assent tips. Retrieved from https://
www.rit.edu/research/hsro/assent_tips
Rothman, D. J. (1982). Were Tuskegee & Willowbrook “studies in nature”? Hastings Center Report,
12(2), 5–7. doi:10.2307/3561798
Sherchand, J. B. (2017). Mission of institutional review board/ethical review committee. Journal of
Institute of Medicine, 39(1), 1–2.
Shivayogi, P. (2013). Vulnerable population and methods for their safeguard. Perspectives in Clinical
Research, 4(1), 53–57.
Shuster, E. (1997). Fifty years later: The significance of the Nuremberg Code. New England Journal
of Medicine, 337(20), 1436–1440. doi:10.1056/NEJM199711133372006
Stone, J. (2012, December 11). A clinical trial and suicide leave many questions: Part 1: Consent? [Blog post].
Retrieved from https://blogs.scientificamerican.com/molecules-to-medicine/a-clinical-trial-and-suicide-leave-
many-questions-part-1-consent/
Waldemar, A. C., Finer, N. N., Walsh, M. C., Rich, W., Gantz, M. G., Laptook, A. R. … Higgins, R. D.
(2010). Target ranges of oxygen saturation in extremely preterm infants. New England Journal of
Medicine, 362(21), 1959–1969. doi:10.1056/NEJMoa0911781

Sciences
UNDERSTANDING THE ROLE OF
LITERATURE IN THE RESEARCH
PROCESS: REVIEWING,
WRITING, AND CRITIQUING CHAPTER GOALS:
3
THE SCHOLARLY ARTICLE THE GOALS OF THIS CHAPTER ARE:
• To provide an understanding of the
role the literature review plays in the
research process.
INTRODUCTION • To introduce how systematically
reviewing the existing literature is
Literature plays an integral part of any research. The term research was
used as a step to determine whether
defined in Chapter 1 by the OHRP as “a systematic investigation, includ-
an idea is indeed researchable.
ing research development, testing and evaluation, designed to develop or
• To provide a working definition of the
contribute to generalizable knowledge” (2008, p. 2). All research begins
terms systematic review of the literature,
and ends with literature. The systematic process used to generate new
literature review, annotated bibliography,
knowledge starts with a comprehensive and systematic investigation into
peer-reviewed, and reference mining.
knowledge that has already been established and written about. This
process is referred to as a systematic review of the literature. The results • To provide the tools neces-
of that investigation set the groundwork for what the researchers wish to sary to critique and evaluate
study. These results are compiled into a literature review, a subset of the the literature review section of
findings that pertain directly to what is known about a topic leading to peer-reviewed articles.
what is unknown (problem statement). This literature review justifies • To provide a template and detailed
the need for the research during the IRB process. In addition, once the explanation of each component of a
study has been conducted and concluded, the same review becomes the written article critique.
introduction of the research article when the findings are written up
and disseminated, therefore becoming part of the body of knowledge
LEARNING OBJECTIVES:
on a topic.
The action steps of reviewing existing knowledge and writing BY THE COMPLETION OF THIS CHAPTER,
new research findings for publication span two research stages that THE STUDENT WILL BE ABLE TO:
were presented in Chapter 1, the conceptual and the dissemination • Define at least 12 terms related to the
stages. This chapter will have three important interrelated sections. systematic review of the literature,
It will provide: the literature review, and compo-
nents of a scholarly article;
• an overview of the steps involved with conducting a system- • Identify the many roles literature
atic review of the literature during the conceptual stage of a plays in the research process;
research study (Table 3.1); • Compare a peer-reviewed jour-
• important information regarding how a researcher organizes nal, a professional journal, and a
and writes a literature review for an IRB during the design popular journal;
stage and for a journal article during the dissemination stage • Differentiate between a quantitative
of a research study (Table 3.1); and research article and a qualitative
research article; and
• Demonstrate the ability to critique a
43 research article.

Sciences
• a review of the format and style of quantitative and qualitative research articles
and guidelines used to critique peer-reviewed journal articles.
Stages Action Steps
Conceptual Having an idea (research problem), systematically reviewing the literature to verify the problem has
(Thinking) the potential to generate new knowledge, writing a problem statement, a research purpose statement
and research question(s)
Design Selecting the best research design and research methods (sampling, data collection, data analysis) that
(Planning) align with research question(s), submitting the research study proposal for IRB review
Dissemination Writing a journal article to share results/finding (new knowledge) with the scientific community
(Sharing)
While the term literature review has numerous meanings, for purposes of clarification it
will be referred to in one of two ways, depending on where it is being implemented within
the research process: the systematic review of the literature or the literature review.
SYSTEMATIC REVIEW OF THE LITERATURE—DEFINED

We must look to the past in order to determine the future. A review of the literature is an
essential feature of the research process that does just that—look to the past. There are 14
different types of literature reviews that look into the past. For example, a narrative litera-
ture review is the type that is often conducted when a student is writing a paper for a class.
Grant and Booth (2009) state that this type of review results in a narrative overview of
the literature, which may or may not be exhaustive, and when this type of literature review
is written the information is often presented in a way that is “chronological, conceptual,
thematic” (p. 95).
This chapter will highlight the one type of literature review that is most often used in
health science research, a systematic review of the literature. Conducting a systematic
review of the literature is the first step a researcher takes after having an idea during the
research study’s conceptualization stage. It involves a methodical, systematic, and exhaustive
evaluation of past research studies in order to identify whether the idea has the potential
to generate new knowledge.
There are two standards that researchers use when conducting a systematic review of
the literature:
• Cochrane Handbook for Systematic Reviews of Interventions

• Centre for Reviews and Dissemination: Guidance for Undertaking Reviews in
Health Care (Grant & Booth, 2009).

Sciences
Chapter 3 Understanding the Role of Literature in the Research Process: Reviewing, Writing, and Critiquing the Scholarly Article | 45
It is beyond the scope of this textbook to detail all the steps involved in a system-
atic review; rather, this chapter will only present highlights of the systematic review
process.
Simply said, a systematic review of the literature
• creates a “firm foundation for advancing knowledge;

• facilitates theory development;
• closes areas where a plethora of research exists; and
• uncovers areas where research is needed” (Webster & Watson, 2002, p. xiii).
A systematic review of the literature is an exhaustive review process involving a multi-

phase and multistage search of all that has already been written on the topic of the proposed
research study. It is the foundation of the research problem statement, which explicitly
identifies the relationship between what has already been studied and the generation of new
knowledge by providing a theoretical framework and rationale for future research studies
(Mongan-Rallis, 2014).
SIDE BOX 3.1 Often in health science the terms literature review or review of the literature are used
interchangeably. It is a shorthand term to reference the systematic process that reviews the previous liter-
ature to identify a problem statement, and it is a term that describes the subset of literature gleaned from
the systematic review that is used in the research proposal and in the introduction section of a journal arti-
cle. Confusing, right!
In this text, we will separate the two terms. We will use the term systematic review of the literature when
we refer to the process of delving into previously written work conducted during the conceptualization
stage of research.
We will use the term literature review when we are referring to the information that is found in the IRB
research proposal and in the introduction section of a journal article.
CONDUCTING THE SYSTEMATIC REVIEW OF THE LITERATURE

This section will explore the purpose of a systematic review of the literature, the differ-
ences between source materials and publication venues, and provide highlights of the steps
involved in a systematic review of the literature.
What Is the Primary Purpose of Conducting a Systematic Review of the Literature?

The purpose of a systematic review of the literature is to alert the researcher to gaps in the
literature that can lead to new areas of research. What does the term gap in the literature
actually mean? A gap in the literature is an area identified during a systematic inquiry into a
chosen topic where the researcher finds a question or problem that has not been addressed
adequately or at all in previous studies. It can also refer to a research topic that has been
previously studied but used a different methodology, population, or method of data col-
lection than the proposed study. Lastly, it identifies a question or idea that can be further

Sciences
developed. Identifying a gap in the literature shows a deep understanding of the body of
knowledge within a field of study (Webster & Watson, 2002).
By locating and reviewing what has already been written about a topic, the researcher can:
• relate work to the larger body of literature that already exists;

• uncover new ideas, perspectives, or approaches that were not considered before
but could be utilized to strengthen a current argument; and
• validate the need for further studies addressing the subject.
As stated earlier, reviewing previously conducted research is an important step in the

conceptualization stage of the research process. It is important to gather as much infor-
mation from previous works as possible on the subject. Once the systematic review of the
literature identifies a gap in the literature, the next step is to establish a purpose statement
and research question(s).
The problem statement, purpose statement, and research question help the reader concep-
tualize a study. They outline, define, and direct the construction of the research. A problem
statement, purpose statement, and research question that contains scientific merit work
toward guaranteeing that the study will generate new knowledge within an area of interest.
More detail on this topic will be found in Chapter 4.
A problem statement is derived from previous studies that have been conducted. Weaving
together findings from previous research helps to identify areas that need further investi-
gation and justify the need for a proposed study to take place. This is an important step in
the research process. The purpose of research is to generate new knowledge by filling a gap
in the current literature or informing practice. Once this is established, a researcher must
clearly state what the current study will accomplish in relation to the problem (purpose
statement) and create questions that will further direct the investigation (research questions).
How Does One Identify the Value of Sources?

There are a myriad of sources in the literature that a health science researcher can turn
to when identifying relevant literature on a topic. All sources, however, do not carry the
same value in research. There are three types of sources: primary, secondary, and tertiary.
• Primary sources: articles written to describe original research. Primary sources

appear in scholarly journals as original research (University of Minnesota, n. d.).
■■ For example, the researcher disseminates the results/findings of their
research through a firsthand account of the study they have conducted.
This lends a deep level of credibility to the information contained in the
article as it is written by the researcher.
■■ Another example is dissertations. This often involves a doctoral student
conducting original research guided by a faculty member, but these stud-
ies have not been published in a peer-reviewed journal and as such have
not undergone a peer-review process.
• Secondary sources: articles written about research studies which are not written
by the actual researcher.

Sciences
■■ Reviews and articles published in scholarly and professional journals that

summarize or evaluate research are still considered academic but are not
as highly valued as primary sources, simply because they are open to inter-
pretation and generalization by the author that may or may not remain in
context with the original research (University of Minnesota, n. d.).
■■ For example, a review of the literature article published in a peer-reviewed
journal presents detailed overview and analysis of the results of a system-
atic review of the literature. The primary goal of the article is to share a
unique perspective on a topic. This type of article is considered a second-
ary source in that it provides a compilation of past research but does not
generate new knowledge.
• Tertiary sources: carry the least credibility of all available sources. This is not to
say that tertiary sources are not factual and true; credibility is based on how far
removed from the original research the information being presented has come
and the expertise of the author presenting the information. They often sum-
marize or provide an overview of a topic but do not delve deeply into previous
research (University of Minnesota, n. d.).
■■ One example of a tertiary source is an encyclopedia or a textbook where
the information is presented in summary written by a third party.
■■ Note: Popular journals and web sources that end in .com carry the least
weight with regard to credibility of tertiary sources and come in at the
bottom when determining source reliability
Figure 3.1 illustrates the hierarchy of value within different sources of information. Pri-
mary sources are information that appears in scholarly or peer-reviewed journals as original
research. This type of source carries the most credence when a researcher is reviewing past
literature. Then articles from professional journals or textbooks coming in second, and gov-
ernment or popular sources are ranked
closer to the bottom. For the most part,
a researcher can determine the merit of FIGURE 3.1 An illustration of the hierarchy of relevant sources.
a source simply by identifying where it
is published. Peer-reviewed journals
Scholarly journals, also referred to as Research studies
academic or peer-reviewed, publish liter- Case studies
ature written by people who are experts Literature reviews
in their field and who have conducted Dissertations original research that is not
extensive research on a topic. Scholarly published in the peer-reviewed literature
literature is the most scrutinized of all Textbooks
the literature published in journals. The Professional journals
process to have an article published in Governmental website .gov
a peer-reviewed journal is often timely Professional websites .org .net .edu
Newspapers
and scrupulous. Authors submit work to Magazines
an editor who decides whether or not to Web sources: such as, blogs, commercial websites
put it up for review where it is evaluated (.com), Wikipedia, family & friends
by a panel of experts in the field (peer

Sciences
reviewers) who evaluate the scientific merit of the work and often suggest possible changes.
The peer reviewers can recommend to the editor to
• publish the article;

• send the article back to the author for revisions; or
• reject it outright.
There are a number of ways to check if a journal is peer reviewed.
• When searching for articles in a library database, use limits built into
the library website search engine to find only articles published in
peer-reviewed journals.
• Review a database website for information on whether or not the journals they
carry are peer reviewed.
• Check the submission page of scholarly or professional journals. The process for
submitting articles to these types of journals is explained along with the level of
review the submissions are subject to before being considered for publication.
It must be noted that not everything published in peer-reviewed journals goes through
this rigorous process. Book reviews, editorials (opinion pieces), and short pieces are often
included in scholarly journals but do not go through the peer-review process and are there-
fore not held to the same standard as those articles that are peer reviewed.
How can you determine whether or not an article is truly peer reviewed? In addition to
being published in a scholarly journal, at a quick glance it should contain one or more of
the following:
• an abstract that briefly summarizes the research design and methods;

• a systematic or thematic review of previous literature;
• a detailed description or discussion of a study that has been conducted; and
• a conclusion that clearly identifies new knowledge gained from the study.
Professional journals, like scholarly journals, contain work written by people working
and conducting research in a specific field. Some articles may describe research, but the
review process for acceptance for publication may be conducted with only an editorial
review. These articles are often written to:
• cover new and emerging topics in the field;

• report on field-related research; and
• evaluate best practice.
Popular journals (also known as magazines) and websites contain articles written by
writers who are employed by the magazine or freelance. These articles cover popular, hot
topics, which may appear at first glance to be research oriented but are not reviewed by
experts in the field. These facts in articles published in popular journals may very well be
accurate, but they are often quoted from opinion or taken from secondary sources. It is also

Sciences
equally likely that the facts in a popular journal article have been taken out of context. The
reader must always make determinations on the validity of the material.
Source material used to create Table 3.2 is a compilation of guidelines for distinguish-
ing the characteristics of material found in the different types of journals discussed above
(Rutgers, the State University of New Jersey, 2017; Illinois State University, 2017).
TABLE 3.2 Comparing Characteristics of Journals by Type
Scholarly Professional Popular
Purpose To present research ɓɓ To present research To inform and entertain

ɓɓ Report field-related research
ɓɓ Cover emerging trends
within the field
Audience Scholars, students, and researchers Professionals within the field The general public
Review Reviewed by a team of other schol- Reviewed by editors who are pro- Reviewed by editors who may not
ars who have extensive knowledge fessionals within the field have extensive knowledge regarding
within the field the subject
Writers Scholars within the field Professionals with specialized Professional writers and journalists
knowledge in a specific field
Characteristics ɓɓ Contain technical and highly ɓɓ Contain professional jargon ɓɓ Contain easy-to-understand
specialized language ɓɓ Cited material used in most language
ɓɓ Extensive cited material in all articles ɓɓ Some articles contain cited
articles ɓɓ Reference list included for material
ɓɓ Reference list included for all most articles ɓɓ Most articles do not contain a
articles ɓɓ Citations and references fol- reference list
ɓɓ Citations and references low style that pertains to a ɓɓ Citations and references nor-
follow style that pertains to a particular field, e.g., APA mally follow style for writing,
particular field, e.g., APA ɓɓ Field-related advertisement e.g., MLA
ɓɓ No advertisement ɓɓ Numerous advertisements not
related to articles
Adapted from https://www.libraries.rutgers.edu/scholarly_articles and http://guides.library.illinoisstate.edu/comparetypes.
What Are the Steps in Conducting a Systematic Review of the Literature?

There are many ways to locate relevant literature on a topic. While the Internet continues to
make it easy to access information from just about anywhere, it is important to not underes-
timate the value the assistance a reference librarian can provide. Not only is a librarian an
invaluable assistant when conducting research, working with a research librarian can save
untold amounts of money when accessing articles through databases, which often charge
individual users of monthly or per-article fees to download. Conducting a systematic review
of the literature through a college or university library website allows the user access to
hundreds of databases from which articles can be downloaded for free.

Sciences
Key word searches or relevant term searches in databases are the easiest way to begin
digging through the literature. Most concepts can be identified through the use of multiple
terms that address the same concept. Start with broad terms that encompass all aspects
of the general area or topic of interest covered in the research question, and move to more
specific terms as you move more deeply into the body of literature.
• An example of this approach can be illustrated with a systematic review of the

literature on the effects of electronic cigarette use. The researcher may start the
search by using a key word or key term, e-cigarettes. Once articles are identified
in the preliminary search, the researcher can locate key words listed on each of
the articles to direct a deeper search into the literature on the topic. This search
could incorporate various key word terms, such as electronic cigarettes, vaping,
and/or vape .
• Conducting the systematic review of the literature requires that each search term
identified be used in each of numerous databases that publish health science
articles. For example, in the Stony Brook University’s Health Science Library,
examples of two databases are CINAHL—this database has over 600 journals
that publish nursing and allied health literature—or PubMed, which publishes
biomedical literature from 1950 to the present.
• It is also important to consider relevant terms when describing the intervention,
or in the case of qualitative research, the phenomenon, that the research question
is addressing (Wakefield, 2014). Combining these terms when searching each
database will increase the likelihood of finding articles more closely related to
the topic being addressed. For example, combining the terms vaping and quitting
smoking will narrow the related articles and exclude any that focus solely on
vaping as a gateway to smoking cigarettes.
• Once the researcher locates a good number of preliminary articles, a more
extensive list of relevant terms can be created using the key words listed in the
beginning of most scholarly articles or by pulling terms used directly in the text.
A rigorous search incorporates the combination of many different key words in
order to exhaust the literature on a topic.
• It is good practice to keep a running list of terms searched to avoid repeating
searches and wasting precious time and energy.
Reference mining is another important strategy. It involves finding and reading the
research articles that are cited in other relevant articles’ literature review. Reviewing the
reference list at the end of a very relevant article will help to
• broaden the scope of the search;

• identify important scholars who have published on the topic; and
• open up new avenues to explore regarding a topic.
When using this practice, it is important to remain aware of the dates of publication for
articles that address the same or similar topics. This will be important when deciding which
articles to include when writing the literature review for a journal article.

Sciences
Another important tool available in most university or college online library systems
is called cited reference searching. This tool allows a researcher to search forward in the
literature and find articles that cite the article being read.
• For example, if the article being read was published in December of 2010 and
presented original research on a new smoking intervention for vaping, using the
cited reference search will find every article published after 2010 that used this
article in their literature review. This is a great way to find more current research
articles on the topic.
• Another use of cited reference searching is to create a stronger literature review
by allowing the incorporation of the primary source, the original research that
has been written by the researcher, into the paper rather than citing an article as
a secondary source. As stated earlier, the use of primary sources ensures that all
the information gathered is not subject to the interpretation of an outside author,
further ensuring the scientific merit of the information cited.
Hand searching the literature is an important step in the systematic review of the litera-
ture. Hand searching is the process of identifying key journals in the researcher’s reference
list and going through the journals page by page. The tools listed above help the researcher
find important articles; however, there is often a limit to the number of key search terms
listed in an article. A hand search can identify new search terms, types of studies that are
not listed under the article’s search term, and other source material for reference mining.
When conducting an exhaustive systematic review of the literature, it is a good idea to keep
track of the type of journal that published the article. This practice allows a researcher to keep
track of the relevance each article holds in relation to the research project. While articles that
are published in scholarly journals will carry more weight and credibility, it is not to say that
the information found in non-reviewed publications is not relevant. It is just important to
evaluate them based on the research question, the claims being made, and the ability to back
up claims with more credible sources (Cronin, Ryan, & Coughlan, 2008, p. 41). For example, a
researcher will want to review unpublished research studies (dissertations), or if a researcher is
studying heart disease they might use statistics found on governmental websites to document
the rates or percentages of people within the population affected by the disease.
When is it time to bring the process of locating relevant

literature for the systematic review to a close?
The researcher can stop the systematic review of the literature when they start to find
repetition. Some repetitions to keep an eye out for are:
• familiar arguments used in a number of articles to make a convincing case

for research;
• similar methodologies beginning to appear in a number of articles describing
research; and
• the same key people and the same studies being cited in multiple articles (Leedy
& Ormrod, 2016, p. 66).

Sciences
Once the researcher is fairly confident that they have exhausted the literature, it is time
to begin the process of organizing the material.
How Is the Systematic Review of the Literature Structured?

There are two organizational steps when creating a systematic review of the literature.
1. Organizing the articles;

2. Organizing the written flow of the actual review.
Organizing the relevant literature and organizing the writing of a literature review for a
journal article are similar but not exactly the same. When organizing the relevant literature
to bring it all together, it is important to first identify common themes (Wakefield, 2014)
and group articles according to those themes. A chart or concept map will help to organize
articles into an outline that traces the argument being made (Mongan-Rallis, 2014).
When conducting a systematic review of the literature, the last step to a comprehen-
sive organization of all the relevant literature is to create an annotated bibliography. This
alphabetical listing and short summary of each article
• keeps a bibliographic record of all research; and

• includes details about each study and findings.
This organizational strategy makes it easy to locate a specific article when writing the
literature review for the journal article. It is important to keep the annotation short and
to the point, giving only details pertaining to the research methodology and design, pop-
ulation, data collection, and analysis methods and findings. Annotations often include the
researcher’s evaluation of the article, a brief section on thoughts, and strengths and weak-
nesses of the article/research study. All should be written in the researcher’s own words,
without the use of quotes, to help facilitate a comprehensive understanding of the article
(Cronin et al., 2008, p. 41).
• An important part of the review process is being able to identify the quality of the
article (source and publication type). Another important component is an eval-
uation of the article; this process is often referred to as the critique of an article.
Yes, even articles published in peer-reviewed journals need to be evaluated/cri-
tiqued by the person conducting the systematic review of the literature.
• Critiquing or evaluating sources helps identify the research methodology,
research design, and research methods used to decide if the article relates to the
proposed study.
• This is an important part of the process of finding relevant articles. People often
hear the term critique and think that the purpose is to rip apart or find fault with
any or all components of the study and subsequently the article as well. Although
the term critique is derived from an archaic word that means criticism, Merri-
am-Webster (2017) defines it as “the activity of making judgments” about various

Sciences
forms of writing; it can be used as either a noun—“a detailed analysis and assess-
ment of something, especially a literary, philosophical, or political theory”—or a
verb “evaluate (a theory or practice) in a detailed and analytical way” (Oxford-
Dictionaries.com, 2017).
Simply put, a critique is an evaluation; it does not by definition have to be negative. How-
ever, if a critique of an article reveals unforgivable flaws in the researcher’s study, design, or
interpretation of the results, then that article should be rejected and not incorporated into
the systematic review of the literature. Said another way, the articles included in a system-
atic review of the literature should be prior research studies that contain a high degree of
scientific merit. More detail on how to evaluate/critique a journal article will be found in a
later section of this chapter.
LITERATURE REVIEW—DEFINED
During the last stage of the research process (dissemination), a literature review is written
as part of a journal article’s introduction section. Its purpose is to provide background
information to review the current and relevant prior research that has been conducted in
relation to the topic, and then help the reader of the article understand the gap in the liter-
ature (problem statement) that led to this new research study.
• At the dissemination stage of research, the introduction or background and sig-

nificance of an article written to share the results of original research is often
characterized as the literature review.
• The literature review found in the introduction section of the journal article is
a highly summarized, integrated, and synthesized version of the exhaustive sys-
tematic review of the literature conducted during the first stage of research. The
literature review in a researcher’s journal article has two important goals:
■■ It must demonstrate that the research topic is important.
■■ It must show that the results presented in the article are filling a gap in
the literature.
Writing the Literature Review for a Journal Article
Should every article found during the systematic review of the literature
be included in the journal article’s literature review?
• The literature review found in the introduction section of a journal article is a

subset of the material uncovered during the systematic review of the literature.
The journal article contains a highly summarized, integrated, and synthesized
version of the exhaustive search of the literature conducted during the first stage
of research.
The literature review in a journal article should provide the reader of the article an
overview of the most current and relevant previous research conducted on the topic

Sciences
to clearly identify the gap in the literature (problem statement) that led to the design
of the study.
■■ Current – the researcher summarized, integrated, and synthesized all
articles that have been published in the past 5–7 years on the research
topic. One exception to this rule can be made for groundbreaking research
(often referred to as seminal research) that has been proven to set the
standard in a field and has been cited repeatedly in relevant articles. In
addition to groundbreaking research articles, a researcher can choose
to include additional research articles that are older than this range
if the articles add important concepts or clarification to the body of
literature. It is always good practice to back up any claims from older
research with similar claims in newer research as a way to defend why the
information found in the older article is relevant enough to be included
(Mongan-Rallis, 2014).
■■ Relevant – the researcher conducts an in-depth digging through all that
has been previously written specific to the purpose of the proposed
research topic and incorporates all relevant information.
Most literature reviews found in journal articles start by introducing classic works
(groundbreaking/seminal studies) early to provide a historical perspective, then move into
research that deals more specifically with the issue being investigated in this research project.
The literature review ends with a statement of the problem leading to the purpose of the
study being conducted. This statement is most often expressed explicitly and is referred to
as the purpose statement. The problem statement and purpose statement, which are both
identifiable in the literature review whether implicitly or explicitly, have been touched on
earlier in this chapter and in Chapter 1. They will be covered more in depth in Chapter 4.
SIDE BOX 3.2 This chapter focuses on how the literature is used during the first and last stage of
research. It must be noted that during the IRB application process described in Chapter 2, the researcher
must demonstrate that the research topic has the potential to generate new knowledge. This is done by
using a highly summarized, integrated, and synthesized version of the exhaustive systematic review of the
literature conducted during the first stage of research.
So, the literature review derived from the systematic review of the literature is used twice: once during
the IRB process and then again when the study has concluded and the researcher is presenting the findings
in a journal article.
WRITING THE LITERATURE REVIEW

Once the researcher’s systematic review of the literature has documented a researchable
problem, it is important to think about how the literature review will be structured. Will
the focus be on:
• comparing and contrasting varying theoretical perspectives;

Sciences
• highlighting conflicting findings on a topic; or FIGURE 3.2 An illustration of the

• on identifying the research progression over time? writing structure for a literature
review.
Adapted from https://www.libraries.
It is important to create an outline that follows the path of the argu- rutgers.edu/scholarly_articles and http://
ment from a general overview to more in-depth attention to each research guides.library.illinoisstate.edu/compa-
question or subproblem included in the argument (Figure 3.2). retypes
Writing a literature review goes far beyond summarizing all the articles
found on a specific topic. It includes critical analysis of the relation- Main Point
ship between works in addition to identifying a theoretical framework
Importance
and a rationale for a new research study (University of California Santa Primary Details
Cruz, n.d.; Mongan-Rallis, 2014). It is expected that researchers will
construct the case for their efforts by showing a sequential, logical Secondary Details
flow of work, in which each subsequent step advances what has been
accomplished previously.
A balanced literature review will include previous research that looks
to both
• support the research question; and

• counter the research question.
The literature review should root the current study in the context of what is known and
what is unknown on both sides of the topic. This approach shows an unbiased focus of the
researcher, an openness to addressing and the ability to counter contradictory findings, and
a deep level of knowledge surrounding all sides of the topic. It is reasonable to expect that
the review of the literature should include some disparity in the previous findings.
Once the path of the argument has been mapped out, the first draft of any literature
review should be written from memory following a path that moves from broad, encom-
passing statements to statements that focus more specifically on the research purpose. Side
notes and reminders in parentheses throughout the writing assist in areas that may need
clarification or fact checking later on. Once this draft is finished, citations from the articles
can be incorporated into the draft to support statements and arguments. The more sources
that are cited to support each statement or argument, the stronger the literature review.
SIDE BOX 3.3 Does every article make it to the final round?
During the writing stage researchers commonly identify articles that appeared at first to relate to the
topic but upon further evaluation determine that the article is not a match. When deciding whether or not
to include every article, a researcher may feel that having an extensive reference section will make the work
appear stronger. However, if an article is not cited anywhere within the work—LEAVE IT OUT!
Always give credit where credit is due! It is important to identify the source of all ideas
in writing. A literature review is just that: a review of information already written on a spe-
cific topic. There are no new ideas being introduced until the research purpose statement
that describes the current study, so it goes to say that all points made will be cited from the

Sciences
research. Being able to connect articles and ideas to each other and a larger framework of
knowledge is an important part of the writing process. Synthesizing literature is an import-
ant step in the literature review process; it goes beyond merely reading articles to identify
whether or not the article relates to the research question. It is pulling together diverse
perspectives and research results into a cohesive document that flows from the overarching
idea (research problem) to the specific focus being addressed (research purpose) (Leedy &
Ormrod, 2016, p. 67).
SIDE BOX 3.4 It is tempting to compile entries from the annotated bibliography as a way to list all
the relevant sources and call it a literature review. It is important to understand that a literature review
is very different from an annotated bibliography. The series of annotations in an annotated bibliography
summarize each article independently and do not look for a relationship between studies and findings.
Taking these summaries of articles and dropping them into a literature review, devoting a paragraph to
one and then moving on to the next one, is low-level writing. Synthesizing the literature entails weaving
sources together, determining those that support each other, and marrying the findings to create an inter-
connected foundation to establish a context of study. Also, the more sources that support a single idea or
thought, the greater the scientific merit of the argument presented.
Another rule to remember when writing a literature review is to minimize quotes. Quoting
sources directly incorporates relevant information into the writing, but it does not show a
deeper understanding and connection to the greater concepts being discussed. This is where
writing from memory, then going back and adding citations to areas that discuss something
covered in previous literature, creates a chance to connect previous literature and include
sources in the context of their relationship to the current topic.
The last step when writing a literature review is to revise, revise, and revise again. Schol-
arly writing is direct and to the point. Revise the writing to eliminate unnecessary words
and repetition throughout the document.
CRITIQUING A JOURNAL ARTICLE—DEFINED

The last goal of this chapter is to provide an overview of how to critique journal articles.
Writing a critique of an article provides the opportunity to not only identify the strengths
and weaknesses of each component of the research article, but it allows one to identify
where, within the article, specific information about the study can be found. This section
of the chapter will
• Distinguish between the types of research article;

• Identify the main components of a research article;
• Help to identify how these components differ based on methodology; and
• Provide an overview of important factors to evaluate when critiquing a
research article.

Sciences
There are many tools available on the Internet to help evaluate the rigor of research
articles. Checklists on how to critique different types of journal articles can often be found
in the libraries of colleges and research universities. One can also find information from
a variety of web sources. For example, Critical Appraisal Skills Programme (CASP) offers
different critique checklists that correspond with the research methodology and research
design of the article under review (CASP, 2018).
Critiquing a Review of the Literature Article

What is the difference between a review of the literature scholarly article and the literature
review section of a research article? This is a great question, as the two sound so alike they
could be exactly the same thing.
A review of the literature scholarly article organizes and presents previous research in
relation to a topic but does not put forth new knowledge. A literature review is the first
section of every research article. It is a comprehensive review whose purpose is to identify
the gap in the literature that led to the new study being discussed.
While a review of the literature is often published in a scholarly journal, its purpose
is to provide an in-depth evaluation and analysis of the related literature, and as such
does not follow the format of a research article. This type of article will use much more
material from the systematic review of the literature. Although it is a scholarly article,
it is considered a secondary source. An important aspect is that it focuses on creating a
relationship between previous works and the topic being addressed in a way that has not
been previously addressed.
• A review of the literature worthy of being published in a peer-reviewed journal

could synthesize prior research studies with a theoretical perspective that has not
been previously presented in the literature.
• Like all scholarly articles, a review of the literature begins with a general
statement that gives a broad overview of the question or theoretical frame-
work guiding the review. This is the overall goal of the work. The purpose is
to encompass the groundbreaking ideology driving the topic, not to make a
global statement.
• The subsequent components are what set it apart from a research article that
introduces new knowledge. Once the researcher has established the ideology
dictating the focus of the review, it is important to indicate the relevance of the
topic being reviewed and discuss the parameters within which the review was
conducted; i.e., time line set, journals included or excluded, search engines used,
and what type of sources have been included (scholarly, professional, secondary,
etc.). This sets the stage for the focus of the review. This section should state
explicitly what will and will not be covered in the review.
• The review should use subheadings within the body to clarify key points as they
are covered and to guide the flow of the research topic. Following the body, the
review should end with a conclusion that restates the purpose and provides clo-
sure to the argument (Mongan-Rallis, 2014).

Sciences
Critiquing a Research Article

This section will detail how a research article should be formatted. The following questions
should be kept in mind when reading a research article:
• Is it from a peer-reviewed source?

• Is the research question implicitly or explicitly stated?
■■ Implicitly stated means that all the information is included, but it does
not follow the exact format of a research question. The reader of the arti-
cle will need to infer what the research question was.
■■ Explicitly stated means the research questions and hypothesis are listed.
• Is it logically organized and easy to follow?
• Are previous studies described and well integrated?
• Are procedures clear and easy to follow?
• Are the data collection method and analysis fully discussed?
• Are the authors’ interpretation and conclusion included?
• What are the strengths and weaknesses of the article?
While these questions create a general guideline to follow when determining the overall
value of an article, it is important to be able to understand the distinct purpose each section
of a research article has to the presentation of the study created and laid out in the article.
This allows the researcher critiquing the article to pull relevant information quickly and
determine not only the scientific merit of the study but the value it holds when contributing
to the current research being proposed.
A research article puts forth new knowledge and provides an examination of the pro-
cess the authors followed when conducting research, as well as an in-depth analysis of the
findings. Research articles describe the components of a research study and are distinctly
constructed to describe the
• methodology followed;
• study design utilized;
• data collection and analysis methods employed; and
• introduction of all new knowledge discovered.
While each of these sections is found in every research article, the methodology of the
study determines the way in which the data is collected, presented, and discussed when
writing about the research study.
The difference between quantitative and qualitative research is very distinct. The breadth
and depth of research included in a literature review is often directly related to the research
methodology employed in the study. Most qualitative research studies look to gain an under-
standing of the underlying how or why of a behavior, situation, or real-life process. They
explore phenomena that have not previously been looked at in research and incorporate less
preliminary research in the literature review than quantitative research studies that have
extensive previous research. Therefore, qualitative research questions are broad, and quan-
titative research questions are very narrowly focused, resulting in an inverse relationship,

Sciences
FIGURE 3.3 An illustration of the

as seen in Figure 3.3, between the research question and the literature inverse relationship between litera-
review based on methodology. ture review and research questions
Regardless of whether the research is quantitative and qualitative, all by qualitative (Qual) and quantita-
articles begin with an abstract, an accurate, well-written, concise, and tive (Quan) methodologies.
specific overview of all the components within the article. The abstract
helps the reader determine whether to continue reading the entire article. Literature review
An abstract is only a paragraph long (five to six sentences MAX!) and
covers only a few basic items. Quan
Qual
1. What is being studied, and why is it important?
2. What is the study methodology? Research question
3. What are the findings, and what is their significance?
That’s it! It is important to remember: the purpose of an abstract is to

summarize an entire journal article so someone reading it can determine
whether or not the article is relevant to the topic being searched and
therefore worth reading in its entirety. The abstract should be succinct
with no extra wording but comprehensive enough that it provides adequate
information about why the research was conducted (problem/purpose),
on whom the research was conducted (sampling), how the research was
conducted (methodology), and what happened (the findings).
Immediately following the abstract should be a key word list. This identifies the key words
searched when compiling the review and allows researchers to conduct their own research
on the topic using the same or similar terms.
QUANTITATIVE RESEARCH ARTICLES

As is illustrated in Figure 3.4, quantitative research articles are usually composed of four
easily identifiable sections: the Introduction, the Methods, the Results, and the Discussion/
Conclusion.
Introduction Section
The introduction, sometimes called the background and significance or background, follows
the abstract. This section is often referred to as the literature review section of the research
article. This section
• provides the highly summarized, integrated, and synthesized version of the sys-
tematic review of the literature conducted during the first stage of research; and
• ends with the research study’s problem statement, purpose statement, and
research question(s).
• Remember, the most important goal of the introduction is to identify the gap in
the literature which sets the stage for the study.

Sciences
FIGURE 3.4 An illustration of the four sections of a quantitative research article.
Interpretation of the data in Review of literature,

relationship to Research problem,
question/hypothesis Discussion/ Introduction Research purpose,
Limitations of the study Conclusion Research question/hypothesis
Suggestions for further research
Results Methods
Research design, and
Data analysis summarized, methods (sampling, data
usually in table, charts, graphs, collection, data analysis)
figures (does not include Detailed overview of research
interpretation of the data) procedures & materials
To review from earlier in the chapter, the problem statement identifies the topic as
researchable and uses a cited summary of previously published research to support any
claims made in the literature review. The research problem statement is a specific fram-
ing of the gap in the literature, which is used to develop the basis for the implementation
of the research being conducted. The research problem statement should come directly
from the literature review, either in a complete statement or pieced together from various
statements made throughout the document. Either way, this information should be found
relatively easily in the article because it is required to demonstrate that the problem rises
to the level of research.
The introduction section often concludes with the purpose of the study (purpose state-
ment) and the research questions that the study will address. The purpose statement and
the research questions can be stated explicitly or implicitly. Either way the reader of the
article should be able to identify the purpose of the study and how the study will add new
knowledge (research questions) to the literature.
Evaluating or Critiquing the Introduction Section

Evaluating or critiquing an article’s introduction section looks to determine the quality of
the literature reviewed as well as the authors’ credibility on the subject. From the beginning,
the author(s) of the research article should be able to convince the reader that their work is
important and that they have the skills and appropriate perspective to have accomplished
their claims. A critique should show that the literature review provides a theoretical frame-
work to ascertain the value or rationale for carrying out the study and determining whether
or not the study accomplished its goal.
When looking to determine the credibility of the literature, it is important to identify
whether the cited material is current and relevant. These terms have been used numerous
times throughout this chapter. They are used in the same context when critiquing an article
as they are in the reviewing and writing process.
In addition, look to determine if the authors emphasize primary, rather than predom-
inantly secondary, sources. Secondary sources are permitted, but they should be used

Sciences
sparingly and always clearly identified: “A study by Jones (as cited in Kottler, 2003) found …”
It is important to remember that efforts are built on the basis of accumulated knowledge
(especially in quantitative studies); always look for a coherent theme in the writing.
Important things to look for when critiquing the introduction
• Researcher bias: a critique looks to clarify researcher bias. Is the review of the lit-
erature balanced, and does it present both sides of the topic or argument? Good
research clearly shows how the literature supports the current research problem
and resulting questions, but it also includes literature that argues all sides of a
topic, not just the literature that supports the research question.
• Link to theoretical framework: the introduction should clearly describe any gap
in the literature that will be addressed. Does the literature review offer sufficient
rationale for the study? There should be a clear link to the theoretical framework
within the literature review. Is the presentation of facts clear and objective?
• Writing style: the writing should be direct and to the point, devoid of flowery lan-
guage and overly wordy descriptions. Language used should portray the general
intent of the study with clarity. Academic writing is fact driven and heavily cited. Look
to ensure that all statements are supported by citations from previous literature.
• Is it important? Finally, the critique must evaluate the importance of the overall
research being conducted. Is this importance significant to society, or is it per-
sonally relevant to the researcher? It is important for a research study to have
personal relevance to the authors and to the readers. Personal relevance is often
what underlies the creation of a research study and the desire for new studies to
contribute deeper to the existing knowledge in current literature, but personal
relevance is not enough. There must be scientific merit and a clear significance to
the field of health science or the population as a whole.
Research Questions and Hypotheses

Research questions and hypotheses, which are stated after the research purpose statement
• drive the quantitative study;

• provide clarity to the research problem statement; and
• give a specific and narrow explanation of the questions the current study will be
looking to determine.
One question to keep in mind when critiquing the research questions is, does the research
question contain an operational definition of each term? Kerlinger (1986) defines the term
operational definition as assigning “meaning to a construct or a variable by specifying the
activities or ‘operations’ necessary to measure it” (p. 28). Therefore, operational definition
of terms is a specific explanation of how something is being measured within a study. While
it is nearly impossible to create the perfect operational definition, the more detailed and
clearly stated it is, the easier it will be for someone to recreate a study. Most operational defi-
nitions follow the specific theory behind the research and are rooted in previous literature.
Knowing this information is critical for the next section of a journal, the methods section.

Sciences
Methods Section
The second section of a quantitative research article identifies how a research question
was addressed. This section is most often labeled methods or methodology. This section
should include the
• methodology;
• design; and
• methods (sampling, data collection, and data analysis)
that were used to conduct the study. This section should be explained in sufficient detail so
that another researcher can use the parameters discussed and replicate the study (recreate
the study).
SIDE BOX 3.5 During a systematic review of the literature, the evaluation of this section of a research
article will often determine whether or not the study should be included in your literature review. As
previously stated, even if a similar research study has been conducted, if the proposed new study plans
to use a new population or new study design, it can have scientific merit. For example, if only descriptive
studies have been done on a topic, then conducting an experimental study on the same topic will add to
the literature.
Evaluating or Critiquing the Methods Section

It is important to look for a full description of the population and how it was identified and
selected. Sampling methods should be listed and should clearly align with the methodol-
ogy, the research design, and research questions. It is important to evaluate whether the
researcher sampled correctly to answer the research question sufficiently and/or obtain
generalizable results.
This section should also include how the data was collected, explicitly naming or detail-
ing the data collection process. Information on the tool or instrument used to collect data
should be provided, including information on the tool/instrument’s validity and reliability
(more detail will be provided in Chapter 5). The more detailed the explanation, the easier
it is to replicate a study.
For example, it is not enough for a research question to say it will measure incidence of
depression in people who have experienced a spinal cord injury. Simply stating that depression
will be measured leaves too many unanswered questions. Rather, an operational definition
and resulting measurement tools should clearly explain how incidence of depression is
defined and measured.
• What symptoms of depression will be measured?

• Will the researchers measure change in appetite, weight loss/gain, suicidal
thoughts, feelings of sadness or worthlessness, loss of interest in everyday activi-
ties, or other signs and symptoms of depression?

Sciences
• Will the measurement take place in a group or alone?

• Will these factors be measured with a questionnaire that rates answers on a scale
or with structured observation?
• How does the researcher know that the measurement tools being used are valid
and reliable?
The final component of this section is a review of the data analysis procedures: statistical
tests used to analyze the data. More detail on data analysis is provided in Chapter 5 and
types of statistical analysis based on research design will be found in Chapters 7–9. The
minimum information this section should include is, “what statistical tests were undertaken,
why these tests were used … studies that are descriptive in design only use descriptive sta-
tistics, correlational studies, quasi-experimental and experimental studies use inferential
statistics” (Coughlan, Cronin, & Ryan, 2007, p. 662). An important question to consider
when reading this section is, can the statistical test(s) utilized in the study actually answer
the stated research question(s)?
In summary, the important thing to look for in this section is a step-by-step description
of the procedures used. The overarching question to be thinking about when reading this
section should be: Is there enough detail in this section that another researcher could easily
replicate the study?
Does this section include a full description of the:
• Research design
■■ Does the design of the study align with the types of research questions
being asked?
• Sampling methods
■■ Is there information on how the participants were recruited and selected?
Is there information on how the number of participants needed for the
study was determined?
• Data collection tools/instruments utilized and information on the tools/instru-
ments’ validity and reliability as data collection methods
■■ If a survey was created specifically for the study, are samples or summa-
ries of the questions along with a description of the process employed to
create the tool and the methods used to ensure it was valid included?
• Data analysis strategies
■■ How was the data analyzed? If there are multiple research ques-
tions or hypotheses are different, were statistical tests used for each
question listed?
• Any other strategies used to increase reliability and validity.
Results Section
The third section of a quantitative research article presents the results of the statistical
data analysis. The results are often presented in both narrative format in descriptive para-
graphs and in numerical format within tables or charts. This section should only present the

Sciences
statistical results of the study’s data analysis. There should be no interpretation or discussion
of the meaning of the results of the study in this section of the article.
Evaluating or Critiquing the Results Section

A critique of this section requires an in-depth knowledge of statistics. The presentation of the
data should be detailed enough that a fellow researcher could critique the appropriateness
of the statistical techniques used. While a deep level of statistical knowledge is beyond the
scope of this textbook, there are some questions to consider while reading this section. Has
the researcher selected the most appropriate statistical procedures to analyze the data? If
the study addressed multiple research questions, was the analysis clearly presented for each
question? If inferential statistics were used, was the alpha level selected by the researcher
appropriate for the study? Did the narrative presentation of the results support the numer-
ical presentation of the results? Were the results presented clearly, logically, and accurately?
Discussion/Conclusion Section
The discussion/conclusion section is the final section of the article. This section includes an
interpretation of the results, limitations of the study, and suggestions for further research
studies that can build upon the results of this study.
While data was laid out in the result section, it was not interpreted; this section gives
the authors a chance to discuss the results of the study in relation to the research questions
and hypothesis posed earlier in the introduction. Said another way, in this section, the
researcher gets to discuss specifically how the study generated new knowledge and added
to the body of scientific literature.
Evaluating or Critiquing the Discussion/Conclusion Section

The following is a list of some questions to ask while reading this section.
• Is the author’s conclusion consistent with the statistical analysis of the study?
■■ Meaning if the statistical test conducted yields correlations between vari-
ables, is the author’s stated conclusion supported by the data analysis?
The researcher should not misinterpret or misrepresent the conclusion
that can be drawn from the data.
• Can the conclusion be linked to the theoretical framework of the study, the lit-
erature review, problem statement, and the specific research questions found in
the introduction?
• Are the limitations of the study clearly stated and discussed?
• In addition to a discussion of how the results of the study generated new knowl-
edge, how did the results of this study open new avenues of inquiry?
SIDE BOX 3.6 In this section the authors have a chance to evaluate their approach and determine if
there were any limitations to the study that they only realized once they were in the midst of conducting
the research.

Sciences
This evaluation is very important to future work in this area. Limitations discovered in this research
can lead to a new study that modifies and corrects for these limitations, thereby providing possibly more
accurate determinations.
In addition to limitations, the authors often point out alternative explanations for their findings in this
section. When outlining these key points, authors often make recommendations for future research.
QUALITATIVE RESEARCH ARTICLES

Qualitative research seeks to develop understanding in an area that has yet to be fully
explored; for example, the how and why of a decision-making process or the shared meaning
of an emotionally charged event. Qualitative study designs are more flexible than quantitative
research, with data collection and patterns and themes emerging throughout the research
process. To this end, qualitative research articles are structured similarly, but the informa-
tion is presented differently. As is illustrated in Figure 3.5, qualitative research articles are
usually composed of four easily identifiable sections: the Background, the Methods, the
Analysis/Findings, and the Discussion/Conclusion.
Questions to ask while evaluating a qualitative research article may vary in terminology
used from questions asked when evaluating a quantitative research article, but the basic
premises being looked at are very much the same.
FIGURE 3.5 An illustration of the four sections of a qualitative research article.
How this is reported is Review of literature

dependent on the design of the Problem statement
study Purpose statement or aim
Should provide reader with a Discussion/ Introduction of the study
rich, descriptive understanding Conclusion Research question
of the phenomenon
Results Methods
How this is reported is Research design and methods
dependent on the design (sampling, data collection, data
of the study analysis)
Can include pictures, drawings, Detailed overview of research
diagrams, large volume of process & role of researcher
quotes from participants
Background Section
The background section contains the review of the literature. This section is similar to the
introduction section in a quantitative research article in that it describes all the previous
research pertaining to this topic. The most obvious difference is that this section is often
significantly shorter in a qualitative article. This is a reflection on the types of questions
qualitative research often looks to address. Much of qualitative research is exploratory. It
is used to gain an understanding of a phenomenon, an underlying reason, a trend, or an
opinion. Much of what is being studied in qualitative research “gets at the how and why of
the story, in ways that quantitative research cannot. The key concept of the study is often

Sciences
referred to as central phenomenon in scholarly writings” (Yates & Leggett, 2016, p. 225).
As qualitative research is exploratory, there is often not a significant amount of previous
literature that addresses the topic. The background section often starts with a statement
identifying why this research is important, then utilizes the research purpose statement or
the aim of the study to narrow the focus. Regardless of the length, it is necessary to consider
if there is a clear statement of the aim of the research.
The purpose statement provides the premise for the study and includes the central phe-
nomenon, the participants, and the location of the study (Yates & Leggett, 2016, p. 225).
The purpose statement is often called the aim of the study and is usually open ended, as
qualitative research questions evolve throughout the course of the data collection and
analysis process. Qualitative research questions are broad and open ended as well, since
the researcher is looking to explore a phenomenon and is therefore not fully certain what
will emerge throughout the course of the data collection.
Evaluating or Critiquing the Background Section

Evaluating or critiquing an article’s background section assesses the quality of the literature
reviewed as well as the authors’ credibility on the subject. From the beginning, the author(s)
of the research article should be able to convince the reader that their work is important and
that they have the skills and appropriate perspective to have accomplished their claims. An
important question to focus on is: Does the literature review lead to a rationale explaining
the gap in the literature? A critique should show that this section provides a theoretical
framework to ascertain the value or rationale for carrying out the study and determining
whether or not the study accomplished its goal.
Methods Section
The methods section should include detailed information on the research design and research
methods used in the study. This section, unlike quantitative research articles, includes
information about the role of the researcher in the study and the relationship between the
researcher and study participants. The reason for this is that unlike in quantitative research,
where the researcher attempts to conduct the study from an objective stance with the goal
of eliminating researcher bias, the qualitative researchers immerse themselves in the data
collection and analysis process. The research design, along with the sampling, data collec-
tion, and data analysis methods, are also included in this section.
Evaluating or Critiquing the Methods Section

While the goal of this section is not to provide enough information that another researcher could
replicate the study as in a quantitative article, there should be enough detail so another researcher
can make judgments about the quality and rigor of the study. Questions to focus on include
• Has the relationship between researcher and participants been adequately

considered? The researcher is often more immersed in the research process
in qualitative research. It is important to critique if the relationship between
researcher and participants has been adequately considered, accounted for, and
explained in this section.
• Was the research design appropriate to address the aim of the research?

Sciences
Sampling in qualitative research uses nonprobability methods, in that a population is

chosen based on a desired characteristic or experience. Questions to keep in mind when
critiquing this section are
• Was the recruitment strategy appropriate to the aims of the research and the
design of the study?
• Is there a detailed description of who the sample was?
Data collection in qualitative research is often accomplished through unstructured

techniques like
• interviewing;
• asking participants to keep diaries;
• collecting images;
• collecting artifacts; and/or
• observation.
This allows the researcher to direct the data collection in a way that allows new infor-
mation to emerge throughout the research process. Since a researcher often does not know
ahead of time what will be significant during qualitative data collection, it is important to
assess if data was collected in an emerging, reasonable, and thoughtful way.
Analysis/Findings Section
The analysis/findings section of a qualitative research article can vary greatly depending on
the research design, the data collection strategy, and the data analysis strategy employed.
Since data can be collected through video recordings, audio recordings, drawing, creative
media, and volumes of written transcripts, to name a few, the presentation of the analysis
can vary greatly. Typically, the analysis/finding section includes pictures, direct quotes from
participants, and/or flow charts showing the data analysis grouped by themes. Interpretation
of the data also integrates the researchers’ experience during the process. A major differ-
ence between a quantitative and qualitative article is that in a qualitative research article,
this section (presentation of the findings) includes an interpretation of the results. This is
an important point to remember: the results section of a quantitative study only includes
the results of the statistical tests. Conversely, the analysis/findings section of a qualitative
study will integrate analysis with the presentation of the findings, often citing literature
to support the findings being presented, a practice forbidden in the Results section of a
quantitative article.
Evaluating or Critiquing the Analysis/Findings Section

Researchers should describe the methods used to protect the analysis from being influenced
by the researchers’ own values and outlooks. Quantitative researchers report on mea-
sures they took to increase the internal validity/external validity of the study. Qualitative
researchers do not think in terms of validity or reliability; instead they focus on establishing
trustworthiness. For example, triangulation, cross-checking data through the use of multiple
data collection strategies, using multiple observers or collecting data at numerous times

Sciences
throughout the research process, and asking participants to review the analysis are common
ways that qualitative researchers establish trustworthiness. The concepts of validity, reli-
ability, and trustworthiness will be covered in greater detail in Chapter 5.
Discussion/Conclusion Section
In this final section it is important that the researchers use rich description to discuss the
findings and conclude the study. Findings should be tied back to the central phenomenon
described in the background. Qualitative articles will weave a discussion of the literature
into the analysis of the findings and discussion of the central phenomenon as appropriate.
Evaluating or Critiquing the Discussion/Conclusion Section

The critique process for a qualitative study centers on an evaluation of the study’s overall
trustworthiness. The most common criteria include a review of the study’s “credibility,
dependability, transferability” (Ryan, Coughlan, & Cronin, 2007, p. 743) with this evaluation
spanning the methods, analysis/findings, and discussion/conclusion sections of the article.
MIXED METHODS RESEARCH ARTICLES

Mixed methods research articles should clearly share the rationale for why conducting both
a qualitative and a quantitative study was required to answer the research questions. Mixed
methods research studies will be defined and described in detail in Chapter 10. For purposes
of this chapter, it is enough to say that mixed methods articles include both quantitative
and qualitative designs and methods and include a discussion of both results and findings.
Critiquing mixed methods research articles requires an understanding of how both qualita-
tive and quantitative research is conducted and how their findings/results are disseminated.
Professional Assessment
While all sections of each type of article have been evaluated at this point, there is still one
more important section of the critique: the reader’s individual interpretation of the article.
This is an opportunity for the reviewer to determine the scientific merit of the study and
the contribution it has made to the literature. The critique is guided by the purpose of
reviewing the literature, EBP, or to create a problem statement. Therefore, it is essential
that the reader of the article focus on the following:
• Is this study appropriate for my needs?

• What are the strengths and weaknesses of the article?
• Was the population sampled appropriately to coincide with the
research question?
• How did the researchers validate their findings?
• Was the data analysis method used the best choice?
It is important to remember, no one research study will answer all questions, and each
research design and method has strengths and limitations. This section is not about what
could have been done better; it is a professional evaluation of the rigor of study that was
painstakingly constructed before the authors knew what the results would be.

Sciences

Literature plays an important role in the research process, starting with the first stage where
it is systematically reviewed to determine if a research question exists, to the final stage
where it is succinctly organized to present the authors’ background information on the topic.
A systematic review of the literature looks to the past to determine what should be studied
in the future. It informs new research and allows researchers to justify the need for new
studies. While reviewing and writing are important components of the research process, cri-
tiquing an article and the various segments within it is an important skill needed to evaluate
the rigor of a study and the value of the knowledge generated with each finding published.
This chapter introduced the concepts of the systemic review of the literature, an import-
ant component of the research process. It is essential that a researcher possess the ability to
understand and correctly evaluate the significance of sources and weave them into a cohesive
document that simultaneously examines past research to incorporate those findings into
the exploration of a new study.
• The systematic review of the literature is an important component of the

research process because it is where the researchers determine that a need for
the proposed research exists.
• The writing of a literature review for a journal article is equally an important part
of the research process. A well-crafted literature review provides the scientific
community an overview of the topic and the rationale for conducting the study.
In addition, the chapter also provided an introduction to the various levels of sources
that can be integrated into writing and provided guidelines for determining the merit of
various sources. And finally, this chapter illustrated the difference between the types of
research articles and outlined what information is important to critique in each section.
STUDY PRACTICE
Using your university’s library database, practice using key word terms to find articles
related to a topic you are currently interested in knowing more about. Try different terms,
and remember to link them together in various ways to maximize the results you get from
each search. Utilize the limiters provided by the website to set parameters of publication
dates or full-text articles only.
Once you have articles that match your topic, read through with a highlighter or pen handy.
When you identify important information in each section, make note of it by highlighting or
circling the information. As you become more familiar with the format of scholarly articles,
you will be able to quickly locate the information you need from each section. This will help
you to quickly determine when an article is appropriate for building a stringent base for
the background and significance of any topic you are searching. This knowledge will help
you to critique scholarly articles, a much-needed skill when moving forward in the world
of academic writing.
Since the reducing notes from 3:1 recommendation cuts across all learning styles, the
first suggestion for studying is to start making flash cards for every bolded/italicized term.
The important words/terms/concepts presented in this chapter include:

Sciences
ɠɠ systematic review of the literature ɠɠ cited reference searching

ɠɠ literature review ɠɠ hand searching
ɠɠ current and relevant ɠɠ critique of an article
ɠɠ gap in the literature ɠɠ annotated bibliography
ɠɠ primary sources ɠɠ article
ɠɠ secondary sources ɠɠ abstract
ɠɠ tertiary sources ɠɠ introduction section
ɠɠ scholarly (academic, refereed, or peer-re- ɠɠ methods section
viewed) journals ɠɠ results section
ɠɠ professional journals ɠɠ discussion/conclusion section
ɠɠ popular journals ɠɠ replicate the study
ɠɠ key word searches/relevant term searches ɠɠ background section
ɠɠ database ɠɠ aim of the study
ɠɠ reference mining ɠɠ analysis/findings section
ɠɠ seminal research

1. Which of the following is not a primary purpose of the literature review?
a. To offer new ideas, perspectives, and approaches about a research problem.
b. To identify a gap in the literature by reviewing the findings from previous studies.
c. To describe theoretical perspectives and previous research findings related to
the problem.
d. To write the study’s research questions.
2. The introduction section of a quantitative research article should include:
a. description of the general problem
b. purpose of the study
c. limitations of the study
d. a. and b.
e. b. and c.
3. The analysis/findings section of a qualitative research article should include:
a. a description of the research methodology.
b. pictures, drawings, diagrams and/or direct quotes from participants.
c. a rich descriptive explanation of the demographic variables of the participants.
d. a detailed overview of the role of the researcher in the research process.
4. When conducting a systematic review of the literature, how would you know
when to stop, meaning how do you know you have thoroughly searched
the literature?
a. When you have evaluated and synthesized previous research in relation to your
problem statement.
b. When you uncover a new idea, perspective, or approach not previously thought of.

Sciences
c. When you see repetitive authors, patterns of arguments, methodologies, and

the articles yield no new insights into your research problem.
d. When you have included both books and peer-reviewed journal articles on
the topic.
5. A peer-reviewed source is one that:
a. allows a reader to leave comments for the author.
b. has been reviewed by experts in the field before publication.
c. has been reviewed by an editor before publication.
d. uses only original sources in its list of references.
6. A discussion/conclusion section of a quantitative research article should include,
all the following except:
a. sufficient information about the methodology, design, and methods for another
researcher to replicate the study.
b. limitations of the study that could impact conclusions.
c. linking conclusions to theoretical framework or other articles in the literature.
d. an explanation of what the results mean in relation to answering the
research questions.
7. Generally speaking, the best time to begin the systematic review of the liter-
ature for a specific research project is:
a. after the research problem statement has been clearly defined.
b. after the data have been collected but before they are analyzed.
c. based on the researcher’s preference; there is no single best time.
d. during the exploration of the research problem.
8. When writing a cohesive literature review, during the dissemination stage, a
high-quality literature review includes all except:
a. many direct quotes from previous research articles.
b. identification of themes and evaluation of previous research.
c. clear summaries that integrate previous research with proper attribution of all
authors of the previous articles.
d. comparison of contrasting theories or findings from previous research.
9. Which example describes the information found in the four sections of a quan-
titative research article?
a. Literature review, research design/methods, statistical analysis of data,
interpretation.
b. Methodology, data, analysis, conclusions.
c. Literature review, problem statement, purpose statement, research questions.
d. Hypothesis, data, research design, data analysis.
10. Which statement below best describes the relationship between the systematic
review of the literature and the literature review?
a. They are the same thing said different ways.

Sciences
b. One is an exhaustive review of everything written, and the other is a highly

synthesized culmination of current studies.
c. They both only utilized previously published peer-reviewed research articles.
d. Both are needed to justify the results section of a research article.
STUDY ACTIVITIES
Now that you know where to locate relevant information in research articles, you can work
on condensing that information and creating a one-paragraph summary of the main points.
This summary will serve as the basis for a critique of the article and also for an annotated
bibliography of all your research. An article critique summarizes the research and the findings
and gives a professional opinion of the article. An annotated bibliography is an alphabetical
listing of all the articles you have located. It contains a bibliographic reference for each
article and a summary of the relevant information. An annotated bibliography allows you to
keep a detailed record of all the articles you have read, which is a great time saver when you
have to go back and locate information and you do not remember exactly where you read it.
An annotation and a critique are similar in that both begin with a summary of the rele-
vant information in the article. Remember, if the article is original research, this paragraph
addresses the four questions: what are they looking to study (purpose); where are they
looking to study it (sampling); how are they looking to find it (methodology); what did they
find (results). If the article is a review of the literature, the first paragraph should identify
the focus, search terms utilized, databases searched, and time line covered. It is important
to summarize all this information into one four-to-five-sentence paragraph; this style of
writing takes practice. Read through the articles you have located a number of times, and
practice summarizing all the information in each section into one cohesive sentence. Use
only the most relevant information. Remember, you are organizing information to make
it easier to find articles that you will go back to when writing your literature review, not
recording all the data from the study.
The second paragraph is where an annotation and a critique differ.
Creating an Article Critique: For an article critique, use the second paragraph to describe
your professional evaluation of the research. Did the methodology, design, and methods
chosen align well with the research question? What were the stated limitations or issues
with the study? Were there any you identified on your own? When writing this paragraph,
practice addressing these issues in the third-person point of view, writing about the
subject, i.e., the research, to keep the focus of the critique professional and not personal.
Creating an Annotated Bibliography: For an annotated bibliography, this paragraph
can serve a number of purposes, or it can be eliminated altogether. Most often, this
paragraph very simply states how this research pertains to the topic being addressed.
For example, does it agree or disagree with your research problem statement? Does it
provide groundbreaking theory-building knowledge on the topic, or does it support
a subtopic of the research purpose? The information included in this paragraph, or
the decision to even have a second paragraph, is at your discretion.

Sciences
REFERENCES
Coughlan, M., Cronin, P., & Ryan, F. (2007). Step-by-step guide to critiquing research. Part 1: Quan-
titative research. British Journal of Nursing, 16(11), 658–663.
Critical Appraisal Skills Programme. (2013, May 31). Critical appraisal skills programme (CASP)—10
questions to help you make sense of qualitative research. Retrieved from http://media.wix.com/ugd/
dded87_9515416 99e9edc71ce66c9bac4734c69.pdf
Cronin, P., Ryan, F., & Coughlan, M. (2008). Undertaking a literature review: A step by step approach.
British Journal of Nursing, 17(1), 38–43.
Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and
associated methodologies. Health Information and Libraries Journal, 26(2), 91–108. doi:
10.1111/j.1471-1842.2009.00848.x
Illinois State University. (2017, January, 18). Types of sources—what’s the difference. Retrieved from
http://guides.library.illinoisstate.edu/comparetypes
Kerlinger, F. N. (1986). Foundations of behavioral research (3rd ed.). San Diego, CA: Harcourt Brace.
Leedy, P., & Ormrod, J. (2016). Practical research: Planning and design (11th ed.). Boston, MA: Pearson
Education.
Merriam-Webster.com. (2017). Critique. Retrieved January 20, 2017, from https://www.
merriam-webster.com/dictionary/critique
Mongan-Rallis, H. (2014, November 21). Guidelines for writing a literature review. Retrieved from http://
www.duluth.umn.edu/~hrallis/guides/researching/litreview.html
Office for Human Research Protections. (2008, October 16). Guidance on engagement of institutions
in human subjects research. Retrieved from Department of Health and Human Services at http://
www.hhs.gov/ohrp/sites/default/files/ohrp/policy/cdebiol.pdf
OxfordDictionaries.com. (2017). Critique. Retrieved January 20, 2017, from https://
en.oxforddictionaries.com/definition/critique
Rutgers, the State University of New Jersey. (2017). Popular literature vs. scholarly peer-reviewed liter-
ature: What’s the difference? Retrieved from https://www.libraries.rutgers.edu/scholarly_articles
Ryan, F., Coughlan, M., & Cronin, P. (2007). Step-by-step guide to critiquing research. Part 2: Qual-
itative research. British Journal of Nursing, 16(12), 738–744.
University of California–Santa Cruz. (n.d.). Write a literature review: Home. Retrieved from http://
guides.library.ucsc.edu/write-a-literature-review
University of Minnesota. (n.d.). Primary, secondary and tertiary sources in the health sciences. Retrieved from
https://hsl.lib.umn.edu/biomed/help/primary-secondary-and-tertiary-sources-health-sciences
Wakefield, A. (2014). Searching and critiquing the research literature. Nursing Standard, 28(39), 49–57.
doi:10.7748/hs.28.39.49.e8867
Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature
review. MIS Quarterly, 26(2), xiii–xxii.
Yates, J., & Leggett, T. (2016). Qualitative research: An introduction. Radiologic Technology, 88(2),
225–231.

Sciences
DEVELOPING THE RESEARCH
PROBLEM STATEMENT,
PURPOSE STATEMENT, AND
RESEARCH QUESTIONS
4
CHAPTER GOALS:
INTRODUCTION
This chapter will focus on the conceptual stage of the research process, which
involves the creation of a problem statement, a purpose statement, research • To provide a working definition of a
question(s) and, when conducting quantitative research, hypotheses that problem statement, purpose state-
define the relationship between variables (Table 4.1). Newman and Covrig ment, research question, hypothesis
(2013) introduce the concept of consistency when considering the logical and variables.
progression of thoughts from the title of a research study through the problem • To explore the relationship between
statement, purpose statement, and research questions. The problem state- a research problem, problem state-
ment has to be in alignment with the title of the study; the purpose statement ment, purpose statement, and
must be in alignment with the problem identified by the researcher; and research question.
lastly, the research question must be worded so it demonstrates alignment • To introduce the FINER and
with both the purpose of the study and the research design. PICOT criteria for writing
research questions.
THE RESEARCH PROBLEM STATEMENT, PURPOSE • To introduce the relationship
STATEMENT, AND RESEARCH QUESTIONS—DEFINED between a methodology and creation
of a problem statement, purpose
The problem statement introduces the importance of the study or, as
statement, and research question.
Newman and Covrig (2013) describe it, the WHY or “so what.” The prob-
lem statement leads to the development of a purpose statement, which is a
statement that clearly and succinctly puts forth the purpose of the study,
which essentially identifies what is to be studied. A research question takes LEARNING OBJECTIVES:
the purpose statement and puts it in the form of a question, which further
refines what will be studied and what the new knowledge will be at the BY THE COMPLETION OF THIS CHAPTER,
conclusion of the study (new knowledge is generated by answering the THE STUDENT WILL BE ABLE TO:
research questions). It is important to note that the problem statement,
purpose statement, and research questions are all written using language • Define the terms indepen-
that is specific for the methodology (i.e., qualitative, quantitative, mixed dent, dependent, mediating, and
methods) and the research design that will be used for the study. moderating variables;
• Recognize the important characteris-
tics of a problem statement, purpose
PROBLEM STATEMENT statement, research question,
What is the difference between the statement of a problem and a prob- hypotheses, and variables;
lem statement? According to Ellis and Levy (2008), a statement of the • Write a purpose statement for each
problem is one or two sentences that outline the problem that the study methodology; and
• Write a research question using the
FINER and PICOT criteria.
75

Sciences
Stages Action Steps
Conceptual Having an idea (research problem), systematically reviewing the literature to verify the
(Thinking) problem has the potential to generate new knowledge, writing a problem statement, a
research purpose statement, and research question(s)
addresses, while the “… problem statement is the statement of the problem and the argu-
mentation for its viability” (p. 27).
The purpose of the problem statement is to help the reader understand why the problem
is important and to clearly articulate the gap in the literature. Meaning, an explanation of
why the proposed study is worth conducting (scientific merit).
During the dissemination stage of research, the researcher should introduce the problem
in a way that catches the reader’s attention, making the reader want to read/learn more. This
part of the statement should be supported by citations from the literature that support the
existence of the problem. This is not a literature review; rather it is an opportunity for the
researcher to focus on previously conducted research as it relates to the current research
being proposed. In other words, the researcher wants to convey to the reader of the article
why this particular problem is important. As the purpose of research is to fill a gap in the
literature, it is also imperative that the researcher identify deficiencies in the literature by
informing the reader what research has been conducted to date and a clear statement of
what is still unknown.
In summary, the problem statement should define what the research problem is, identify
studies from the literature that have addressed the problem, identify deficiencies in the
literature as it relates to the topic, and clearly state what the significance of the study is for
the intended audience. Simply said, it should clearly justify that the topic to be researched
has the potential to generate new knowledge.
PURPOSE STATEMENT
The purpose statement clearly states what the intent of the research study is. In other words,
what is it that the researcher intends to accomplish in the study? One cannot underestimate
the importance of the purpose statement as it is the springboard for all other elements of
the study (i.e., research question, hypothesis).
This is an opportunity for the researcher to clearly state what the significance of the
study is; the person reading the article should be able to answer the questions below from
what is written in the introduction section of a journal article.
• How the study specifically addresses the gap in the literature:

■■ Will it duplicate a current study using a different research design,
data collection tool, or a setting and/or population that has not been
sufficiently investigated?
■■ Will the study contribute to basic knowledge, improve current practice,
develop a new theory, or create new policy?

Sciences
Chapter 4 Developing the Research Problem Statement, Purpose Statement, and Research Questions | 77
Throughout the creation of the purpose statement, it is important to consider the language
used as it relates to the research approach. For instance, if a qualitative approach is being
used, the purpose may be described as exploring a concept or phenomenon. A quantitative
approach may be described as understanding the impact of a variable(s) on an outcome(s),
and a mixed methods approach explicitly states the research purpose for combining both
qualitative and quantitative approaches.
Although the language used to create the purpose statement is specific to the type of
research study performed, there are certain basic components of a purpose statement.
The purpose statement should be written in a clear, concise manner, and this usually
can be accomplished in one sentence. It should be directly related to the research prob-
lem identified and be written in such a way that the reader can tell if it’s a quantitative,
qualitative, or mixed methods study. It should also be written in such a way that it does
not predict the outcome of the study. It should include the research design and the
participants in the study as well as the location at which the research will take place.
Quantitative studies usually examine the relationship between variables, and the variables
must be clearly defined in the purpose statement. Qualitative studies usually explore a
central phenomenon.
Quantitative Purpose Statement
Script
The purpose of this study is to (verb used: i.e., determine/compare/test) the (Independent
Variable) on (Dependent Variable) in (study participants) at (research location).
Example
The purpose of this pretest-posttest control group study is to determine the impact of a
service-learning activity on the development of professional behaviors in first-year health
science students at a research university on Long Island.
Qualitative Purpose Statement
Script
The aim of this study is to (verb used: i.e., understand/explore) the (central phenomenon)
of (population) in (research setting).
Example
The aim of this phenomenological study is to understand the meaning of person-centered
promotion of movement quality physical therapy services to patients with complicated
diagnoses by inviting physical therapists to describe, through interviews, how they pro-
mote movement quality in their clinical settings. This example is adapted from Skjaerven,
Kristofferson, and Gard (2010).
Mixed Methods Purpose Statement

A purpose statement for a mixed methods research study includes both a qualitative and
quantitative purpose statement plus the rationale for mixing the two studies. Mixed methods

Sciences
research is covered in greater detail in Chapter 10, but for now consider the following example
adapted from Pickard, Wainer, Bailey, and Ingersoll’s (2016) research study.
Example
The purpose of this mixed methods research study is to assess the “feasibility and accept-
ability of a telehealth-based parent-mediated intervention for children with autism spectrum
disorder” (p. 845); combining both quantitative and qualitative data will allow a “more
thorough understanding of the variables that might facilitate or impede the use of the two
versions of ImPACT Online in community settings” (p. 847).
THE RESEARCH QUESTION

Lipowski (2008) describes a research question as “… the purpose stated in the form of a
question” (p. 1668). In other words, the research purpose statement and research question(s)
should align with one another. Kwiatkowski and Silverman (1998) added that “the research
question drives the development of the study protocol” and that “… it must be shaped and
narrowed into an answerable format” (p. 1116). Agee (2009) explained that “good questions
do not necessarily produce good research, but poorly conceived or constructed questions
will likely create problems that affect all subsequent stages of a study” (p. 431).
Research Questions in Quantitative Research

Quantitative research questions are nondirectional; they can range from simple to complex,
and studies can have more than one research question (Connelly, 2015). Research questions
should not be written in a yes/no format. In other words, research questions should be
written in such a way that they are not biased with regard to the terminology used or the
position taken on a particular topic.
Consider the following question as an example: To what extent is patient-centered teaching asso-
ciated with self-reported reduction of dietary fat and salt content in individuals who have had an
acute cardiac episode within the past six months? This question does not make an assumption about
the association between patient-centered teaching and self-reported reduction of dietary fat and
salt content; it merely asks to what extent does one variable influence another.
A researcher must fully comprehend what has previously been studied about a particular
topic before being able to further the knowledge on what is already known. It is the research-
er’s familiarity with the subject that assists in delineating an appropriate research question for
a study (Farrugia, Petrisor, Farrokhyar, & Bhandari, 2010), as well as to ensure that the ques-
tion has not previously been answered in the literature or is not important. It is this in-depth
knowledge about the topic that has the potential to generate future research questions.
Kwiatkowski and Silverman (1998) note that clinical experiences provide many opportuni-
ties for the development of research questions, whether it be the testing of a new intervention
for a specific clinical problem or testing an accepted clinical practice for a new indication.
The acronym FINER has often been suggested in the literature as a method to judge
the quality of a research question. Table 4.2 provides detailed information regarding the

Sciences
FINER criteria. These criteria succinctly capture the common qualities that are inherently
important when crafting a well-written research question.
TABLE 4.2 FINER Criteria
F Feasible When writing a research question, the researcher has to consider if the scope of what is being performed
is feasible. For instance, when considering the research question, will there be a sufficient number of
subjects to participate in the study, does the researcher have sufficient technical expertise in the required
area to conduct the study, does the researcher have the time and sufficient funds for the study, and lastly,
is the study manageable with regard to the scope of what is being proposed?
I Interesting Will the answer to this question be of interest to the researcher, the members of the audience to which
the research is directed, and the scientific community at large?
N Novel Will the answer to this question generate new knowledge by confirming previous findings in the litera-
ture (for instance, in a different setting, utilizing a different population, etc.), refuting previous findings
or extending previous findings?
E Ethical Is the question amenable to a study that the Institutional Review Board (IRB) would approve?
R Relevant Will the answer to the question be relevant to scientific knowledge, clinical and health policy issues, fu-
ture research, etc.?
Adapted from Patricia Farrugia, et al., “Box 1. FINER criteria for a good research question,” Canadian Journal of Surgery, vol. 53, no. 4,
p. 279. Copyright © 2010 by Canadian Medical Association.
Farrugia et al. (2010) note that “Whereas the FINER criteria outline the important aspects
of the question in general, a useful format to use in the development of a specific research
question is the PICOT format …” (p. 279).
Table 4.3 illustrates that the acronym PICOT encompasses the population (P), inter-
vention (I), comparison group (C), outcome (O), and time (T). Time in the PICOT formula
describes the period of time over which data collection takes place (Riva, Malik, Burnie,
Endicott, & Busse, 2012). The PICOT format has been shown to be useful in studies that
explore the effect of therapy (Riva et al., 2012).
Example of a Research Question Using the PICOT Criteria

“In adults with chronic neck pain, what is the minimum dose of manipulation necessary to
produce a clinically important improvement in neck pain compared to supervised exercise
at 6 weeks?” (Riva et al., 2012, p. 168).
Variants of the PICOT acronym include PICO and PICo. PICO includes population (P),
intervention (I), comparison group (C), and outcome (O), while PICo includes population
(P), phenomenon of Interest (I), and Context (Co). PICO can be used when time is not a
relevant factor in the study and PICo can be used when there is no outcome or comparison
being made (Stern, Jordan, & McArthur, 2014). As such, the PICo research question format
is only used in qualitative research studies.

Sciences
TABLE 4.3 PICOT Criteria
P Population What population are you studying?
Will your population be:

• comprised of students?
• patients with a particular disease?
• members of a community?
I Intervention (only used for What intervention will you be studying?

interventional studies)
Will you be:
• using a drug to see if it relieves symptoms of a particular disease?
• evaluating a new treatment for an existing disease?
• evaluating the efficacy of a screening test for a disease in a population?
C Comparison Group What is being compared to the intervention?
• Does this group get no intervention or a standard treatment?
O Outcome of Interest What is it that the researcher is looking for?
• This could be a measurement of a study administered, the effect of a particular

treatment, or an observation of a task performed.
T Time When will the outcome be assessed?
Will it be:
• 6 months after the intervention?
• assessed at monthly intervals, over the course of a year, after the intervention?
Adapted from Patricia Farrugia, et al., “Box 2. PICOT Criteria,” Canadian Journal of Surgery, vol. 53, no. 4,” Canadian Journal of Surgery,
vol. 53, no. 4, p. 279. Copyright © 2010 by Canadian Medical Association.
Research Questions in Qualitative Research

According to Mantzoukas (2008), “The content of a qualitative research question needs to
reveal an area of interest or problem emerging from the researcher’s professional and/or
personal experience or the literature, where a gap of knowledge exists or where contradic-
tory and unexplored facets are present” (p. 373).
Qualitative research questions are not as narrow and specific as quantitative research
questions. Rather, they are broad questions focused on “the why and where of human
interactions” (Agee, 2009, p. 432). In other words, researchers are curious about people’s
experiences of an event, events, and/or a condition and seek to uncover the perspectives of
an individual, a group or more than one group (Agee, 2009).
When conducting research, the qualitative researcher cannot know ahead of time what will
be described or what will emerge from the study. As a result, qualitative research questions are
open ended and written in a way that seeks to interpret and/or describe and/or explore how
or why something occurs, rather than seeking to find the relationship between variables. “…

Sciences
The ongoing process of questioning is an integral part of understanding the unfolding lives
and perspectives of others” (Agee, 2009, p. 432). Verbs such as understanding, exploring, and
explaining are used when considering experiences, feelings, perspectives, views, etc. (Mant-
zoukas, 2008). When formulating a qualitative research question, the research design used to
conduct the study should be included in the question (Chapter 6 will provide an in-depth review
of four qualitative research designs). “A good qualitative research question requires to be struc-
tured in such a manner as to successfully convey sufficient information about the topic of the
study, the participants of the study, the context of the study, the time of the study and the way
the study will be conducted” (Mantzoukas, 2008, p. 375). Over the course of a research study,
sub-questions may be added. This may occur if the initial research question was too limited
and precluded the researcher from fully addressing the phenomenon under study (Agee, 2009).
Research Questions in Mixed Methods Research

Mixed methods research questions combine the qualitative and quantitative approaches of
research. Tashakkori and Creswell (2007) put forth three possibilities for writing questions
for mixed methods research. The researcher can “Write separate quantitative and qualitative
questions, followed by an explicit mixed methods question (or, more specifically, questions
about the nature of integration)” (p. 208). A research question written for a study that con-
currently collects qualitative and quantitative data could be “Do the quantitative results and
the qualitative findings converge?” (p. 208). For a sequential study, the research question
could be “How do qualitative results explain (expand on) the experimental outcomes?”
(p. 208). The wording of these questions clearly conveys the purpose for intermingling of
the qualitative and quantitative components of the research study.
Tashakkori and Creswell (2007) provide a second way to write a mixed methods research
question, which can then be broken down into quantitative and qualitative sub-questions
to be answered in the various stages of the study. An example of this type of question in a
parallel or concurrent design would be “What are the effects of Treatment X on the behaviors
and perceptions of Groups A and B?” (p. 208). When broken down, the qualitative question
would be “What are the perceptions and constructions of participants in groups A and B
regarding treatment X?” (p. 208), and the quantitative question would be “What are the
effects of Treatment X on the behaviors of Groups A and B?” (p. 208).
Lastly, research questions can be written as the study evolves; this is especially true for
grounded theory research. If the first phase of the study is qualitative, the research question
would be framed as such, and if the second phase is quantitative, a research question would
be framed as a quantitative question (Tashakkori & Creswell, 2007).
The following two topics, hypotheses and variables, only apply to quantitative
research studies.
Hypotheses in Quantitative Research

“A research hypothesis is a specific statement that predicts the direction and nature of the
results of the study” (Connelly, 2015, p. 435). “… a good hypothesis must be based on a good
research question at the start of a trial and, indeed, drive data collection for the study”
(Farrugia et al., 2010, p. 280).
The hypothesis should be written as a declarative statement and in such a way that it
outlines the relationship (i.e., association, cause and effect) between two or more variables

Sciences
(Farrugia et al., 2010). When we refer to the relationship between two variables, we are usu-
ally referring to an independent variable and a dependent variable, which will be discussed
in the next section of this chapter. Hypotheses can provide direction for the researcher as
to what type of design should be used, what type of data should be collected, and how the
data should be analyzed. The hypothesis is written in such a manner that it is a tentative
declarative statement regarding the findings of the study (the relationship between the
variables). There are several hypothesis formats; this chapter reviews three: null, direc-
tional, and nondirectional. The null hypothesis (often written as H0) predicts there will
be no significant difference or relationship between the variables. The nondirectional
hypothesis predicts there will be a relationship between the variables, but it does not state
what that relationship will be. The directional hypothesis not only predicts there will be a
relationship between the variables, but it states how that relationship will be expressed (e.g.,
as one variable increases, the other will decrease, etc.), ultimately stating what the study
results will be (Creswell, 2009; Jacobsen, 2017; Toledo, Flikkema, & Toledo-Pereya, 2011).
• Null hypothesis example: In a Hobbs, Nahar, Ford, Bass, and Brodell (2014)
study entitled Skin Cancer Knowledge, Attitudes, and Behaviors in Collegiate Athletes,
the researchers wrote four null hypotheses that in combination included the
variables that would be studied. Here they state there will be no significant cor-
relation between any of the variables.
■■ There will be no significant correlation of knowledge with age, race,
and gender.
■■ There will be no significant correlation of attitudes with age, race,
and gender.
■■ There will be no significant correlation of behaviors with age, race,
and gender.
■■ There will be no significant correlation between behaviors and knowledge
(Hobbs et al., 2014, p. 2).
• Nondirectional hypothesis example: Using the Hobbs et al. (2014, p. 2) and the
Winning et al. (2017, p. 754) hypotheses examples, this format clearly expresses
there will be a relationship or association between variables but does not express
what form that relationship or association will take.
■■ There will be a correlation between knowledge, age, race, and gender.
■■ There will be a correlation between attitude, age, race, and gender.
■■ There will be a correlation between behavior, age, race, and gender.
■■ There will be a correlation between behaviors and knowledge.
■■ There will be an association between PTSD symptoms and PA over time.
■■ Severity of PTSD symptoms will be associated with PA in a dose–
response manner.
• Directional hypothesis example: The Winning et al. (2017) study entitled
Post-traumatic Stress Disorder and 20-Year Physical Activity Trends Among Women
explained that reduced physical activity (PA) is associated with increased inci-
dents of heart disease, but to date the relationship between post-traumatic stress

Sciences
disorder (PTSD) and PA had not been studied in a cohort of nurses. It is clear in
how the hypotheses are written the researchers are stating that they expect to
find an inverse relationship between PTSD and PA rates (as PTSD rates go up, PA
rates decrease).
■■ Women who developed PTSD symptoms would show greater decreases
in PA over time than women who experienced no trauma, or trauma but
no symptoms.
■■ Higher versus lower PTSD symptom severity would be associated with greater
reduction in PA in a dose–response manner (Hobbs et al., 2014, p. 754).
How the researcher chooses the hypotheses format is based on several factors: the
researcher’s discipline, findings of previous research studies, and theoretical framework of the
study (Teddlie & Tashakkori, 2009). For example, in medical research the standard is to use a
null hypothesis and an alternative hypothesis (nondirectional). As stated above, the hypoth-
esis is a tentative proposition which, in the end, may or may not be supported by the data
analysis (Leedy & Ormrod, 2016). Table 4.4 provides a summary of the hypotheses format.
TABLE 4.4 Hypotheses Formats
Type Prediction
Null Predicts there will be no significant difference or relationship between the variables
Directional Predicts there will be a relationship between the variables, and states what that relationship will be
Nondirectional Predicts there will be a relationship between the variables, but it does not state what that relationship
will be
Independent, Dependent, and Other Variables in Quantitative Research

Variable is a term frequently used in quantitative research. According to Flannelly, Flan-
nelly, and Jankowski (2014), a variable is “… something that takes on different values; it
is something that varies” (p. 162). Although there are different types of variables, two
commonly used categories are dependent and independent. An independent variable is
presumed to have an effect on a dependent variable and a dependent variable “depends”
on an independent variable.
Let’s say a researcher wanted to study the impact of resistance exercise on bone den-
sity in middle-aged women. Participants were randomly assigned to an exercise program
of resistance exercise or a program that did not include resistance exercise. This example
illustrates a research study seeking to identify a cause-and-effect relationship; resistance
exercise (the cause or independent variable) is presumed to have an effect on bone density
(the effect or dependent variable). This effect would be observed as a difference in the bone
density of women in the resistance exercise group, when compared to the bone density of
women in the other group.

Sciences
Although most quantitative research focuses on independent and dependent variables, two
other types of variables that need to be defined are mediating and moderating. A mediating
variable is a variable that accounts in some part for the relationship between an independent
and dependent variable, or, stated a different way, “… the process by which two variables are
related” (MacKinnon, 2011, p. 2). MacKinnon explains that there are a limited number of
causal relationships in studies with only two variables. The addition of a mediating variable
increases the number of causal relationships as there is the relationship between the indepen-
dent variable and the mediating variable, as well as the relationship between the mediating
variable and the dependent variable. A moderating variable is a variable that “modifies the
form or strength of the relation between an independent and a dependent variable” (p. 2).
These concepts are also true for research that does not seek to identify a cause-and-effect
relationship. Consider Nima, Rosenberg, Archer, and Garcia’s (2013) study involving the
exploration of mediating and moderating variables on depression in university students.
Note that the term affect means self-reported moods and feelings. For example, students were
asked to rate words on a five-point scale, words like “proud, strong” and “afraid, irritable”
to measure positive and negative affect (p. e3). Two of the research questions that guided
this study are listed below (p. e3).
1. To investigate if anxiety mediated the effect of stress, self-esteem, and affect
on depression.
2. To examine moderation effects between anxiety, stress, self-esteem, and affect
on depression.
The researchers investigated whether anxiety (the mediating variable) mediated the
effect of stress, self-esteem, and positive or negative affect (independent variables) on
depression (the dependent variable). It was found that anxiety mediated the effect of stress
and self-esteem (independent variable) on depression (dependent variable). However, when
looking at moderating effects among all the variables, there was an “interaction between
stress and negative affect and between positive and negative affect influenced self-reported
depression symptoms” (p. e7). So while “anxiety partially mediated the effects of stress
and self-esteem on depression” … “there was a significant interaction between stress and
negative affect, and positive affect and negative affect on depression” (p. e7). These results
(the combination of examining the mediating and moderating effects on variables) provided
a much more in-depth understanding of the relationship among independent variables and
of their combined impact on depression.

This chapter introduced the problem statement, purpose statement, research questions,
hypotheses, and variables, as well as their relationship to quantitative, qualitative, and mixed
methods research. Once a researcher has identified a researchable problem, the crafting of
a problem statement clearly explains the gap in the literature, which leads to the creation of
a purpose statement to clearly describe the intent of the study. The purpose statement can
then be written in the format of a research question, which then drives the development of a

Sciences
study protocol. In quantitative research studies, hypotheses can be written in a format that
outlines the impact of variables on one another (e.g., the impact of an independent variable
on a dependent variable). The next chapter will explore how decisions a researcher makes
in terms of the methods (sampling, data collection, data analysis) have a direct impact on
the validity, reliability, or trustworthiness of a study’s results/findings.
STUDY PRACTICE
Students should now be familiar with the purpose and format of the research problem,
purpose statement, and research question as well as hypotheses and variables. At this time,
given a summary of a research problem, students should be able to write a research problem
statement, purpose statement, and research question that is appropriate for the methodology
(qualitative, quantitative, mixed methods) of the proposed study. In addition, given a hypoth-
esis, students should be able to ascertain the direction (null, directional, nondirectional)
of the hypothesis. Lastly, students should be familiar with the characteristics of variables
(independent, dependent, mediating, and moderating). Revise your notes/flash cards made
in Chapters 1–3 to include the new information presented in this chapter.
The important new terms/concepts presented in this chapter include:
ɠɠ Problem statement ɝɝ mediating variable
ɠɠ Purpose statement ɝɝ moderating variable
ɠɠ Research questions
ɠɠ Hypothesis
ɠɠ FINER
ɝɝ null hypothesis
ɠɠ PICOT
ɝɝ directional hypothesis
ɠɠ Variables
ɝɝ nondirectional hypothesis
ɝɝ dependent variable
ɝɝ independent variable

1. There will be no differences in the diagnostic accuracy between the imaging tests
dual emission computerized tomography and X-ray in the diagnosis of gout.
This is an example of a:
a. problem statement
b. nondirectional hypothesis
c. null hypothesis
d. directional hypothesis
2. Which of the following is the best example of a research question containing
all parts of the PICOT format?
a. In cardiac clinic patients who have smoked for over 20 years, how does a smoking
cessation program that uses guided imagery compare to one that uses acu-
puncture influence the rate of self-reported tobacco use measured at 6 months
following the conclusion of the program?

Sciences
b. Of 2014 freshmen enrolled in a northeastern university, how will mandatory

seminars that include a 1-hour talk with the campus nutritionist compare to
seminars that only provide a handout about the campus nutrition program, on
the rates of visits to see the nutritionist?
c. In patients aged 65–70+, which variable(s) (gender, age, access to health care)
have the highest degree of association with blood pressure medication compli-
ance over a 6-month period?
d. How does patient-centered teaching improve health outcomes (medication and
cardiac diet compliance) in patients who have been recently discharged (within
the past 6 days) from the hospital into home health care services?
3. The research question is “What impact does massage therapy have on the satisfaction
rate of women who have had a cesarean delivery?”
Which of the following is/are the dependent variable(s)?
a. Massage therapy
b. Satisfaction rate
c. Massage therapy and satisfaction
d. Satisfaction and type of delivery
4. What is the difference between a null hypothesis and a directional hypothesis?
a. One is a declarative sentence; the other is a question.
b. One assumes a relationship; the other denies that one exists.
c. One is researchable; the other is statistical.
d. One includes at least two variables; the other does not.
5. In the example below, which of the following is the mediating variable?
Stress Depression
Social
Support
a. Stress
b. Social support
c. Depression
d. There is no mediating variable
6. Research problem statements:
a. can only be identified by experts in the field after a consultation with peers.
b. clearly and completely identif y how the topic under study will
advance knowledge.
c. typically compare two sets of data to show a relationship or synthesize infor-
mation in a novel way.
d. are the scientific merit statement for the study and as such must be written in
a yes/no format.

Sciences
7. Cancer centers that include alternative healing services (e.g., reiki) will have a reduc-
tion in patients’ self-reported chemotherapy pain compared to cancer centers who
do not offer alternative healing services.
b. research question
c. null hypothesis
8. There are several studies that have documented the benefits of utilizing alternative
healing services with physical therapy patients’ chronic pain. However, it is cur-
rently unknown whether integrating alternative healing services (e.g., reiki) into
routine cancer service will improve patients’ pain and discomfort while undergoing
chemotherapy treatment.
b. research question
c. null hypothesis
9. What is the effect of adding alternative healing services (e.g., reiki) for patients
receiving chemotherapy treatment at the Mayo Clinic, on the reduction of
self-reported pain and discomfort associated with chemotherapy treatment?
This is an example of what type of research question?
a. PICOT
b. Quantitative
c. Qualitative
d. None of the above; it’s a hypothesis.
10. Does the number of hours playing video games increase the risk of type 2 diabetes
in teenage boys?
a. nondirectional hypothesis
b. poorly written research question
c. correctly written research question
d. purpose statement
STUDY ACTIVITIES
The following scenarios are meant to test yourself without your notes. This will help you
determine if there are concepts you are still struggling with. Find the evidence you need to
identify the methodology being described and, based on the information provided in this
chapter, write a problem statement, purpose statement, and at least one research question
for both cases below. Remember, there must be alignment between the methodology and

Sciences
purpose statement, problem statement, and the research question(s). In the literature sum-
mary, find scientific evidence to develop your problem statement.
Case 1: Summary of Literature Review

Night-shift nurses are critical to the delivery of health care as they are responsible to manage
and care for patients throughout the night. As such, it is important to understand the expe-
riences that give meaning to their work, given the fact that they oftentimes work under
difficult conditions. Nurses working the night shift often experience fatigue; there is less
access to expert advice (i.e., supervisors) and involvement with managers, and for the most
part an absence of clinical leadership. Nurses on the night shift often express that they feel
disconnected from the organization and professional activities. As a result, they are dissat-
isfied with their work, which may in turn have an adverse effect on their work. A review of
the literature revealed a paucity of information regarding the challenges nurses face when
working the night shift. Scenario adapted from Powell (2013).
Write a problem statement:
Write a purpose statement:
Write one research question:
Case 2: Summary of Literature Review

Skin cancer is one of the most commonly diagnosed cancers in the United States, with mela-
noma the most commonly diagnosed skin cancer. The greatest risk for skin cancer is exposure
to ultraviolet radiation (UVR), and to reduce exposure to UVR, skin cancer organizations

Sciences
recommend preventive measures such as using sunscreen, wearing sun-protective clothing,

and avoid being in the sun between 10 am and 4 pm when UVR is at its highest level.
Outdoor athletes are at an increased risk of skin cancer due to their prolonged exposure
to UVR during outdoor practices and competitions. They develop the more common types
of cancer (basal cell and squamous) as well as melanoma, a potentially life-threatening
cancer. Outdoor athletes tend to not use sunscreen because they feel it may interfere with
their performance and uniforms, (even if they are sun protective), and do not cover the
areas that are most commonly exposed to the sun (i.e., face, arms, legs).
Write a problem statement:
Write a purpose statement:
Write one research question:
REFERENCES
Agee, J. (2009). Developing qualitative research questions: A reflective process. International Journal
of Qualitative Studies in Education, 22(4), 431–447.
Connelly, L. M. (2015). Understanding research. Research questions and hypotheses. Medical Surgical
Nursing, 24(6), 435–436.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd
ed.). Thousand Oaks, CA: SAGE Publications.
Ellis, T. J., & Levy, Y. (2008). Framework of problem-based research: A guide for novice researchers
on the development of a research-worthy problem. Informing Science: International Journal of an
Emerging Transdiscipline, 11, 17–33.

Sciences
Farrugia, P., Petrisor, B. A., Farrokhyar, F., & Bhandari, M. (2010). Practical tips for surgical research.
Research questions, hypotheses and objectives. Canadian Journal of Surgery, 53(4), 278–281.
Flannelly, L. T., Flannelly, K. J., & Jankowski, K. R. (2014). Independent, dependent, and other vari-
ables in healthcare and chaplaincy research. Journal of Health Care Chaplain, 20(4), 161–170. doi:1
0.1080/08854726.2014.959374
Hobbs, C., Nahar, V. K., Ford, M. A., Bass, M. A., & Brodell, R.T. (2014). Skin cancer knowledge, atti-
tudes, and behaviors in collegiate athletes. Journal of Skin Cancer, 2014, 1–7.
Jacobsen, K. J. (2017). Health research methods. A practical guide. Burlington, MA: Jones & Bartlett
Learning.
Kwiatkowski, T., & Silverman, R. (1998). Research fundamentals II. Choosing and defining a research
question. Academic Emergency Medicine, 5(11), 1114–1117.
Leedy, P. D., & Ormrod, J. E. (2016). Practical research, planning and design. Upper Saddle River, NJ:
Pearson Education.
Lipowski, E. E. (2008). Developing great research questions. American Society of Health-Systems
Pharmacists, 65(17), 1667–1670. doi:10.2146/ajhp070276
MacKinnon, D. P. (2011). Integrating mediators and moderators in research design. Research on Social
Work Practice, 21(6), 675–681. doi:10.1177/1049731511414148
Mantzoukas, S. (2008). Facilitating research students in formulating qualitative research questions.
Nurse Education Today, 28(3), 371–377.
Newman, I., & Covrig, D. M. (2013). Writer’s forum—Building consistency between title, problem
statement, purpose & research questions to improve the quality of research plans and reports.
New Horizons in Adult Education and Human Resource Development, 25(1), 70–79.
Nima, A. A., Rosenberg, P., Archer, T., & Garcia, D. (2013). Anxiety, affect, self-esteem, and
stress: Mediation and moderation effects on depression. PloS ONE, 8(9), e1–e8. doi:10.1371/
annotation/49e2c5c8-e8a8-4011-80fc-02c6724b2acc
Pickard, K. P., Wainer, A. L., Bailey, K. M., & Ingersoll, B. R. (2016). A mixed-method evaluation of
the feasibility and acceptability of a telehealth-based parent-mediated intervention for children
with autism spectrum disorder. Autism: International Journal of Research & Practice, 20(7), 845–855.
doi:10.1177/1362361315614496
Powell, I. (2013). Can you see me? Experiences of nurses working night shift in Australian regional
hospitals: A qualitative case study. Journal of Advanced Nursing, 69(10), 2172–2184.
Riva, J. J., Malik, K. M., Burnie, S. J., Endicott, A. R., & Busse, J. W. (2012). What is your research
question? An introduction to the PICOT format for clinicians. Journal of the Canadian Chiropractic
Association, 56(3), 167–171.
Skjaerven, L. V., Kristofferson, K., & Gard, G. (2010). How can movement quality be promoted in
clinical practice? A phenomenological study of physical therapist experts. Physical Therapy, 90(10),
1479–1492. doi:10.2522/ptj.20090059
Stern, C., Jordan, Z., & McArthur, A. (2014). Developing the review question and inclusion criteria.
American Journal of Nursing, 114(4), 53–56.
Tashakkori, A., & Creswell, J. W. (2007). Exploring the nature of research questions in mixed methods
research. Journal of Mixed Methods Research, 1(3), 207–211.
SAGE Publications.
Toledo, A. H., Flikkema, R., & Toledo-Pereyra, L. H. (2011). Developing the research hypothesis. Journal
of Investigative Surgery, 24(5), 191–194. doi:10.3109/08941939.2011.609449
Winning, A., Gilsanz, P., Koenen, K. C., Roberts, A. L., Chen, Q., Sumner, J. A., … Kubzansky, L. D.
(2017). Post-traumatic stress disorder and 20-year physical activity trends among women. American
Journal of Preventive Medicine, 52(6), 753–760. doi:10.1016/j.amepre.2017.01.040

Sciences
SELECTING METHODS TO
INCREASE RIGOR: PLANNING
A STUDY WITH VALIDITY
AND RELIABILITY OR
5
TRUSTWORTHINESS CHAPTER GOALS:
• To provide working definitions of the

INTRODUCTION terms validity and reliability.
This chapter builds on an overarching concept first presented in Chapter 1, • To explore the relationship between
scientific merit. All researchers must consider issues of validity, reli- methods and validity, reliability,
ability, or trustworthiness when answering the question: Has this study and trustworthiness.
been planned in such a way that the potential to generate new knowl- • To introduce divergent practices
edge exists? Also presented in Chapter 1 was an adaption of Whittemore and language, across methodologies,
and D’Eramo Melkus’s (2008) description of research as a five-phase associated with the concepts of valid-
process: Conceptual, design, empirical, analytic, and dissemination. ity, reliability, and trustworthiness.
Understanding the divergent practices across methodologies associated
• To explore the differences and
with increasing the validity, reliability, or trustworthiness is an essential
common practices between methods
step in assisting the researcher in selecting the most appropriate design
across methodologies.
and methods to conduct the study. The material in this chapter adds
important information to the design, empirical, and analytic phases,
which are illustrated in Table 5.1.
LEARNING OBJECTIVES:
VALIDITY AND RELIABILITY—DEFINED
Understanding the concepts of validity and reliability and how they THE STUDENT WILL BE ABLE TO:
relate to a research study is another factor that goes into designing a
study with scientific merit. • Define validity, reliability,
and trustworthiness;
• Differentiate among research
Reliability is a very complex concept. For the purpose of this book,
practices across methodologies to
it will be defined as issues related to the soundness of the data
enhance rigor; and
collection procedures.
• Compare the various methods used
Validity is the authenticity of the results. There are several types of
across methodologies.
validity a researcher must consider; here we will limit the discus-
sion to two types of validity:
External validity: the extent to which the results of the study
will be true for different groups of people or similar people
in different settings. Meaning, a researcher conducts a study
91

Sciences
Phase Action Steps
Design (Planning) Selecting the best research design and research methods (sampling, data collection, data analysis)
that align with research question(s), submitting the research study proposal for IRB review
Empirical (Doing) Obtaining IRB approval to conduct the study, recruiting/selecting the sample, obtaining informed con-
sent from participants, collecting the data
Analytic (Analyzing) Utilizing the best analysis strategies to yield meaningful results from data (answering the research
questions and simultaneously uncovering new avenues of inquiry)
with a smaller group of people with the hopes of then applying those results to
much larger groups of people. A researcher’s decisions about research design
and sampling method will impact the degree of external validity in each study.
Internal validity: the extent to which the results of the study are true. That is, when
a researcher conducts a study, they want to make sure the results they get are
because the intervention worked instead of as a result of some confounding
variable. A confounding variable is a variable the researcher is unaware of and
that has an impact on the outcome of the study. A researcher’s decisions about
research design, sampling method, and data collection method will impact the
degree of internal validity a study has.
In research, there can be many threats to internal validity that a researcher must consider
and attempt to control for to the best of their ability. The very simplified definitions below
have been summarized from Cook and Campbell’s (1979, pp. 50–64) very detailed expla-
nation of these concepts.
The results of a research study might be influenced by the following threats to validity:
History: an outside event that occurred during the research study that can impact the
results of the study.
Controlling for this threat: Select a research design that includes a control group
(more detail in Chapter 7).
Maturation: time passed and the participants grew “older, wiser, stronger, [or] more
experienced” (p. 52).
Controlling for this threat: Select a research design that includes a control group.
Testing: the participants simply get better at taking the test because they have become
familiar with it.
Controlling for this threat: Select a testing instrument (data collection tool)
that has very high validity and reliability (more on this in the next section of
this chapter).

Sciences
Chapter 5 Selecting Methods to Increase Rigor: Planning a Study with Validity and Reliability or Trustworthiness | 93
Instrumentation: changes in accuracy of measurements from the start to conclusion of

study. For example, the individuals collecting the data become more experienced at
collecting data as the study progresses, or the instrument is not sensitive enough
to detect actual changes.
Controlling for this threat: Select an instrument (data collection tool) that has
very high validity and reliability.
Statistical regression: participants with extremely high scores on a first test tend to
perform lower on second test, and those with extremely low scores on first test tend
to perform better on second test.
Controlling for this threat: Select a data collection tool that has very high validity
and reliability.
Placebo effect: the participants’ or researchers’ expectations that something will work
can impact the results of the study.
It has been well documented that simply the expectation that something will
work can in the short term cause self-perceived change. For example, in one
study, all participants (Waber, Shiv, Carmon, & Ariely, 2008) were given the
same placebo pill and told it was a new pain relief drug. After receiving a mild
shock to the wrist, the participants were told to rate the effectiveness of the new
pain relief drug. The group that was told the pill was very expensive ($2.50 per
pill) rated pain relief properties of the pill as significantly higher than the group
that was told the pill was inexpensive (10 cents per pill). This change occurred
despite the fact that both groups were given the same placebo pill.
In another study, it was found that expectations of change caused short-term
physiologic changes in the brain (Mayberg et al., 2002). The researchers found
similar brain activity in participants who received a placebo for depression (as
measured by a Positron Emission Test, or PET scan) as those given the drug. The
placebo effects lasted for only about 3 weeks; then it was clear the active drug
outperformed the placebo in relieving the symptoms of depression.
Controlling for this threat: Using techniques known as double-blind and/or
placebo-controlled. Using both in combination increases the validity of the
study. Note: The participants must be told during the informed consent process
(Chapter 2) that they will be randomly assigned to either an experimental or
control group, but they will not find out which group they were in until the end
of the research study.
■■ Double-blind: Neither the researchers interacting with participants (to
enroll participants, collect data, etc.) nor the participants know who is in
the experimental or control groups. Only one person (who has no interac-
tion with participants) has access to the code book revealing who is in the
experimental group and who is in the control group.
■■ Placebo-controlled: The experimental group gets the real treatment and
the control group gets a fake treatment. For example, in a research study
on the efficacy of a new drug, the experimental group would receive the
new drug and the control group would receive an inactive substance that
looks similar to the new drug.

Sciences
Hawthorne effect: research participants will change their behavior simply because they
know they are being observed.
Controlling for this threat: Select a research design that includes a control group
and a placebo. In observational studies (more detail in Chapter 8), conduct sus-
tained observations and employ unobtrusive observation methods.
Selection bias: how the researcher selects people to participate in the study can impact
the results of the study. Meaning, a researcher must consider whether there are
differences between participants in the experimental and control groups that influ-
ence the outcome of the study. Or are there some inherent differences between the
population and those who volunteer to participate in the study (the sample) that
impact the validity of results.
Controlling for this threat: Use one of the probability sampling methods (more
detail on sampling later in the chapter).
Attrition (also known as lost to follow-up): people leaving the research study for a variety
of reasons (e.g., loss of interest, death, moving out of the area, etc.) can influence
results of a study, especially a clinical drug trial.
Controlling for this threat: There is no agreed-upon technique; researchers
can use a variety of techniques including increasing the size of the sample to
compensate for lost to follow-up.
TRUSTWORTHINESS—DEFINED
Validity and reliability as defined in the previous section are exclusive to quantitative research.
The primary goal is to control for factors and variables that could negatively impact the
results of the inquiry. Leung (2015) aptly describes the inherent differences between qual-
itative and quantitative inquiries:
While human emotions and perspectives from both subjects and researchers are considered undesir-
able biases confounding results in quantitative research, the same elements are considered essential
and inevitable, if not treasurable, in qualitative research as they invariably add extra dimensions
and colors to enrich the corpus of findings (p. 324).
SIDE BOX 5.1 Remember paradigms from Chapter 1:

Practices that are unacceptable in one methodology are acceptable practices (sampling, data collec-
tion, and data analysis) in the other.
That doesn’t mean that qualitative researchers are not concerned about the rigor of
their study. Rather, it means the standard practices of reliability and validity used in
quantitative research are unsuitable for qualitative inquiries. Guba and Lincoln (1994)
proposed parallel qualitative practices to establish the Trustworthiness of the findings:
Transferability, credibility, dependability, and confirmability. The following definitions

Sciences
are summarized from Merriam’s (2009) very detailed explanation of these concepts
unless otherwise cited.
Instead of the quantitative concept of external validity, qualitative researchers consider
issues of:
Transferability: Merriam explains that the reader of the research study determines
the extent to which findings can be transferred to their settings or group. In other
words, the person who reads the research determines if the findings of the study
are a good “fit” to their situation (p. 226).
One way that helps the reader make this determination is to have a lot of detailed
information about the participants, the research setting, and findings in the
journal article. This practice is known as providing the reader a thick description.
Consider this abbreviated (200 words) example below of Purves, Logan, and Marcell’s
(2011) thick description (over 900 words) of the participant in their case study on the appli-
cation of a software program to facilitate communication. The thick description helps the
reader determine whether the findings of this study will be useful (transfer) to their situation.
Don’t focus on the technical aspects (e.g., CVA, aphasia, concomitant apraxia of speech),
which would only make sense to a speech/language pathologist. Rather, focus on how the
description provides a well-developed feel for who the participant (Skip) in the study was.
Skip … in his fifties, suffered a CVA resulting from trauma to his left carotid artery sustained in a
motorcycle accident in 2001. Auditory comprehension is relatively well preserved, although Skip does
sometimes request repetition (for example, to verify times and meeting places for appointments) …
Skip’s spontaneous verbal output is mostly limited to single words and short high frequency phrases
that he uses regularly in everyday life (e.g., “how are you,” “no words”) … He is acknowledged as a
powerful public speaker—even if he can’t actually speak very well … he has taken on public speaking
engagements at conferences … and, on one occasion, travelling by motorcycle around the country
to … give interviews to local newspapers in order to raise public awareness of aphasia …” (p. 692).
Instead of the quantitative concept of internal validity, qualitative researchers consider

issues of:
Credibility: the confidence in the truth of the findings.

There are numerous data collection and data analysis strategies a researcher can use
to increase the credibility of the findings. Two of these strategies are presented below.
Triangulation: collecting different types of data (verbal, textual, images, observa-
tion), collecting data at different times, and/or having two researchers collect
and analyze data. An example of triangulation can be found in the Johnson et
al. (2017) study of paramedics’ decision making. The researcher’s data collec-
tion included: “document review, interviews, observation, digital diaries, focus
groups and workshops,” which the researchers found yielded “different ways
of seeing reality, yet similar issues were highlighted in data generated by each

Sciences
method” of data collection (p. 10). Looking at many different types of data helps
the researcher have confidence that they have uncovered all the data required
to gain a complete understanding of the research question.
Member checking: sharing the data analysis with the research participants and/or
experts in the field working with participants. That is, when the researcher begins
to see meaning develop from the data, the researcher asks the participants if the
meaning revealed from the data is true. This practice assists the researcher in
uncovering any hidden bias the researcher might have or the researcher’s misin-
terpreting the meaning of the data. An example of this practice can be found in
the Hasankhani et al. (2014) study. During data analysis, “Preliminary findings
of the study were shared and discussed with academics in the field and clinicians
who had clinical experience in providing care to angioplasty patients” (p. 145).
Instead of the quantitative concept of reliability, qualitative researchers consider issues of:
Dependability: dependability relies on whether the results of the study make sense to
another researcher. Here the practice is not for a researcher to replicate the study and
achieve identical results; rather it is to ask the question “are the results consistent
with the data collected” (Merriam, 2009, pp. 221–223).
One way to accomplish this is through an audit trail. An audit trail is a detailed
reporting of how the researcher conducted the study, especially the collection and
analysis of the data. This reporting can include the researcher sharing reflections,
problems, insights they had during the collection and analysis of data (p. 223).
Decision on the types of methods to use in a study has a great impact on the rigor of both
quantitative and qualitative studies. Designs will be covered in subsequent chapters, but
for now a more detailed understanding of methods and the nature of data are warranted.
METHODS: SAMPLING
In Chapter 1, sampling was simply defined as the way a researcher recruits or selects indi-
viduals to be in the study. If sampling is the technique a researcher uses to recruit/select
individuals for the study, then what is the sample?
Plichta and Kelvin (2013) define the term sample as: “A group selected from a population in
the hopes that the smaller group [the sample] will be representative of the entire population”
(p. 463). A new term is introduced in this definition: population. A population is a group
that shares a common characteristic as defined by the researcher. Consider this example: A
researcher seeks to understand the percentage of people living with cardiovascular disease
(referred to simply as heart disease in the rest of the chapter) who adhere to their prescribed
cardiac diet (e.g., limit salt and sugar, avoid saturated fat, include more fruit and vegetables
in diet). In this case, heart disease would be the common characteristic.
According to the Centers for Disease Control and Prevention (CDC, 2017) the number
of adults with diagnosed heart disease in the United States is 27.6 million. It would
be impossible to conduct a study with the entire population (27.6 million adults with

Sciences
diagnosed heart disease); it is simply not feasible (FINER, Chapter 4). Additionally,
since there is no master list of all 27.6 million adults with diagnosed heart disease, the
researcher does not have access to this population. Therefore, a researcher needs to find
a population of adults diagnosed with heart disease they have access to and then sample
from that population. For example, if the researcher is working in a large multi-hospital
health system that serves a very diverse community, with approval they could sample
from that population.
Inclusion and Exclusion Criteria

Another important consideration in defining the population and determining who from
that population will be an acceptable participant for a study is the researcher’s inclusion
criteria and exclusion criteria.
Inclusion criteria are used to determine who is suitable to be a participant in the research
study. The inclusion criteria are based on certain characteristics that are defined by
the researcher based on the purpose of the research study.
Example: A researcher is conducting a study of a new high blood pressure medi-
cation for adults with heart disease. The researcher knows that not all individuals
with heart disease require blood pressure medication and that not all individuals
on blood pressure medication have heart disease. So, the researcher would only
enroll (include) adults with heart disease and high blood pressure that requires
medication. For this example, the inclusion criteria would be listed as:
Inclusion criterion = A diagnosis of high blood pressure at a level that must be
controlled by mediation and being 18 years or older living with heart disease.
Exclusion criteria are used to determine participants who have met the inclusion criteria
(18 years or older with a diagnosis of high blood pressure that requires medication)
but should not be included in the study.
Example: A study of a new medication for adults with high blood pressure might
exclude pregnant women from the study if the potential harm the new medication
has on fetal development is unknown. It must be noted, this is only an example.
Pregnancy or the potential to become pregnant does not automatically exclude
women from being participants in all research drug studies. For this example,
the exclusion criteria would be listed as:
Exclusion criteria = Pregnancy or refusal to take birth control measures for the
duration of the study.
Godwin et al. (2003) explain that a researcher’s inclusion and exclusion criteria, espe-
cially in clinical drug trials, directly impact the study’s internal and external validity. Strict
inclusion criteria increase the study’s internal validity (dependability of the results) but can
decrease the study’s external validity (generalizability). Meaning, if the inclusion/exclusion
criteria are too strict, they may limit the population to whom the results of the study can be
generalized. Balancing factors that can help address these issues are limiting the exclusion
criteria, using randomized control groups, and the use of double-blind whenever possible.

Sciences
For example, Table 5.2 illustrates the inclusion/exclusion criteria that Fogel et al. (2015, p.
803) used in their study. This study tested the efficacy of a behavioral intervention program
to reduce HIV and sexually transmitted infection rates among incarcerated women. The
only exclusion criteria in the study is one that the researchers believed would negatively
impact the validity of the results.
TABLE 5.2 An Example of Inclusion and Exclusion Criteria
Inclusion Criteria Exclusion Criteria
1. 18 years or older. 1. Documented symptoms of acute psychosis.

2. English speaking.
3. HIV negative status.
4. Planning to live in the local area for the length of the study.
5. Having had or expecting to have sexual activity with a man.
6. Being able to provide verbal and written consent.
7. Newly incarcerated women who had a sentence length of only
12 months, with less than 6 months of their sentence left to serve.
Source: Catherine Fogel, et. al., Efficacy of an Adapted HIV and Sexually Transmitted Infection Prevention Intervention for Incarcerated
Women: A Randomized Controlled Trial. American Journal of Public Health, vol. 105, no. 4, p. 803. American Public Health Association, 2015.
Probability and Non-probability Sampling Methods
SIDE BOX 5.2 Random sample should not be confused with random assignment.
Random assignment = everyone in the random sample has an equal chance of being assigned to either
the experimental or control group (more in Chapter 7).
How to sample is another choice facing the researcher. There are two main techniques a
researcher can use to obtain the sample; each of these techniques has varying degrees of
external validity. These sampling techniques are: probability and non-probability methods.
Probability methods allow the researcher to obtain a random selection of individuals

from the population. When a researcher uses one of the probability methods, they
have obtained a random sample. Probability sampling methods are thought to yield
a sample that statistically is representative of the population. This practice yields
a study with a high level of external validity; the results of the study can be gener-
alized to other groups of people or similar groups of people in a different setting.
It also increases internal validity because statistically it controls for selection bias.
Using the previous example, a researcher working in a large multi-hospital health system
that serves a very diverse community has a master list of the thousands of people receiving
treatment and services for heart disease in that health system. There are different probability
methods the researcher could use to obtain a sample from the population.

Sciences
Note: It is assumed in all examples that the researcher obtains informed consent from
all individuals.
Probability sampling methods include:
Simple random: everyone in the population has an equal chance of being selected as
a participant.
Example: It is determined that the study would need 500 participants. A com-
puter randomly selects 500 names (the sample) from the master list of the
thousands of people receiving treatment and services for heart disease in the
health system (the population).
Stratified random: the researcher identifies a subgroup or subgroups in the population and
wants to ensure that the sample represents the subgroup(s) found in the population.
Example: The researcher needs 500 participants for the study but realizes that
race or ethnic identity is an important consideration for the study. The researcher
would divide the master list of the thousands of people receiving treatment and
services for heart disease in that health system into subgroups (also known as
strata) by race/ethnicity and then randomly select an equal amount from each
subgroup to get to a representative sample of the population.
Proportional stratified: the researcher identifies a subgroup or subgroups in the popula-
tion that are very unequal in size and wants to ensure that the sample will represent
the population.
Example: The researcher needs 500 participants for the study but realizes that
race and ethnic identity are not equally distributed across the thousands of
people receiving treatment and services for heart disease in that health system
(White, 25%; Black/African American, 35%; Asian, 10%; Hispanic/Latino, 30%).
The researcher would divide the population into subgroups by race/ethnicity
and then randomly select the same percentages as found in the population to
get to a representative sample.
Systematic: the researcher selects participants based on a randomly chosen number.
Example: Starting at a random point in the master list of the thousands of people
receiving treatment and services for heart disease in that health system, the
researcher would randomly select a number and sample every nth person. If the
random number is 23, then starting at a random point in the list, the researcher
would select every 23rd person from the list to get a representative sample.
This would continue until the 500 participants needed for the study have been
identified. Another example: A researcher and their research assistants stand
outside the numerous heart disease treatment clinics within the multi-hospital
health system and the researcher asks every tenth person exiting the clinic to
complete a survey (every tenth person would be the representative sample).
Cluster: selecting an intact homogeneous group(s) from within the population.
Example: The researcher breaks down the master list of the thousands of people
receiving treatment and services for heart disease in that health system by the
various heart disease treatment clinics in the health system. The researcher

Sciences
sees that the patients in each of the 20 clinics are very similar, so instead of
randomly selecting participants from each clinic, the researcher randomly selects
5 entire clinics to serve as the sample. Meaning, only those patients receiving
services in those 5 randomly selected clinics are selected to serve as the sample
of the population.
The difference between stratified and cluster sampling is often confusing. This distinction
in some cases can be attributed to the researcher’s research question and the variables that
a researcher believes is important. Said another way, how does the researcher view the pop-
ulation? Is the population seen as one homogeneous group with naturally occurring smaller
homogeneous groups or as discrete subgroups within one diverse population? Consider the
comparison examples in Table 5.3.
TABLE 5.3 Comparing Stratified and Cluster Sampling
Example: Stratified Example: Cluster
Population All residents of XYZ County All residents of XYZ County
Subgroups 9 zip codes within the county 9 zip codes within the county
How researcher views Vast differences in socioeconomic status (SES) They are all residents of the county. Based on
population exist across the 9 zip codes. There are several zip research question differing levels of SES does
codes where the majority of residents are wealthy. not impact research; therefore, the researcher
Some zip codes where the majority of residents fall determined there is no difference between the
in the middle-income range while in the remaining 9 zip codes.
zip codes the majority of residents are living below
poverty line.
How researcher ob- Divide zip codes into subgroups by SES status: Randomly select 3 zip codes.
tains the sample wealthy, middle income, and below poverty line. Ran-
domly select 1000 people from each SES subgroup.
Sample Randomly selected 1000 people from each Everyone living in the 3 randomly selected zip
SES subgroup to represent the entire county codes represents entire county (population).
(population).
Sometimes it is either impossible or unethical to use probability sampling methods.

Non-probability methods utilize approved sampling procedures, but since the sampling
procedures do not include random selection, there is no way to guarantee that the selected
individuals (the sample) will be representative of the population. In other words, there
might be unknown differences between those selected/those who volunteer to be in the
study and the population. Utilizing one of the non-probability sampling methods yields a
quantitative study with low levels of external validity—meaning the results of the study
might not generalize to other groups of people or similar groups of people that are in a
different setting. This type of sampling also impacts the study’s internal validity; see the
previous section on selection bias.

Sciences
In terms of qualitative studies, external validity does not apply. These non-probability
methods allow the researcher to carefully select a small number of individuals that are
best able to answer the research question. Remember, the purpose of qualitative research
does not include finding statistically generalizable results or proving that one intervention
is better than another.
Using a variation on the previous example, the researcher is working in a large
multi-hospital health system that serves a very diverse community, but the researcher
does not have access to a master list of all the people receiving treatment and services for
heart disease within that health system. There are different non-probability methods the
researcher could use to obtain a sample from the population. Note: It is assumed in all
examples that the researcher obtains informed consent from all individuals.
Non-probability sampling methods include:
Convenience: the selection of individuals that the researcher has easy access to.
Example: The researcher works at two of the heart disease clinics in the health
system. The researcher would ask the patients they know/are working with to
be in the study since the researcher has convenient access to this group. If the
study design required two groups, the researcher could make all the patients
(who consent) in clinic A the experimental group and all patients (who consent)
in clinic B the control group.
Quota: the researcher needs to fill discrete groups at a predetermined number of par-
ticipants. The groups have certain characteristics that are needed to answer the
research question.
Example: The researcher is conducting a study on women with heart disease. The
researcher posts flyers in all the heart disease clinics within the health system,
inviting women with a diagnosis of heart disease ages 21 and older to participate.
The researcher has seven discrete age ranges they would like to study: 21–30,
31–40, 41–50, 51–60, 61–70, 71–80, and 81 and older. The researcher needs 100
participants in each age range. The researcher will enroll participants until that
age range is filled. Once an age range has been filled with 100 participants, the
research will no longer accept participants for that age group. That is, the first
100 women ages 21–30 to volunteer to participate are in the study. The 101st
woman (age 23 years old) who volunteers would be politely turned away; the
researcher has filled the quota for age range 21–30 years old but would continue
to enroll participants in the other age-range groups until they have been filled.
Purposive: the researcher purposefully selects individuals with specific characteristics
or specific experiences who can best answer the research question.
Example: The researcher is conducting a qualitative study on lived experiences of
individuals with heart disease; they seek to explore the experiences of patients’
post-angioplasty (angioplasty is a surgical procedure). The researcher would
purposefully select the individual that could best answer this research question
to participate in the study. When this researcher published the finding of this
study, the research article would include a detailed explanation of the researcher’s
thought processes on why each of the 10–20 participants was selected.

Sciences
Theoretical sampling: Purposive sampling typically identifies and purposefully selects

all the participants at the beginning of the study. Theoretical sampling is the prac-
tice of selecting participants over the course of the study (in phases) based on the
results of the emerging data analysis. Meaning, that the researcher starts collecting
data from a few participants, and the results of the data analysis will reveal which
type of participants should be sampled next. This type of sampling is typically used
in qualitative, grounded theory research.
Example: Hunter, Keady, Casey, Grealish, and Murphy (2016) sampled in three
phases as they were building their theory “of staff psychosocial intervention use
with residents with dementia in long-stay care” (p. 2024). At first, they sampled
and collected data from three people with dementia who lived in the facility,
three nurses, three nurse managers, and one nursing assistant. The data analysis
informed who should be sampled next, meaning who had more information that
would be useful in building the theory. For example, in phase two and phase
three of the study, ten additional nursing assistants were sampled in each phase
because the researchers realized the nursing assistants could provide important
information about how the residents socially interacted (pp. 2025–2026).
Snowball: the researcher identifies an individual with specific characteristics of inter-
est or a specific life situation. The researcher then asks that person to refer similar
people to the researcher. The researcher continues to ask for referrals from each
person referred until the researcher has an adequate number of participants. Mack,
Woodsong, MacQueen, Guest, and Namey (2005) explain that “snowball sampling
is often used to find and recruit ‘hidden populations’, that is, groups not easily
accessible to researchers through other sampling strategies” (p. 5).
Example: The researcher is conducting a study that requires the sample to be
women who use cocaine. This population is difficult to find, so the researcher
asks all the drug counselors in the health system to refer any of their female
patients who are currently using or have ever used cocaine. The researcher then
would ask each of the referred women to invite other women they know who
use this substance to contact the researcher.
How Many Participants Are Needed for a Study?

In the previous section the sample size examples (i.e., 9000, 500, 100, 20, etc.) were merely
illustrative. There are specific practices a researcher must use when selecting the size of the
sample. While the practices naturally vary between methodologies, sample size is directly
related to the multifaceted concept of the scientific merit: Does the study have enough
participants so that the results of the study will be meaningful?
Qualitative Sample Size

Qualitative researchers use non-probability sampling methods such as purposive, conve-
nience, and snowball.
The determination of the number of participants is often dictated by the research design
(more in Chapter 6) and a concept known as data saturation. Marshall (1996) explains the
relationship between sample size and data collection.

Sciences
An appropriate sample size for a qualitative study is one that adequately answers the research ques-
tion. In practice, the number of required subjects usually becomes obvious as the study progresses,
as new categories, themes or explanations stop emerging from the data (data saturation) (p. 523).
The researcher will continue to enroll participants and continue to collect data until
no new analytic insights are gleaned. In quantitative research the methods (sampling,
data collection, and data analysis) are related but distinct research steps. In qualitative
research the methods are more fluid and interconnected, where one method informs the
practice of another and where the methods of sampling, data collection, and data analysis
can occur simultaneously.
Quantitative Sample Size

In quantitative studies the answer is simple: the size of the sample is determined by how
large the population is. Krejcie and Morgan (1970) developed the widely used table that deter-
mines, for the researcher, the minimum number of participants (sample) required based on
the size of the population. This table was developed using specific data analysis levels (alpha
and degree of accuracy, which will be discussed in more depth in the data analysis section).
As Krejcie and Morgan’s (1970) table shows, the general rule states that as the population
gets larger, fewer randomly selected participants are required in the sample:
• if the population is a 100, the sample should be 80;

• if the population is 1000, the sample should be 278;
• if the population is 10,000, the sample should be 370; and
• if the population is 1,000,000, the sample should be 385.
Yes, a sample of 385 people can represent a population of a million.

The more complex answer is directly related to the main theme of this chapter, the
level of rigor a researcher needs in order to generate new and valid knowledge. There are
numerous factors that may impact a researcher’s decision regarding the size of the sample;
these “include: effect size, probability, significance level and type I error, and power and
type II error” (Devane, Begley, & Clarke, 2004, p. 297). Significance levels and type I and
type II errors will be covered in the data analysis section. Here the important concept to
understand is that decisions made during the planning phase of a research study about the
size of the sample are directly related to the meaningfulness of the results (generation of
new knowledge).
If the researcher is conducting experimental research, two statistical tests called power
analysis and sample size estimation are used to determine how many participants are
needed to be enrolled in the study. Combined, the tests take into account many factors:
type of research design, type of data collected, data collection procedures, type of sta-
tistical tests used for data analysis, size and nature of the population, and the study’s
confidence level and confidence interval (Jacobsen, 2017; Phillips & Jiang, 2016; Plichta
& Kelvin, 2013).
• Power analysis: in addition to the type of data and statistical test used to ana-
lyze the data (Plichta & Kelvin, 2013), power analysis considers the alpha level,

Sciences
the amount of power in the study, and the effect sizes (Creswell, 2015; Plichta &
Kelvin, 2013).
■■ Information on alpha levels can be found in the quantitative data analysis
section of this chapter.
■■ Typically, researchers set the power at .80, which means the research will
be able to reject the null hypothesis correctly “80% of the time” (Creswell,
2015, p. 609).
■■ Effect sizes can be defined as the magnitude of “the impact made by the
independent variable on the dependent variable” (Plichta & Kelvin, 2013,
p. 457).
ɝɝ Banerjee, Chitnis, Jadhav, Bhawalkar, and Chaudhury (2009)
add that “selecting an appropriate effect size is the most difficult
aspect of sample size planning”; the researcher often sets the
effect size based on the findings of previous research studies.
However, “when there are no data with which to estimate it, he
can choose the smallest effect size that would be clinically mean-
ingful” (pp. 130–131).
• Sample size estimator
■■ Confidence level: how confident that the results obtained from the sample
will be true for the population (90%, or 95%, or 99%).
■■ Confidence interval: the margin of error.
ɝɝ Example: When a researcher shares that 68% of the sample said they
strongly agreed with X statement, they also show a margin of error.
This means at a 95% confidence interval, the scores would have a
margin of error of +/-5% with the score between 73–63% (New York
State Department of Health, n.d.).
Now take another look at the Krejcie & Morgan (1970) sample size estimator
with the added information of confidence levels and confidence intervals: as the
confidence levels and confidence intervals change, the number of individuals who
need to be sampled increases.
■■ Confidence level set at 95% and confidence interval set at 5%, if the
population is:
ɝɝ 100, the sample should be 80
ɝɝ 10,000, the sample should be 370
ɝɝ 1,000,000, the sample should be 384
■■ Confidence level set at 99% and confidence interval set at 3%, if the
population is:
ɝɝ 10,000, the sample should be 1561
ɝɝ 1,000,000, the sample should be 1846

Sciences
The size of the sample is directly related to the quality and validity of the data analysis
results. Most IRBs will require the health science researcher conducting experimental
research to justify the size of the sample; the researcher does this by sharing the results of
the power analysis and sample size calculation.
METHODS: DATA COLLECTION

In Chapter 1, data collection was defined simply as the type of data that will be collected and
the procedures/processes a researcher uses to collect that data. Quantitative researchers
use tools and instruments to collect numerical data. In qualitative research, the researcher
is the data collection tool. In this section, the primary focus will be on providing examples,
since the relationship between data collection and internal validity/credibility, external
validity/transferability, and reliability/dependability has already been established.
Qualitative Data Collection

Data collection in qualitative research often requires many hours spent interacting with
people throughout the course of their daily lives (observing in the field). Lincoln and Guba
(1985) explain two relevant data collection processes that enhance credibility of the study:
prolonged engagement and persistent observation. Prolonged engagement means the
researcher spends enough time in the field with people so that the researcher can develop
an overall understanding of the environment. This is an important component in providing
a thick and rich description of the study setting. In addition, this allows time for the par-
ticipants to build trust with the researcher and feel comfortable enough to let their guards
down. Once the researcher has a detailed understanding of the environment and has built
trust with participants, the researcher can focus on collecting the data (persistent observa-
tion) that will best answer the research question. Lincoln and Guba (1985) summarize the
two concepts: “if prolonged engagement provides scope, persistent observation provides
depth” (p. 304).
During data collection “a single method can never adequately shed light on a phenomenon.
Using multiple methods can help facilitate deeper understanding … qualitative researchers
generally use this technique to ensure that an account is rich, robust, comprehensive and
well-developed” (Cohen & Crabtree, 2006, para. 3–4).
As previously stated, in data collection the concept of triangulation involves collecting
different types of data: document, interview, image, observation; collecting data at differ-
ent times, from different places, and from different people; or having multiple researchers
collect data. As such, the volume of data collected during a study is immense. Qualitative
researchers use data saturation to determine when to stop the data collection process.
Hasankhani et al. (2014) used triangulation when collecting data. In this study, the
researchers collected data at different times. For example, four of the “participants were
interviewed twice to provide more in-depth data and elaborate on the issues raised in the
previous interview” (p. 145). The researchers purposed to interview 15 patients in their study
of the lived experiences of patients post–coronary angioplasty (angioplasty is a surgical pro-
cedure), but the researchers found after interviewing 13 patients they were not identifying
any new information. The researchers found “data saturation was achieved at interview
13, but the researchers continued on data collection by conducting 2 more interviews to

Sciences
ensure data saturation” (p.145). In this case, the research team erred on the side of caution,
by interviewing the final two participants. By doing so, they clearly demonstrated they had
reached data saturation, thus increasing the credibility of the study’s findings.
Quantitative Data Collection

The tool or instrument a researcher uses to collect data is directly related to internal validity
(testing and instrumentation threats). As previously stated, these tools/instruments must
be both reliable and valid, otherwise the results of the study can be invalid. These concepts
are known as instrument validity and reliability.
Instrument reliability “refers to the extent to which a given instrument consistently

measures an attribute, variable, or construct that it is supposed to measure,”
and instrument validity “refers to the extent to which the instrument actually
measures what it is intended to measure” (Whittemore & D’Eramo Melkus, 2008,
p. 210).
This means the data collection tool/instrument should be precise and accurate. Con-
sider these two examples. A data collection tool/instrument can be “reliable without being
valid … a scale that is incorrectly calibrated may yield exactly the same, albeit inaccurate,
weight values” every time it is used (Kimberlin & Winterstein, 2008, p. 2278). Pirtle (1994)
tested 161 radiographic X-ray machines and found that 30% failed one or more calibration
test. Therefore, if a researcher used one or more of these machines to collect data, the data
(X-ray image) would not be accurate. These two examples underscore the importance of a
data collection tool/instrument being both reliable and valid.
Quantitative researchers, especially in health science, sometimes need to collect data
that is more conceptual (e.g., patient satisfaction, motivation, beliefs, quality of life) than
tangible (e.g., weight, blood pressure, X-ray). Regardless of what type of data is being col-
lected, the tools/instruments can be put through a variety of precision and accuracy tests
before being used in a research study. Some of these tests are very subjective, while others
are highly quantifiable.
Unless otherwise indicated, the simplified definitions below have been summarized from
Kimberlin and Winterstein’s (2008) very detailed explanation of these concepts.
Instrument Validity:
Content: refers to how thoroughly the concept can be measured using this instrument.
“Because there is no statistical test to determine whether a measure adequately
covers a content area or adequately represents a construct [or a concept], content
validity usually depends on the judgment of experts in the field” (Kimberlin &
Winterstein, 2008, p. 2279).
Example: A researcher develops a new questionnaire on empathy; the researcher
would ask an expert, someone who is well known in the field for defining and
measuring empathy, to review the question to make sure all the aspects/dimen-
sions of empathy are covered in the questionnaire.

Sciences
Criterion: refers to testing the new tool/instrument against another tool/instrument or

other measurement if another tool/instrument does not exist. The scores from the
new tool/instrument should correlate to the other tool.
Example: A researcher develops a new critical thinking test specifically for health
science students. The researcher would give a group of health science students
the new critical thinking test and the standard critical thinking test, which has
already been shown to be valid and reliable. If the new test is valid, the scores
should correlate to the standard (valid and reliable) test.
Construct: refers to the “accumulation of evidence from numerous studies using a specific
measuring instrument …” (Kimberlin & Winterstein, 2008, p. 2279).
Example: A test designed to measure quality of life should perform as expected.
The instrument should have lower quality-of-life scores in chronically ill patients
and higher scores quality-of-life scores in healthy patients being seen for rou-
tine checkups. “… Correlations that fit the expected pattern contribute to the
evidence of construct validity” (p. 2279).
Instrument Reliability:
Interrater: is required if the data collection involves judgments or ratings by different
observers. Interrater reliability is a statistical comparison between the scores recorded
between/among the people using the same data collection tool.
Example: Two individuals use a rating scale (data collection tool) to collect data
on third graders’ food choices (healthy or unhealthy) following a healthy eating
intervention program. Both raters would independently and with no discussion
use the rating scale tool to score the children’s food choices and eating behav-
iors in the cafeteria. The scores would be compared, and if the raters have high
interrater reliability, the scores should be equivalent.
Test-retest: data is collected using the same tool/instrument at different times with the
same individuals. The repeat testing should yield identical (medical device) or highly
correlated (questionnaire) measurements/scores.
Example: A person is weighed several times using the same scale, so the mea-
surements should be identical. A person is given the same questionnaire three
days apart; the scores should be almost identical.
Equivalent forms: a data collection tool/instrument has two versions that are
almost identical.
Example: Some research requires the use of a pretest and a posttest. The pretest
and posttest can use different questions as long as the questions are equivalent
and measure the same exact concepts and content. Meaning that all the ques-
tions should be similar and one test should not be more challenging than the
other. To test this, both the pretest and the posttest would be given to a group
of people. Those individuals would take both tests, and the scores on the tests
should be almost identical.
Internal consistency: the scores on a group of items measuring the same concept within
a tool/instrument are highly correlated.

Sciences
Example: A questionnaire about sedentary lifestyle might ask the same question
in opposite ways: “I prefer leisure activities that include a high level of physical
activity” and later in the questionnaire “I dislike leisure activities that require
a lot of physical activity.” If the subject agrees with the first question, then the
subject should also agree with a subsequent question that states the same concept
with different wording. These answers should then correlate to later questions
that ask how many leisure hours are spent biking, walking, hiking, siting and
reading, watching TV, or playing video games.
METHODS: DATA ANALYSIS
SIDE BOX 5.3 Remember, in qualitative research, data analysis guides the researcher’s data collection
and sampling; these processes are linked.
During the design phase of a study, the researcher must determine how the collected
data will be analyzed. Table 1.2 in Chapter 1 provided a simplified explanation of the data
analysis process:
• Quantitative studies: use descriptive and inferential statistical procedures on var-

ious types of numerical data. Data analysis is conducted at conclusion of study;
if the correct test is selected, analysis is quick and straightforward; the results of
analysis can be presented in tables and charts.
• Qualitative studies: use an iterative process of breaking data into small constructs
to find patterns that reveal the essence of meaning. Data analysis begins while
data is being collected; the process is lengthy, and the presentation of findings
includes rich textual descriptions, direct quotes, and/or images.
This section will expand on the previous explanations of data analysis, now that concepts
of power, validity and reliability, and trustworthiness have been presented.
Qualitative Data Analysis

Saldana (2009) states that there are 29 distinct methods to code data. He explains there is
no best method for coding data; rather, the researcher must select the appropriate combi-
nations of methods based on the goals of the study. It is beyond the scope of this book to
explore in depth the multiple methods used to analyze qualitative data. This section will
provide a general description of the data analysis processes: immersion, first-cycle coding,
memo keeping, second-cycle coding, data saturation, and credibility of the findings.
Saldana (2009, pp. 3–4) explains that a code is a word or short phrase that summarizes
the meaning of the segment of data. Coding is a multistage process. The commonality shared
among all coding methods is that prior to data analysis, the researcher must immerse them-
selves in the data. Immersion includes a review of the research purpose statement and research
question, in conjunction with reading and rereading the data numerous times. Merriam

Sciences
(2009) endorses the practice of the researcher transcribing their own data to facilitate the
immersion process, engaging in reflection on the data, and subsequent first-cycle coding.
Immersion is an important process that leads to insights required for first-cycle coding.
Saldana (2009) explains that first-cycle coding is the process of dissecting and examining
the data for similarities and differences. The first-cycle coding also includes the researcher
keeping an ongoing memo. Merriam (2009) demonstrates that a memo is the researcher’s
written record of notes, hunches, ideas, and evolving questions that arise while coding the
data. These notes often guide the researcher to the realization of what new data and what type
of data should be collected next (triangulation) in order to fully answer the research question.
Immersion and first-cycle coding begin as soon as the researcher begins to collect data.
The researcher then collects more data, codes that data, and compares those codes with
previous codes and memo. This first-cycle coding (collecting data, coding data, reflecting,
comparing codes from previous data collection) continues until patterns (known as themes)
begin to emerge from the data analysis. This emergence of patterns, categories, and themes
is called second-cycle coding. Second-cycle coding reveals the meaning of the data. This
continues until the researcher obtains data saturation (Merriam, 2009; Saldana, 2009).
Inherent in this process is that the researcher would use one of the numerous techniques
to check the credibility of the findings (i.e., member checking). Another variation on member
checking can be found in Hasankhani et al. (2014). Instead of checking data analysis find-
ings with participants, they choose an alternative method where the “preliminary findings
of the study were shared and discussed with academics in the field and clinicians who had
clinical experience in providing care to angioplasty patients” (p. 145).
Below is an example of first-cycle codes coalescing into second-cycle codes. In this study,
health science students were asked to reflect on their critical thinking skills. The examples
below are data segments collected from students (Zelizer, 2013); the bolded words in brackets
illustrate some of the first-cycle codes for the data.
“I used to consider myself an independent student, {thinking changed} one who pre-
ferred to finish tasks alone and work independently, {working alone better} but after
coming this far in the health sciences, I realize {experience=thinking changed}
that I work and think much better {working alone not better} when working in a
team” {team=improved thinking; team=improved work}
“My way of thinking has changed after the … semester {experience=thinking changed}
because I learned that you cannot do everything on your own all the time {working
alone not better}. The interactions amongst others is key to success {group=suc-
cess} we learn to work in groups and tackle problems {group solution}. A group is
stronger than individual when handling a task” {group=improved work}
“Working in groups as a health science major, has taught me {experience=change} that
everyone has unique ideas and skills {group=more ideas/skills}. When these skills
are combined {group=more skills} and focused on one goal it creates an outcome
that is greater than the sum of its parts (ideas)” {group=better outcomes}
The data above was ultimately coded under a theme called Peer Collaboration. The theme
was defined as: teamwork that yields greater possibilities or better results; the ability to

Sciences
consider others’ ideas when assessing problems or generating solutions; or the ability to
communicate “effectively with others in figuring out solutions to complex problems” (Foun-
dation for Critical Thinking, 2011, para. 12). Even though the three students used different
words, the theme (peer collaboration) summarized the essential meaning of the data.
Quantitative Data Analysis

As previously stated, a researcher collects data, which must be turned into numerical form.
There are different types of numerical data that require the use of different data analysis
methods. The different types of numerical data can be classified as four types: nominal,
ordinal, interval, and ratio.
Data Scales
Nominal data: a type of data that allows a researcher to label a difference without putting
a value on the difference. Here, different simply means different.
Example: A survey question asks participants to answer a demographic question
on gender. The survey gives two choices: do you self-identify as female or male?
During data analysis the researcher turns all data into numbers. The answer
female is assigned the number 1, and the answer male is assigned the number 2.
In this case the numbers do not represent value difference; 1 is not better than 2
and 2 is not double number one. It simply is a number representing the answer.
Ordinal data: a type of data scale that identifies difference by yielding a rank ordering
of difference. Participants rank order answers; for example, best, second best, and
third best. Ordinal data shows a difference—this is better than that but does not
reveal by how much.
Example: A survey asks participants to select their top three leisure activities
from a list and then rank order the preference (which one is the first choice,
second choice, and third choice).
Choices:
Biking
Walking
Running
Hiking
Kayaking
Sitting and reading
Binge watching TV for 6 or more hours in one session
Playing video games
Playing cards
Surfing the web
Posting on social media
For example, a participant rank ordered three choices: 1) binge watching TV for 6 or more
hours in one session; 2) siting and reading; and 3) walking. The researcher now knows the
top three preferences, but it remains unknown whether the participant really has only a

Sciences
slight preference between the three, whether that preference is equally weighted, or whether
one is the clear preference and the other two choices were selected because the participant
was instructed to select three.
Figure 5.1 demonstrates that the amount of difference can be extremely variable, but an
ordinal scale cannot capture the difference.
Interval and Ratio data: unlike ordinal, these are types of data that allow a researcher to
measure the exact difference. These types of data have increments that are consistent
and can be measured. The only difference between interval and ratio is whether the
instrument used to measure the increments has a true zero. Interval and ratio data
can be analyzed using inferential statistics.
Example: Interval: a thermostat or thermometer measures temperature. There are
discrete units of incremental measurement; the difference between 40 degrees
and 50 degrees is 10 degrees. The same difference of 10 degrees applies between
70 degrees and 80 degrees. However, since the thermometer doesn’t have a true
zero, meaning a Celsius temperature reading of zero is not the absence of all
heat, there is no absolute zero from which the tool can say measurement begins.
Example: Ratio: a dollar is an incremental unit of measure. If a person earns
10 dollars per hour and works only 6 hours and another earns 20 dollars per
hour and works only 6 hours, we know that the person earning 20 dollars per
hour has double, or 2x, the amount of income as compared with the individual
earning only 10 dollars per hour. Income would be ratio data; because this scale
FIGURE 5.1 An illustration of the inability of ordinal data to measure the amount of difference between
ranked orders of preference.
No large difference Clear weighted Strong and clear

between preferences preference preference for one choice
• 1st: Binge watching TV • 1st: Binge watching TV • 1st: Binge watching TV

• 2nd: Reading
• 3rd: Walking
• 2nd: Reading
• 2nd: Reading
• 3rd: Walking • 3rd: Walking

Sciences
has a true zero, a person can have zero income. Other examples of ratio include
any instrument that has a zero-starting point. In health science research, the
most common ratio scales are “biomedical variables (e.g., weight, height, blood
pressure, pulse rate)”; for example, to make sure the scale a researcher is using
to measure weight would be “balanced at 0” so there is no margin of error in
recording weights (Plichta & Kelvin, 2013, p. 24).
Statistics: Descriptive and Inferential

Data analysis a very complex topic and requires a fair amount of expertise in determining
which statistical test to run. The purpose and scope of this section of the chapter is to
introduce common terms associated with quantitative analysis in a simplified manner. The
statistical analysis a researcher can perform varies based on whether the data is parametric
data (interval and ratio) or nonparametric data (nominal and ordinal). Below is a sample of
the most common statistical tests.
• Parametric data (interval and ratio)

■■ Descriptive statistics: Central tendency and description of relative
position
■■ Inferential statistics: Significant difference: t-test, ANOVA, ANCOVA,
MANOVA, Regression or Multiple regression, Pearson r, Odds ratio, etc.
• Nonparametric data (nominal and ordinal)
■■ Descriptive statistics: Frequencies, percentages
■■ Inferential statistics: Mann-Whitney U test, Chi squared, etc.
Descriptive statistics: these analysis techniques describe the data. The results of a descrip-
tive study might be expressed in descriptive statistics, which includes reporting
frequencies, percentages, central tendency (mean, median, mode), and description
of relative position (range, standard deviation) of the data. For example, a study’s
results might find that 85% of the individuals completing the survey strongly agreed
with a particular statement. Additionally, all studies should provide the reader with
a descriptive analysis of demographic characteristics and the data when appropriate.
Mean – the average of all the scores (i.e., if the data set was 12, 2, 12, 5, 5, 7, 5, 9, 5,
the average score would be 6.8).
Median – the score found directly in the center of the data when the data is arranged
in order from lowest to highest (i.e., if the data set was 2, 5, 5, 5, 5, 7, 9, 12, 12,
the score directly in the middle/center would be 5).
Mode – the score that occurs most frequently (i.e., if the data set was 12, 2, 12, 5, 5,
7, 5, 9, 5, the most frequently occurring score would be 5).
Range – represents the full range of scores, the lowest value to highest value in a
data set (i.e., if the data set was 12, 2, 12, 5, 5, 7, 5, 9, 5, the range of scores would
be 2–12).
Standard deviation – the dispersion of data around the mean; for example, if the
data set was 12, 2, 12, 5, 5, 7, 5, 9, 5 with a mean of 6.89, the standard deviation

Sciences
(SD) would be 3.44. Meaning, the scores are spread out and not tightly grouped
around the mean. One SD would show that the 68% of all scores are clustered
around the mean, an SD of 3 means the scores are spread apart from each other
(12 and 5). In this case, to calculate an SD of 1, the data would look something
like this: 7,6,6,6,5,5,5,8,6 and the mean would change to 6. When there are only
nine values, it is easy to see how close the values are to mean, so it becomes
more challenging to see how values cluster around the mean when there are
hundreds of data points. For example, in Table 5.5, when comparing mean of the
two groups, it is important to show how the data is grouped around the mean;
in this case the reader can see that the academic variables (both mean and how
they were grouped around the mean) were actually very similar between the
two groups. Another noteworthy point is how the two groups differed in age in
Table 5.5. While the mean was very similar, the range and standard deviation
are variable. Actually, there was one person age 47 in the group accounting for
the difference.
Review examples are found in Tables 5.4 and 5.5; notice in Table 5.4 only frequencies are
reported, as central tendencies, range, and standard deviations cannot be calculated on
nominal data. Unlike in Table 5.5, which illustrates that ratio (age) and interval data (grade
point average, GPA) can include analysis of mean, range, and standard deviations so the
reader can easily see the distribution of data around the mean.
TABLE 5.4 Descriptive Analysis: Race/Ethnicity
N Percent
African American & Latino/Latina 141 28%
Asian/Pacific Islander 154 31%
Caucasian 181 36%
Response blank 26 5%
Source: Deborah A. Zelizer, “Critical Thinking: Comparing Instructional Methodologies in a Sen-

ior-Year Learning Community.” ProQuest: UMI Dissertations Publishing, 2013.
Inferential statistics – these data analysis techniques allow the researcher to draw
inferences (conclusions based on the type of study that was conducted: causation,
correlation, etc.) from the sample and then apply those results back to the pop-
ulation. Inferential statistical tests are used to test hypotheses and require the
researcher set an alpha level and a confidence level (presented earlier in the chapter)
during the design phase of the study. These data analysis decisions impact sampling
(determining the size of the sample). An alpha level is the level at which statistically
significant results are determined. For example, a researcher has a null hypothesis
that is being tested:

Sciences
TABLE 5.5 Descriptive Analysis: Age and Grade Point Averages
Group 1 Group 2
(n = 92) (n = 79)
Range Mean (SD) Range Mean (SD)
Age 19–29 21.42 (1.51) 20–47 21.56 (3.05)
Academic variables
Overall GPA 2.00–3.83 2.86 (0.43) 2.09–3.95 2.93 (0.40)
Math GPA 0.85–4.00 2.67 (0.72) .83–4.00 2.73 (0.70)
Science GPA 1.12–3.73 2.48 (0.55) 1.29–4.00 2.48 (0.56)
Note: Group 1 represents the intervention group and Group 2 represents the control group.
Source: Deborah A. Zelizer, “Critical Thinking: Comparing Instructional Methodologies in a Senior-Year Learning Commu-
nity.” ProQuest: UMI Dissertations Publishing, 2013.
There will be no difference in the efficacy of Drug A and Drug B in controlling

patients’ cholesterol levels that are in excess of 340 mg/dl.
The alpha level is the level at which the researcher can reject the null hypothesis and
conclude there was a statistically significant difference between the drugs; Drug A was
more effective than Drug B. Typically, the alpha level for educational intervention studies
is set at .05; however, for a drug trial the researcher might want to set a more rigorous alpha
level before rejecting a null hypothesis, perhaps .01 or .001.
Alpha levels – the alpha level, also known as the p value, is the level that the researcher
can state the results of the study did not occur by random chance.
• p < .05 means that only 5 results out of 100 might have occurred by chance, not as
a result of the experiment
• p < .01 means 1 result out of 100 might have occurred by chance, not as a result of
the experiment
• p < .001 means 1 result out of 1000 might have occurred by chance and not as a
result of the experiment
When the results of the analysis reveal findings at or lower than the alpha (p value), the
researcher can conclude that outcomes of the study were the result of something (interven-
tion or correlation between variables) other than random chance. This is called statistical
significance. Said another way, the researcher can reject the null hypothesis and say there
was a statistically significant difference between the performance of Drug A and Drug B,
with Drug A performing better.

Sciences
The researcher must consider other statistical information before making the final
determination that drug A was indeed more effective than drug B. It is beyond the scope of
this book to cover all of these factors, but one concept we should review is the possibility
of type I errors and type II errors.
• Type I error – occurs when the null hypothesis is falsely rejected. The researcher
believes that there was a significant difference when none actually exists: the
researcher’s statistical analysis leads them to believe that Drug A is more effective
than Drug B, when in actuality it is not. A way to mitigate this type of error is to lower
the alpha level, thus reducing the chance the null hypothesis will be falsely rejected.
• Type II error – occurs when the null hypothesis is falsely accepted. The sta-
tistical analysis showed that there was no difference between the performance
between Drug A and Drug B, when in actuality there is a difference. Unfortu-
nately, lowering the alpha level increases the chances that the researcher will
falsely accept the null hypothesis (incorrectly concluding there is no difference).
The statistical tests called the power analysis and sample size calculation (pre-
viously mentioned in the sampling section) allows the researcher to determine
how many participants are required in the study to achieve statistical power to
help mitigate type II errors. The best way to help reduce type II errors is to have a
larger number of participants in the study.
Finally, the last concept addressed in this section is applied/clinical significance. The
researcher rejects the null hypothesis due to statistically significant results, but the researcher
has one additional step. The researcher must determine if this statistical finding of relation-
ship or difference has any practical (applied) or medical (clinical) value. The health science
researchers must ask themselves: Do the results of this study make a real-world difference
in the quality of patient care? For example, it was found that Drug A was indeed statistically
more effective than Drug B. However, upon a closer look, it only improved patients’ cholesterol
levels by 10 points, not nearly enough to make a significant impact when the recommended
level is 200mg/dl. As a result, it might be decided that the results of the study had no clinical
significance and that researchers need to continue the search for a more effective medica-
tion to manage the health issue. There are many factors in determining clinical significance
such as sample size, confidence intervals, effect sizes, etc. The quantitative researcher must
have a well-developed understanding of statistics in order to determine whether the study’s
finding indeed had applied or clinical significance.

This was a very detailed and complex chapter. The chapter presented foundational information
on rigor and expanded the information on methods presented in Chapter 1. Table 5.6 provides
a summary of the common threats to a quantitative study’s internal validity and practices a
researcher can use to mitigate threat and increase the study’s internal validity. Understanding
this concept will be very important to understanding the purpose of the designs presented
in Chapter 7. Table 5.7 provides a summary of the practices a qualitative researcher would
employ to increase a study’s trustworthiness, which will be expanded upon in Chapter 6.

Sciences
TABLE 5.6 Summary: Threats to Internal Validity
Internal Validity Threats Controlling for the threats to internal validity by using a
History research design with a control group
Maturation research design with a control group
Testing data collection tool that has very high validity and reliability
Instrumentation data collection tool that has very high validity and reliability
Placebo effect double-blind and/or placebo-controlled techniques
Statistical regression data collection tool that has very high validity and reliability
Hawthorne effect process of sustained and unobtrusive observation methods
Selection bias probability sampling methods
Attrition larger sample to compensate for lost to follow-up
This chapter also provided a more developed explanation of research methods used in
qualitative and quantitative studies than originally presented in Table 1.3. Table 5.8 outlines
these expanded concepts/practices by methodology. When reviewing Table 5.8, take special
note as to how the terms are related to one another and how they are related to each meth-
odology. For example, in terms of sampling methods, notice how quantitative can use either
probability or non-probability methods (with the exception of theoretical and purposive).
Qualitative uses non-probability sampling methods (with the exception of quota) but does
not use any of the probability sampling methods.
TABLE 5.7 Summary: Trustworthiness
Trustworthiness Practices That Increase Trustworthiness
Transferability Thick description of the participant and study setting
Credibility Triangulation
Member checking
Dependability Audit trail

Sciences
TABLE 5.8 Methods by Methodology Revised
Quantitative Methods Qualitative
ɓɓ Power analysis ɓɓ Non-probability sampling

ɐɐ Effect size ɐɐ Snowball
ɐɐ Alpha level ɐɐ Theoretical
Sampling
ɓɓ Sample size estimation ɐɐ Convenience
ɐɐ Population ɐɐ Purposive
ɐɐ Confidence interval ɓɓ Sample until data saturation is
ɐɐ Confidence level reached
ɓɓ External validity
ɓɓ Inclusion and exclusion criteria
ɓɓ Probability sampling
ɐɐ Simple random
ɐɐ Proportional stratified
ɐɐ Stratified random
ɐɐ Systematic random
ɐɐ Cluster
ɓɓ Non-probability sampling
ɐɐ Snowball
ɐɐ Convenience
ɐɐ Quota
ɓɓ Numerical ɓɓ Visual, verbal, narrative, artifacts

Type of
ɐɐ Nominal
data
ɐɐ Ordinal
ɐɐ Interval/ratio
Data Collection
ɓɓ Interviews
ɓɓ Collects data with tools and instru- ɓɓ Prolonged engagement
ments that are valid and reliable ɓɓ Persistent observation
ɐɐ Content, criterion, and con- How data is ɓɓ Triangulation of data
struct validity collected ɓɓ Memoing
ɐɐ Interrater, test-retest, equiva- ɓɓ Collect data until data saturation is
lent forms, and internal con- reached
sistency reliability
ɓɓ Descriptive statistics ɓɓ Immersion

ɐɐ Mean, median, mode, range, ɓɓ Reflective bracketing
standard deviation Data Analysis ɓɓ Coding
ɓɓ Inferential statistics ɓɓ Themes
ɐɐ Alpha levels ɓɓ Thick description
ɐɐ Effect size ɓɓ Member checking
ɐɐ p value ɓɓ Audit trail
ɓɓ Statistical significance ɓɓ Data saturation
ɓɓ Clinical significance
ɓɓ Type I and type II errors
ɓɓ Type of statistical test used related
to data scales

Sciences
STUDY PRACTICE
You now have a much more detailed understanding of the methods used in quantitative and
qualitative research. The chapter presented many new concepts and expanded upon concepts
presented in previous chapters. Revise your notes/flash cards with this new information.
Rather than simply presenting a word list for your new flash card construction, use the
tables provided in the summary section of this chapter as your word list. Almost all of the
new concepts and terms presented in this chapter can be found in Tables 5.6–5.8.
At this point you should be able to immediately determine a study’s methodology and
the methods used to sample, collect data, and analyze the data. You will need a strong
command of these concepts to successfully learn the material presented in the second
half of this book.
ɠɠ Reliability ɓɓ Proportional stratified
ɠɠ Validity ɓɓ Systematic
ɝɝ External validity ɓɓ Cluster
ɝɝ Internal validity ɠɠ Non-probability methods
ɝɝ Threats to validity ɝɝ Convenience
ɓɓ History ɝɝ Quota
ɓɓ Maturation ɝɝ Purposive
ɓɓ Testing ɝɝ Theoretical sampling
ɓɓ Instrumentation ɝɝ Snowball
ɓɓ Statistical Regression ɠɠ Qualitative sample size
ɓɓ Placebo effect ɝɝ Data saturation
ɐɐ Double-blind ɠɠ Quantitative sample size
ɐɐ Placebo-controlled ɝɝ Power analysis
ɠɠ Hawthorne effect ɝɝ Sample size estimation
ɠɠ Selection bias ɝɝ Confidence level
ɠɠ Attrition (loss to follow-up) ɝɝ Confidence interval
ɠɠ Trustworthiness of findings ɠɠ Qualitative data collection
ɝɝ Transferability ɝɝ Prolonged engagement
ɓɓ Thick description ɝɝ Persistent observation
ɝɝ Credibility ɠɠ Quantitative data collection
ɓɓ Triangulation ɝɝ Instrument reliability
ɓɓ Member checking ɓɓ Interrater
ɝɝ Dependability ɓɓ Test-retest
ɓɓ Audit trail ɓɓ Equivalent forms
ɠɠ Population ɓɓ Internal consistency
ɠɠ Inclusion criteria ɝɝ Instrument validity
ɠɠ Exclusion criteria ɓɓ Content
ɠɠ Probability methods ɓɓ Criterion
ɝɝ Random assignment ɓɓ Construct
ɝɝ Random sample ɠɠ Qualitative data analysis
ɓɓ Simple random ɝɝ Immersion
ɓɓ Stratified random ɝɝ First-cycle coding

Sciences
ɝɝ Memo ɝɝ Median
ɝɝ Themes ɝɝ Mode
ɝɝ Second-cycle coding ɝɝ Range
ɠɠ Data scales ɝɝ Standard deviation
ɝɝ Nominal data ɠɠ Inferential statistics
ɝɝ Ordinal data ɝɝ Alpha level (p value)
ɝɝ Interval and ratio data ɝɝ Statistically significant
ɠɠ Descriptive statistics ɝɝ Types I and II errors
ɝɝ Mean ɝɝ Applied/clinical significance

A researcher collects water samples from three streams (Stream A, Stream B, Stream C) near
a large industrial plant. Exactly 1 liter of water is collected from each stream. Then, back
at the lab, the researcher measures the amount of a specific health-jeopardizing bacterium
in each sample. A newspaper reporter interviews the researcher about the findings of the
study. The next day an article in the local newspaper states that Stream B had the highest
level of health-jeopardizing bacteria, Stream C had the second-highest level, and Stream A
had the lowest levels.
1. In the study, what measurement scale does the measurement of bacteria in
each sample reflect?
a. Nominal
b. Ordinal
c. Interval
d. Ratio
2. In the study, what measurement scale was used in the labeling of (Stream A,
Stream B, Stream C)?
a. Nominal
b. Ordinal
c. Interval
d. Ratio
3. In the newspaper article, what measurement scale was used by the reporter
to describe the findings?
a. Nominal
b. Ordinal
c. Interval
d. Ratio
It has been well documented that patients with cardiac disease have problems following
the strict guidelines required for a cardiac diet. The purpose of this study was to gain an
in-depth, generalizable understanding of the specific barriers and difficulties patients with
cardiac disease experience in regard to cardiac diet compliance. The study used specially

Sciences
trained social work students to conduct in-depth interviews with a small group of pur-
posefully selected patients. That information was then used to construct a survey that was
disseminated to 5000 randomly selected patients.
4. What type of sampling was used?
a. Simple random and convenience
b. Purposive and one of the probability sampling methods
c. Convenience and stratified random
d. Non-probability sampling and probability sampling
5. Which answer best outlines the data analysis process for the qualitative research
conducted in this study?
a. Collect the data, determine which statistical test to use (descriptive, inferential,
or both), consult with a statistician if necessary.
b. Read and reread the data, break data down into small components, look for
meaning in the data, find patterns in data that reveal meaning.
c. Not only conducting in-depth interviews but also obtaining artifacts, documents,
and relevant images.
d. Selecting predetermined variables that are measurable.
6. A researcher collects different types of data (verbal, documents, images, observation)
so they will have more confidence in the truth of the findings.
This is an example of a method to increase
a. dependability
b. credibility
c. internal validity
d. transferability
7. A double-blind experiment, in which both the researcher and the subject don’t know
whether the subject is in the experimental group or the control group, is an example
of a strategy a researcher might use to eliminate other possible explanations for the
observed results.
This is an example of a method used to increase
a. internal validity
b. Hawthorne effect
c. triangulation
d. external validity
8. Which statement best describes the term quota selection? A researcher:
a. selects everyone enrolled in the fall semester of a PSY 100 course
b. selects every 25th person from a master list of hospital employees
c. selects people to participate who have been referred to the researcher by others
d. selects a specific number of students based on predetermined categories (25
females, ages 18–20; 25 males, ages 18–20; 25 females, ages 21–23; etc.).

Sciences
9. A researcher wants to gain an understanding of the borough’s (Brooklyn) concept

of health care reform. Brooklyn has 37 different zip codes; the researcher randomly
selects 100 households from each zip code to complete the survey.
What sampling method was used?
a. Stratified random
b. Cluster
c. Systematic random
d. Simple random
10. A researcher working in a large health system wants to determine whether face-to-face
nutrition lectures woven into guided grocery store tours would result in lowering
high risk patients’ cholesterol when compared to lecture-only nutrition classes. The
researcher finds statistically significant results (alpha level of p < .05) between the
two teaching groups, but when she reviewed how much the cholesterol was lowered
in the experimental group despite the significant results, she decided not to imple-
ment the guided tours nutrition teaching method throughout the health system.
Which of the factors below most likely explains why?
a. Lack of clinical significance
b. Statistical significance was found, but the alpha was set too high
c. Statistical significance was found, but the alpha was set too low
d. Lack of data saturation
STUDY ACTIVITIES
The following scenarios are meant for you to test yourself without your notes. This will
help you determine if there are concepts you are still struggling with. Find the evidence of
methods, rigor, and methodology. Don’t make things up—meaning not all the scenarios will
include information on all three methods or in enough detail to determine which sampling
method under probability or non-probability sampling was used. However, there should be
enough information (key words and research practices) to determine methodology.
1. What are the methods (sampling, data collection, and data analysis) used in the
scenario below? Assess indicators of rigor and any practices or information related
to internal validity, external validity, or trustworthiness in the scenarios below.
A questionnaire was sent to 2500 randomly selected women who had given birth (May
2016–June 2017) in any one of the six hospitals within a large health system. The purpose
of the survey was to evaluate the impact of new procedures used in the health system to
increase patients’ feelings of safety, confidence in medical care, and overall satisfaction with
the birthing experience. The health system used a national survey known for its high scores
on validity and reliability measures. During data analysis the researcher wanted to see if
any variables (demographic information, hospital location, etc.) were associated with the
mother’s safety, confidence, and satisfaction ratings. An interesting result emerged, but only
for the women who self-identified both married and as lesbian. While their ratings on the

Sciences
parameters of confidence in medical care and feelings of safety were similar to the overall
group of women, these mothers rated the overall experience as significantly less satisfac-
tory. The study continued, and in-depth interviews were then conducted with 15 women in
order to develop a deeper understanding of their experiences. A pattern began to emerge
from the analysis; it was the importance of having their spouse viewed by the health care
team as a mother to the newborn. The researchers asked the women if the emerging theme
“acknowledge both mommies” was a true representation of their experiences. Scenario
adapted from adapted from Dahl et al. (2013).
A group of physical therapists working in a home-care agency wanted to determine

whether a home-based walking exercise program that included a cognitive behavioral inter-
vention would improve the patients’ pain tolerance and functional walking performance
(confidence level 95%, alpha level .05). The research team recruited 100 elderly patients
(aged 75 or older) from their caseloads and assigned patients either to the new home-based
cognitive behavioral walking intervention or the standard physical therapy treatment offered
by the home care agency. Data was collected after the completion of the program using the
Walking Impairment Questionnaire (WIQ), Pain Composite Scores from the 12-item short
form, and a physical therapy assessment of walking performance. The WIQ and 12-item short
form were selected due to the instruments’ validity and reliability scores. The researchers
developed the rating tool to assess walking performance, and prior to using it in the study,
the research team rated 20 similar patients using the assessment tool. The researchers found
that scores were equivalent across the 20 pilot patients. The results of data analysis showed
a statistically significant increase in walking performance (p < .01) and pain tolerance (p <
.05) in the intervention group. Scenario adapted from Rejeski et al. (2014).
What are the day-to-day behaviors, verbal and nonverbal communication patterns, and
cultural norms related to substance use and treatment for substance abuse in a remote,
insulated community living in the upstate region of New York?
Little is known about the day-to-day explicit and implicit patterns of communication,
behavior, and norms surrounding substance use and/or abuse. The researcher is given per-
mission by the community elders to observe ceremonies and interview community members
if they are willing to talk to the researcher. The researcher collected data by conducting 15
face-to-face interviews and amassed about 200 hours of observations (attending community
events: weddings, festivals, funerals, etc.) over a 6-month period. As the study progressed,
the researcher asked the community elders for permission to review historical documents,
take photos of the community’s art, and to visit the community’s schools. The elders granted
the request, and the researcher continued the data collection process.

Sciences
REFERENCES
Banerjee, A., Chitnis, U., Jadhav, S., Bhawalkar, J., & Chaudhury, S. (2009). Hypothesis testing, type I
and type II errors. Industrial Psychiatry, 18(2), 127–131.
Centers for Disease Control and Prevention. (2017, January 19). National Center for Health Statistics:
Heart disease. Retrieved from https://www.cdc.gov/nchs/fastats/heart-disease.htm
Cohen, D., & Crabtree, B. (2006, July). Qualitative research guidelines project. Retrieved from http://
www.qualres.org/HomeTria-3692.html
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings.
Chicago, IL: Rand McNally.
Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and
qualitative research (5th ed.). Upper Saddle River, NJ: Pearson Education.
Dahl, B., Fylkesnes, A., Sorlie, V., & Malterud, K. (2013). Lesbian women’s experiences with healthcare
providers in the birthing context: A meta-ethnography. Midwifery, 29(6), 674–681.
Devane, D., Begley, C. M., & Clarke, M. (2004). How many do I need? Basic principles of sample size
estimation. Journal of Advanced Nursing, 47(3), 297–302.
Fogel, C. I., Crandell, J. L., Neevel, A. M., Parker, S. D., Carry, M., White, B. L., … Gelaude, D. J. (2015).
Efficacy of an adapted HIV and sexually transmitted infection prevention intervention for incar-
cerated women: A randomized controlled trial. American Journal of Public Health, 105(4), 802–809.
doi:10.2105/AJPH.2014.302105
Foundation for Critical Thinking. (2011). Defining critical thinking. Retrieved from http://www.
criticalthinking.org/pages/defining-critical-thinking/410
Godwin, M., Ruhlan, L., Casson, I., MacDonald, S., Delva, D., Birthwhistle, R., … Seguin, R. (2003).
Pragmatic controlled clinical trials in primary care: The struggle between external and internal
validity. BMC Medical Research Methodology, 3(28), e1–e7. doi:10.1186/1471-2288-3-28
Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In N. K. Denzin
& Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 105–117). Thousand Oaks, CA: SAGE
Publications.
Hasankhani, H., Gholizadeh, L., Mohammadi, E., Zamanzadeh, V., Allahbakhshian, A., Ghaffari,
S., & Allahbakhshian, M. (2014). The lived experiences of patients post coronary angioplasty:
A qualitative study. Journal of Vascular Nursing, 32(4), 144–150. doi:10.1016/j.jvn.2014.04.001
Hunter, A., Keady, J., Casey, D., Grealish, A., & Murphy, K. (2016). Psychosocial intervention use in long-
stay dementia care: A classic grounded theory. Qualitative Health Research, 26(14), 2024–2034.
Jacobsen, K. H. (2017). Introduction to health research methods. A practical guide (2nd ed.). Burlington,
MA: Jones & Bartlett Learning.
Johnson, M., O’Hara, R., Hirst, E., Weyman, A., Turner, J., Mason, S., ... Siriwardena, A. N. (2017).
Multiple triangulation and collaborative research using qualitative methods to explore decision
making in pre-hospital emergency care. BMC Medical Research Methodology, 17(1), 1–11. doi:10.1186/
s12874-017-0290-z
Kimberlin, C. L., & Winterstein, A. G. (2008). Validity and reliability of measurement instruments
used in research. American Journal of Health-System Pharmacy, 65(23), 2276–2284. doi:10.2146/
ajhp070364
Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational
& Psychological Measurement, 30, 607–610.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: SAGE Publications.
Leung, L. (2015). Validity, reliability, and generalizability in qualitative research. Journal of Family
Medicine and Primary Care, 4(3), 324–327. doi:10.4103/2249-4863.161306

Sciences
Mack, N., Woodsong, C., MacQueen, K. M., Guest, G., & Namey, E. (2005). Qualitative research meth-
ods: A data collector’s field guide. Retrieved from http://www.fhi.org
Marshall, M. N. (1996). Sampling for qualitative research. Family Practice, 13(6), 522–525.
Mayberg, H. S., Silva, J. A., Brannan, S. K., Tekell, J. L., Mahurin, R. K., McGinnis, S., & Jerabek,
P. A. (2002). The functional neuroanatomy of the placebo effect. American Journal of Psychiatry,
159(5), 728–737.
John Wiley & Sons.
New York State Department of Health. (n.d.). Confidence intervals—Statistics teaching tools. Retrieved
from https://www.health.ny.gov/diseases/chronic/confint.htm
Phillips, G. W., & Jiang, T. (2016). Measurement error and equating error in power analysis. Practical
Assessment, Research & Evaluation, 21(9), e1–e12.
Pirtle, O. L. (1994). X-ray machine calibration: A study of failure rates. Radiologic Technology, 65(5),
291–298.
Plichta, S. E., & Kelvin, E. (2013). Munro’s statistical methods for health care research (6th ed.). Phila-
delphia, PA: Wolters Kluwer Health, Lippincott Williams & Wilkins.
Purves, B., Logan, H., & Marcell, S. (2011). Intersections of literal and metaphorical voices in aphasia.
Aphasiology, 25(6–7), 688–699. doi:10.1080/02687038.2010.536842
Rejeski, W., Spring, B., Domanchuk, K., Tao, H., Tian, L., Zhao, L., & McDermott, M. M. (2014). A
group-mediated, home-based physical activity intervention for patients with peripheral artery
disease: Effects on social and psychological function. Journal of Translational Medicine, 12(29),
e1–e8. doi:10.1186/1479-5876-12-29
Saldana, J. (2009). The coding manual for qualitative researchers. London, UK: SAGE Publications.
Waber, R. L., Shiv, B., Carmon, Z., & Ariely, D. (2008). Commercial features of placebo and therapeutic
efficacy. JAMA, 299(9), 1016–1017.
Whittemore, R., & D’Eramo Melkus, G. (2008). Designing a research study. Diabetes Educator, 34(2),
201–216. doi:10.1177/0145721708315678
Zelizer, D. (2013). Critical thinking: Comparing instructional methodologies in a senior-year learning
community. (Doctoral dissertation). ProQuest: UMI Dissertations Publishing.

Sciences
QUALITATIVE RESEARCH:
QUALITATIVE DESIGNS 6
CHAPTER GOALS:
INTRODUCTION
The second half of this book (Chapters 6–10) will be devoted to research
designs. This and subsequent chapters build on concepts previously • To introduce four qualitative
presented in the first half of the book. Specific to this chapter, Chapter 1 research designs.
provided a definition of the qualitative methodology and linked pur- • To explore the purpose and special
pose and practice to the constructivist paradigm. Chapter 3 discussed features of each design.
the role of the literature review in qualitative research, and Chapter 4 • To build on knowledge presented in
explored how a qualitative researcher would construct a research problem Chapters 1–5.
purpose statement and research questions. The information found in
Chapter 5 provided detailed information on how the qualitative researcher
approaches sampling, data collection, and data analysis as well as strat-
egies to enhance the rigor of the scientific inquiry. LEARNING OBJECTIVES:
The researcher’s selection of a research design (Table 6.1) is the plan-
ning stage of research that is directly related to how best to explore the
study’s research question(s). In quantitative research, the research ques- THE STUDENT WILL BE ABLE TO:
tions explore whether X is better than Y, how often does X occur, what
impact will X have on Y, and other questions that attempt to quantify phe- • Compare research designs for
nomena. Qualitative research focuses on research questions that “explore defining features;
why or how a phenomenon occurs, to develop a theory, or to describe the • Align research questions and/or pur-
nature of an individual’s experience” (Fetters, Curry, & Creswell, 2013, pose statements to research designs;
p. 2135). Said another way, qualitative research focuses on the human • Define 11 new terms related to quali-
experience, attempting to bring to light the meaning people attach to tative research designs;
an event, experience, or a social situation. The researcher attempts to • Recognize the unique characteristics
uncover how people make sense of the world through understanding how of qualitative research designs;
the participants see the world (Merriam, 2009).
• Demonstrate the ability to choose
In planning a qualitative research study, the researcher aligns the
the most appropriate design for a
research question to a specific research design. To that end, the purpose
research study; and
of this chapter will be to clearly define and explain the purpose of the
• Apply previously presented concepts
most commonly used qualitative research designs utilized in health sci-
in a new context.
ence research: Case study, focused ethnography, phenomenological, and
grounded theory. Table 6.2 provides a summary of the purpose of each
design with an example of the type of research question that would be
answered by the design (Chan, Fung, & Chien, 2013; Crowe et al., 2011;
125

Sciences
Dill, 2017; Dunn, Margaritis, & Anderson, 2017; Fusch, Fusch, & Ness, 2017; Merriam, 2009;
Padgett, Gossett, Mayer, Chien, & Turner, 2017).
TABLE 6.1 Stages of Research: Action Steps Highlighted in this Chapter
Stages Action Steps
Design (Planning) Selecting the best research design and research methods (sampling, data
collection, data analysis) that align with research question(s), submitting the
research study proposal for IRB review
TABLE 6.2 Aligning Research Designs with Research Questions
Design Purpose Examples of Research Questions
Case study An in-depth exploration of one person, What are the effects of establishing a telemedicine
one event, or one program to understand/ center, which would provide virtual appointments
illustrate an outcome, a procedure, or how with a physician for individuals living in the remote
something changed over time. county of Oakwood whose closest health care pro-
vider is 100 miles away?
Focused Ethnography Understand the culture of a group related to What are the perceptions of chronic pain and
a specific health issue. health-seeking behaviors for chronic pain in individ-
ɓɓ Norms, values, and beliefs uals working as day laborers on the eastern end of
ɓɓ Verbal, nonverbal, and symbolic Long Island, New York?
language
ɓɓ Day-to-day behaviors
Phenomenological Understand the meaning of an event What are the experiences of women currently receiv-
through understanding a group of people’s ing prenatal care whose previous pregnancy resulted
shared thoughts, feelings, and experi- in stillbirth?
ences as the result of living through that
experience.
Grounded theory To develop a theory or conceptual model What is the theory that explains the decision-making
that explains real-life processes, actions, or processes for women aged 18 to 45 who select surgi-
interactions between individuals. cal interventions subsequent to discovery they have
BRCA1 and BRCA2 genetic mutations?
QUALITATIVE RESEARCH DESIGNS—DEFINED

The strength of qualitative research is its ability to provide complex textual descriptions of how
people experience a given research issue. It provides information about the “human” side of an
issue … often [revealing the] contradictory behaviors, beliefs, opinions, emotions, and relationships
of individuals. Qualitative methods are also effective in identifying intangible factors, such as
social norms, socioeconomic status, gender roles, ethnicity, and religion … Although findings from
qualitative data can often be extended to people with characteristics similar to those in the study

Sciences
Chapter 6 Qualitative Research: Qualitative Designs | 127
population, gaining a rich and complex understanding of a specific social context or phenomenon
typically takes precedence over eliciting data that can be generalized … (Mack, Woodsong, Mac-
Queen, Guest, & Namey, 2005, pp.1–2).
This section will define each research design and summarize the unique and specific features
of each design. Only unique considerations regarding methods and rigor for each design will
be noted. Otherwise, all the previously presented information on sampling (non-probability
methods), data collection (i.e., narrative, document, verbal, image, triangulation, prolonged
engagement, persistent observation, etc.), data analysis (i.e., data saturation, immersion,
member checking, thick and rich descriptions, audit trails, etc.), and strategies to improve
the trustworthiness of the study apply for each of these designs.
Case Study Design

“A case study is a research approach that is used to generate an in-depth, multi-faceted
understanding of a complex issue in its real-life context” (Crowe et al., 2011, p. e1). It can
describe, explain, or explore why something happened or how something changed as the
result of something new occurring. There are numerous variations on case study designs;
for the purpose of this chapter, only the three main types of case studies will be explored:
intrinsic, instrumental, and collective (Crowe et al., 2011).
• Intrinsic case studies explore a unique occurrence of a real-life situation; the

focus in this type of research is on exploring and explaining the uniqueness of
the case.
• Instrumental case study research selects a single case from a group of cases that
when explored will help develop a better understanding of the real-life issue.
• Collective case studies are similar to the purpose of the instrumental case study
(to develop an understanding of the case); however, the difference is that in
the collective case study, several similar cases are combined. This allows the
researcher to develop an even more in-depth understanding of the issue beyond
what is possible by exploring a single instrumental case.
In summary, case study research has the power to “answer ‘how’ and ‘why’ type ques-
tions, while taking into consideration how a phenomenon is influenced by the context within
which it is situated. It enables the researcher to gather data from a variety of sources and to
converge the data to illuminate the case” (Baxter & Jack, 2008, p. 556).
Specific Features of Case Study Design
• Sample size is one. For example, a case can be a single person, one program, one
policy, one event, or one group of similar cases.
• Multiple types of data are typically collected (triangulation-interviews, obser-
vations, documents) and analyzed. Sometimes the data in a case study can even
include some quantitative data if it helps expound on the understanding of the
case. This practice would not be considered mixed methods research (more on
what constitutes mixed methods research can be found in Chapter 10).

Sciences
• While almost all qualitative studies (with the exception of grounded theory) have
a theoretical underpinning, it takes on special importance when conducting case
study research. In case study research, the underlying theory or framework guides
the researcher through all five stages of the research study.
• Kerlinger (1973) defines theory as “a set of interrelated constructs (concepts),
definitions, and propositions that present a systematic view of a phenomenon
by specifying relations among variables with the purpose of explaining and pre-
dicting phenomena” (p. 9). Examples of theories that have been used in health
science case study research include, but are not limited to, the “theoretical lens
of health promotion,” “Galtung’s Theory of Violence,” and the “Chronic Illness
Trajectory” model (Bradbury-Jones, Taylor, & Herber, 2014, pp. 138–139). When
theory is applied to the exploration of a particular case (case study research), it
helps broaden the applicability of the findings (transferability) to other settings
(Baxter & Jack, 2008; Crowe et al., 2011).
• The results of case study research are offered as examples that can shed light on
the how and why something happened. Case study research can serve as a model
to improve practice; it is the reader who determines if the findings will be useful
in their situation or setting (Padgett et al., 2017).
Example
Padgett et al. (2017) explain that despite recent health care policies and procedures, there
are still numerous preventable medical errors occurring. Instituting theory and practices
from high-reliability organizations (HRO) like the airlines into health care might mitigate
this risk. The researchers report the findings from a single case that “explored the introduc-
tion of a safety culture and subsequent improvements in patient safety” (p. 410) based on
the theory and practices from HRO. The research question guiding this study was, “What
components of HRO theory were useful in reducing risks to patient safety in a subacute
nursing facility?” (p. 412). The researchers collected numerous types of data (observations,
interviews, document review) from 14 nursing and respiratory care staff, as this program
was implemented in the subacute nursing home. The researchers report that the results of
this successful program implementation “can serve as a model for other organizations in
determining areas for improvement to improve patient safety” (p. 425).
In this example, the theoretical underpinning of the study was HRO theory and its
related practices and procedures that have been shown to reduce error and increase safety
in other high-risk industries. So despite the fact that this research study only examined
one unique case (patient safety program in a subacute nursing facility), because of the
theoretical underpinning of the study it can be offered as a model/example to other similar
health care organizations.
Focused Ethnography Design

To understand focused ethnography, a description of a full ethnography is first provided.
The primary reason for selecting this design is when the goal is to understand the day-
to-day aspects of a culture. The researchers would immerse themselves in the culture and
collect data over the course of several years. It is important throughout this process to

Sciences
take extra precautions to ensure that their own biases and cultural beliefs do not taint the
data collection and analysis. In this type of study, the researcher is looking specifically to
understand day-to-day behaviors and interactions among members. Ethnography research
studies all forms of communication (verbal, nonverbal, symbolic) and interactions (both
the explicit and implicit patterns of behavior).
In health science research, the researcher conducts a focused ethnography, which is used
to study a specific and clearly defined issue (Fusch, Fusch, & Ness, 2017). Focused ethnog-
raphies are commonly utilized in health science research to study a specific health-related
belief/issue/practice within a culture-sharing group. A culture-sharing group is any group
of individuals that share norms, behaviors, rituals, beliefs, and language. The health science
researcher conducting a focused ethnographic study might be studying similar aspects of the
day-to-day interactions within a culture-sharing group (shared norms; explicit and implicit
patterns of behavior; beliefs; verbal, nonverbal, symbolic communication) but only as they
relate to understanding the specific health-related issue from the culture group’s social and
cultural perspective (Higginbottom, Pillay, & Boadu, 2013).
Specific Features of Focused Ethnographic Designs
• The focus is on understanding a specific health-related issue within the culture

group versus understanding every aspect of the group’s culture.
• Focused ethnographies take significantly less time to conduct, with data collec-
tion ranging from weeks to months (Fusch, Fusch, & Ness, 2017).
• Gatekeeper – often used to gain entrance into this group. The researcher must
find a member of the group who will guide the researcher and make an introduc-
tion to the group. It takes time to build trust with the group.
• A significant portion of data collection occurs in the field (interviews and obser-
vations), and the researcher often keeps detailed field notes. Merriam (2009)
states that field notes should include detailed descriptions of the location (can
include sketches of interaction patterns) and the researcher’s reflections about
the environment.
• In addition to collecting interview and observation data from members of the
culture-sharing group, the researcher can identify a key informant from whom
to collect data. A key informant is someone that has special knowledge of the
group or a special relationship with the group. The researcher would use the key
informant to explain certain aspects of the group’s behavior and communication
patterns in more detail (Fusch, Fusch, & Ness, 2017). By using someone with
insider knowledge, it helps the researcher control bias in the study, which in turn
helps the researcher control misinterpretations. However, in this type of study,
the triangulation of data is vitally important; the varying sources of evidence in
combination yield credible findings.
Example
Ensign and Bell (2004) conducted a focused ethnography to document health-seeking
behaviors in 15- to 23-year-olds who were homeless and living in Seattle, Washington. Data

Sciences
collection included, but was not limited to, conducting interviews with 30 homeless youth,
observations of the youth at the youth clinic and on the street where the youth interacted
with the mobile van outreach staff, and data collected from key informants. In this study,
key informants were individuals in the same age range who had been homeless but were now
in stable housing. The key informants were used to help “clarify unfamiliar terms used and
situations described by the homeless youth” (p. 1242). The findings of the study included,
but were not limited to, differences based on age and gender. “Most youth under age 18
stated that they were often denied health care at hospitals because of their underage status”
(p. 1239), and those over 18 years old stated that health care bills were a contributing factor
in their homelessness. The researchers found “most male youth reporting embarrassment
about needing to seek care, and female youth reporting fears over safety issues while ill
and homeless” (p. 1239).
In this example, the culture-sharing group is homeless youth living in a specific area in
Seattle, Washington. The narrow focus on this study was to explore health-seeking behaviors
in this culture group. Interviews and observational data collection were conducted in the field
(where the youth hung out). The use of key informants who had previously been homeless,
in combination with numerous other data collection strategies (triangulation-participant
diaries, observation, field notes, focus groups, and individual interviews), was one of the
precautions that helped ensure that the researcher’s own biases did not taint the data col-
lection and analysis.
Phenomenological Design
The primary reason a researcher would select this design is when the researcher’s goal is
to develop an understanding of an event, life situation, or experience through the study of
people who have lived through the event/situation/experience. This design has also been called
the lived experiences, since the ultimate “goal in carrying out phenomenological research
is to gain an in-depth understanding of the lived experienced of the participants” (Chan,
Fung, & Chien, 2013, p. 6). As such, this type of research is heavily focused on collecting
data from first person experiences; the researcher combines the shared understanding and
the multiple perceptions and perspectives of the individuals to uncover the meaning of the
event, life situation, or experience.
Specific Features of Phenomenological Designs
• This research relies heavily on interviews with a purposefully selected group of

people (5 to 25).
• The researcher starts with loosely structured interview questions but then
allows the participants to drive the interview process so the participants’ voices,
experiences, and interpretations come through.
• Member checking is often used to ensure the participants’ voices, experiences,
and interpretations are expressed in data analysis.
• At times this research can explore emotionally charged events.
• Researchers must suspend preconceived notions or personal interpretations of
the data; this is accomplished through a practice called bracketing. Reflective

Sciences
bracketing involves the researcher intentionally putting aside their own knowl-
edge, experiences, and feelings to allow only the participants’ experiences and
interpretation to guide the research process (Chan, Fung, & Chien, 2013). For
example, during data immersion, once the researcher becomes aware of their own
thoughts/experiences/etc., they will actually put that [thought/experience/feeling
in a bracket in the margin] as a reminder to suspend preconceived notions or per-
sonal interpretations of the data.
Example
Creel and Tillman (2011) conducted a study that explored “the phenomenon of stigmatiza-
tion of obese persons by nurses” (p. 1331). The researchers conducted in-depth interviews
with eight patients who had “reported having experienced stigmatization from nurses
related to their weight” (p. 1337). The researcher found six common themes in the combined
patients’ experiences. Three of the six themes will be presented here. One theme was labeled
“unintentional harm,” and the researchers provided the following verbatim data as an exam-
ple: “They couldn’t get it in the crook of my arm and they made a comment that if I wasn’t so fat
their needles would work” (p. 1342). Another theme was labeled “anxiety in seeking health
care” presented with this supporting comment from a participant: “I try to stay away from
hospitals! It definitely does make you less likely to seek health care … because … you’re gonna be
treated like crap when you get there” (p. 1345). The last theme, marginalization, is illustrated
by the following comment shared by one of the participants: “I question whether or not I’m
getting the same quality of health care as somebody who is not fat, because they don’t see me as
deserving the same quality of health care” (p. 1345).
Reporting the findings from most qualitative research includes the use of low inference
data (direct quotes from participant interviews). However, it is particularly important in
phenomenological research that the participants’ voices, experiences, and interpretations
come through in analysis. For example, compare word choices between how the research
question is written (obese) versus the word used when sharing the data analysis ( fat). Through
the use of the participants’ exact words, the reader of this research is more fully exposed
to the lived experiences of these eight participants.
Grounded Theory Design

The primary reason a researcher would select this design is when the researcher’s goal is to
develop a theory about a “process, action, or interaction, shaped by the views of participants”
(Dill, 2017, slide 11). Often, grounded theory research questions focus on understanding
what the process was, how a situation unfolded, what strategies were used to navigate
the situation, or explain how/why something occurred (Dill, 2017). The outcome of this
research is a practical theory or conceptual framework about a real-world situation. The
theory/framework is created by exploring the real-life experiences, views, and actions of the
participants (Merriam, 2009). Dunn, Margaritis, and Anderson (2017) add that grounded
theory “goes beyond the description of the phenomena to the development of a theory or
model, designed to better explain the process and actions, which could lead to improved
methods” in health care delivery (p. 199).

Sciences
Specific Features of Grounded Theory Designs
• Often there is very little previous peer-reviewed literature on the topic.

• Often uses a larger sample than other qualitative studies; sample size can range
from 20 to 50 participants.
• Does not use a theoretical framework in developing the research question or
study design; rather, the researcher gathers data, and it is the interplay between
data collection and data analysis that refines the focus of the study.
• Sampling method often involves theoretical sampling methods as opposed to
purposive sampling. Glaser (1978) explains the process of theoretical sampling
as the interplay between collecting, coding, and analyzing data in order to deter-
mine which participant should be sampled next. This fluid yet purposeful way of
sampling allows the researcher to select participants over the course of the study
based on what data is needed to build the theory.
■■ Draucker, Martsolf, Ross, and Rusk (2007) maintain that the use of theoret-
ical sampling assists researchers in determining which avenues need further
exploration in the development of the theory. The authors also express that
the researcher’s “audit trail will be strengthened by having a theoretical sam-
pling guide for each category” or core component of the theory (p. 1146).
• The primary focus of a grounded theory study is not solely to understand the
experiences of participants; rather, the primary focus is uncovering processes and
actions which are rooted in participant experience.
Example
Nelson, Henriksen, and Keathley’s (2014) study sought to develop a theory of a mother’s recov-
ery process when her son was struggling with a substance abuse problem. The researchers
noted that previous literature existed but only related to coping with a spouse’s substance
abuse. None had explored the unique relationship between mother and son. The researchers
used theoretical sampling “throughout the data collection process as we chose new cases
[participants] to compare with those already studied until we found no new or discrepant
data” (p. 5). The researchers found “that regardless of age, ethnicity, social status, education,
or career” all of the women “experienced a process highlighted by maternal expectations”
(p. 8) that occurred in three stages: “(a) the pre-addiction stage in which mother and son
did not experience abnormal relational stress or conflict, (b) the addiction stage in which
mother and son experienced extreme relational stress and conflict, and (c) the mother’s
recovery stage in which the mother experienced confusion in her relationship with her son
and his addiction leading to her own recovery and wellness” (p. 1).
It is clear from this example above that grounded theory research focused on the
construction of a theory, directly derived from the participants’ real-life experiences. In phe-
nomenological research the purpose is finding the meaning of the event; in grounded theory
the purpose is on developing a theory that illuminates the processes and/or stages a person
goes through when dealing with a real-life situation. As the researchers built the theory,
sampling participants was ongoing over the course of the study and was based on selecting
participants that could expand, challenge, or confirm the emerging theoretical components.

Sciences

This chapter provided an overview of four qualitative research designs, the unique features
of each design, and provided an example of how each design was used in health science
research. Table 6.3 provides an illustration of the structural issues within the design and
methods that are often used in each design. The next chapter will explore experimental
and quasi-experimental designs.
TABLE 6.3. Qualitative Designs: Purpose, Special Features, and Methods
Special Features of the Design (SF)

Methods: sampling (S), data collection
(DC), & data analysis (DA) specific to
Design Purpose the design
Case study An in-depth exploration of a case to illustrate (SF) Must use a theoretical framework to
an outcome, a procedure, or how something structure research
changed over time
(S) One person, program, or event
(DC) Triangulation
(DC) Can include numerical data
Focused Ethnography Understand the culture of a group (S) Gatekeeper

related to a specific health issue (DC) Key informant
ɓɓ Norms, values, beliefs, practices (DC) Field notes
ɓɓ Verbal, nonverbal, and symbolic language (DC) Triangulation
ɓɓ Day-to-day behaviors specific to the issue
under study
Phenomenological Understand the meaning of an event through (DA) Reflective bracketing

understanding a group of people’s shared (DA) Member checking
thoughts, feelings, and experiences as the result (DA) Low inference data
of living through that experience
Grounded theory To develop a theory or conceptual model that (SF) Does not use a theoretical framework to
explains real-life processes, actions, or interac- structure study
tions between individuals (S) Theoretical sampling
The variations of the word theory have been used in several different contexts. The
outline below is a quick review of the terms and context:
• Theoretical sampling: a type of non-probability sampling

■■ Never used for quantitative sampling
■■ Used in qualitative research to sample, especially in grounded theory
research

Sciences
• Theoretical underpinning or using a theoretical framework: use a theory

■■ A theory is something that has already been established; it defines and
explains the relationship between variables to provide a foundational
understanding of the issue. Used in qualitative, especially case study
research, to help explain the case based on an established foundation,
which assists in transferability.
■■ Used in quantitative to make a data collection tool, intervention, or
define variables.
• Theoretical framework: make a theory
■■ Purpose of grounded theory research
STUDY PRACTICE
At this point, you should begin to see how previously presented information applies to qualita-
tive research designs. The second half of the book (Chapters 6–10) will require that you begin
integrating the material presented in the first half of the book (Chapters 1–5). Begin putting it
all together. Make new outlines that include the designs; return to previous chapters reread the
examples, with special attention paid to the qualitative information; or add additional colors
to your flash cards to specify research designs. Pay special attention to the information on
sampling (non-probability methods), data collection (i.e., narrative, document, verbal, image,
triangulation, prolonged engagement, persistent observation, etc.), data analysis (i.e., data
saturation, immersion, member checking, thick and rick descriptions, audit trails, etc.), and
strategies to improve the trustworthiness of the findings that apply for each of these designs.
ɠɠ Case study design ɠɠ Phenomenological design
ɝɝ Intrinsic ɝɝ Lived experiences
ɝɝ Instrumental ɝɝ Reflective bracketing
ɝɝ Collective ɠɠ Low inference data
ɠɠ Focused ethnography
ɝɝ Gatekeeper
ɝɝ Key informant

1. The purpose of this study was to explore how young college-age women negotiate
or fail to negotiate safer sex with their casual partners. The researcher sought to
develop a framework of decision-making processes and actions. For example, how do
women sort through medical information or avoid information about risk reduction?
Who are the trusted support people, what self-care actions are taken, etc.?
Which study design would be most appropriate for this study?
a. Case study
b. Focused ethnography

Sciences
c. Phenomenological
d. Grounded theory
2. The purpose of this study was to explore the shared experiences of women when
they were presented with a diagnosis of heart disease. The researcher purposefully
selected 20 women with whom to conduct in-depth interviews. The researcher
read and reread transcripts of the verbal interview data to identify common themes
among the experiences. The researcher found that the most common experiences
the women had related to a theme, labeled Women can get heart disease, too.
a. Case study
c. Phenomenological
d. Grounded theory
3. The purpose of this study is to chronicle an individual’s journey from a self-proclaimed
lifelong smoker of unfiltered cigarettes for 50 years to a redefined identity as a non-
smoker subsequent to diagnosis of lung cancer.
a. Case study
c. Phenomenological
d. Grounded theory
4. What are the perceptions and behaviors related to vaccinations for influenza and
pneumonia in the U.S. Myanmar refugee community? Little is known about com-
munity members’ beliefs and behaviors surrounding the role vaccinations take in
preventive health care.
a. Case study
c. Phenomenological
d. Grounded theory
5. “New nurses are expected to quickly transition from nursing student to critical care
nurse with limited transitional time” (Lewis-Pierre, Kovacich, & Amankwaa, 2017,
p. 1207). What is the theory that explains the process of “workplace readiness and
needs of new graduates entering the ICU from the viewpoint of managers, clinical
educators, preceptors, and new registered nurse (RN) graduates?” (p. 1207).
Which study design would be most appropriate for this research study?
a. Case study
c. Phenomenological
d. Grounded theory

Sciences
6. What are health care providers’ “frustrations and successes in attempting to provide
effective, sustainable pain management options for undergraduate students with both
lasting injuries and chronic diseases” through the universities’ campus health services
at two northeastern universities. The findings of this study will provide an under-
standing of the health care providers’ experiences working with students living with
chronic pain (Nowakowski, Barningham, Buford, Laguerre, & Sumerau, 2017, p. 1885).
a. Case study
c. Phenomenological
d. Grounded theory
7. Researchers purposefully selected 11 people from a respiratory care outpatient clinic
who are “living with chronic respiratory failure” to explore the “transition from
spontaneous breathing to reliance” on long-term mechanical ventilation. Analysis
of the “transition journey was revealed to be a time of psychological, physical, and
spiritual challenge. ‘Sustaining self’ was revealed as the essence of ventilator users’
transition experience” (Briscoe & Woodgate, 2010, pp. 57–58).
a. Case study
c. Phenomenological
d. Grounded theory
8. A rural community in the Midwest of the United States has an unusually high prev-
alence of diabetes; however, nothing is known about the community’s day-to-day
explicit and implicit patterns of communication, behavior, and norms regarding
following dietary recommendations for individuals living with type 2 diabetes. The
researcher is given permission by the community leaders to observe ceremonies
and interview community members. The researcher collects data by conducting
15 face-to-face interviews and collects about 200 hours of observations (attending
community events: weddings, festivals, funerals, etc.) over a 6-month period.
a. Case study
c. Phenomenological
d. Grounded theory
9. A commonality among administrators that cover the night shifts in hospitals is
that there are fewer institutional resources available to them: reduced numbers of
employees across all members of the health care team and virtually no upper man-
agement personnel on site during these hours. The researchers found that despite the
reduced resources, the administrative supervisors’ viewed themselves as champions of
patient safety. They valued communication and engaged best practices that promoted
a culture of patient safety despite the lack of quality, risk management, or safety

Sciences
personnel being on site during the night shift. The findings of this study revealed
a conceptual framework that illustrated the process of achieving best practices in
patient safety despite limited resources. Adapted from Weaver and Lindgren (2017).
a. Case study
c. Phenomenological
d. Grounded theory
10. What are the strategies used in the drowsy-driving prevention program offered
through the Oakwood County high school driver education programs? This unique
program was developed using the Health Belief Model in the context of drowsy
driving prevention. It can serve as a model for other school districts.
a. Case study
c. Phenomenological
d. Grounded theory
STUDY ACTIVITIES
Reducing notes from 3:1 recommendation cuts across all learning styles. The first suggestion
for studying is to start making flash cards for every new bolded/italicized term presented
in this chapter.
As previously stated, this chapter builds on concepts presented in the first half of the
book. Now take your flash cards created from Chapters 1–5, and ask yourself: Does
the word on this flash card apply to concepts presented in this chapter?
Challenge yourself by asking different questions; for example, using five stages of research
and corresponding action steps (Table 1.1), develop a hierarchal outline using the new
flash cards for each design integrated with the relevant flash cards from Chapters 1–5.
Create a table of the special features for each design presented in this chapter, or from
memory, list the special features of each design.
REFERENCES
Baxter, P., & Jack, S. (2008). Qualitative case study methodology: Study design and implementation
for novice researchers. Qualitative Report, 13(4), 544–559. Retrieved from http://nsuworks.nova.
edu/tqr/vol13/iss4/2
Bradbury-Jones, C., Taylor, J., & Herber, O. (2014). How theory is used and articulated in qualitative
research: Development of a new typology. Social Science & Medicine, 120, 135–141. doi:10.1016/j.
socscimed.2014.09.014
Briscoe, W. P., & Woodgate, R. L. (2010). Sustaining self: The lived experience of transition to long-
term ventilation. Qualitative Health Research, 20(1). 57–67. doi:10.1177/1049732309356096

Sciences
Chan, Z. C., Fung, Y., & Chien, W. (2013). Bracketing in phenomenology: Only undertaken in the data
collection and analysis process. Qualitative Report, 18(30), 1–9. Retrieved from http://nsuworks.
nova.edu/tqr/vol18/iss30/1
Creel, E., & Tillman, K. (2011). Stigmatization of overweight patients by nurses. Qualitative Report,
16(5), 1330–1351. Retrieved from http://nsuworks.nova.edu/tqr/vol16/iss5/7
Crowe, S., Creswell, K., Robertson, A., Hubby, G., Avery, A., & Sheikh, A. (2011). The case study approach.
BMC Medical Research Methodology, 11(100), e1–e9. doi:https://doi.org/10.1186/1471-2288-11-100
Dill, L. J. (2017). Engaging in qualitative research methods: Opportunities for prevention and health
promotion [presentation slides]. Retrieved from https://prevention.nih.gov/docs/programs/mind-
the-gap/mtg2017-dill-presentation.pdf
Draucker, C. B., Martsolf, D. S., Ross, R., & Rusk, T. B. (2007). Theoretical sampling and cat-
egory development in grounded theory. Qualitative Health Research, 17(8), 1137–1148.
doi:10.1177/1049732307308450
Dunn, P. J., Margaritis, V., & Anderson, C. (2017). Use of grounded theory in cardiovascular research.
Qualitative Report, 22(1), 197–212. Retrieved from http://nsuworks.nova.edu/tqr/vol22/iss1/11
Ensign, J., & Bell, M. (2004). Illness experiences of homeless youth. Qualitative Health Research,
14(9), 1239–1254.
Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs—
Principles and practices. Health Services Research, 48(6), 2134–2156. doi:10.1111/1475-6773.12117
Fusch, P. I., Fusch, G. E., & Ness, L. R. (2017). How to conduct a mini-ethnographic case study:
A guide for novice researchers. Qualitative Report, 22(3), 923–941. Retrieved from http://go.
galegroup.com.proxy.library.stonybrook.edu/ps/i.do?p=AONE&sw=w&u=9211haea&v=2.1&it=r
&id=GALE%7CA491033026&asid=001200ad494c12dbea7cb43f2ef3d13c
Glaser, B. (1978). Theoretical sensitivity: Advances in grounded theory. Mill Valley, CA: Sociology Press.
Higginbottom, G., Pillay, J. J., & Boadu, N. Y. (2013). Guidance on performing focused ethnographies
with an emphasis on healthcare research. Qualitative Report, 18(17), 1–6. Retrieved http://www.
nova.edu/ssss/QR/QR18/higginbottom17.pdf
Kerlinger, F. N. (1973). Foundations of behavioral research (2nd ed.). New York, NY: Holt, Rinehart
and Winston.
Lewis-Pierre, L. J., Kovacich, J., & Amankwaa, L. (2017). The application of grounded theory: An
example from nursing workforce research. Qualitative Report, 22(5), 1269–1283. Retrieved from
http://nsuworks.nova.edu/tqr/vol22/iss5/7
Mack, N., Woodsong, C., MacQueen, K. M., Guest, G., & Namey, E. (2005). Qualitative research meth-
ods: A data collector’s field guide. Retrieved from http://www.fhi.org
John Wiley & Sons.
Nelson, J. A., Henriksen, R. C., & Keathley, R. S. (2014). Mothers of sons with substance use disor-
ders: A grounded theory approach revealing maternal expectations and three stages of change.
Nowakowski, A. C., Barningham, K. E., Buford, C. D., Laguerre, M., & Sumerau, J. E. (2017). “That
pain is genuine to them”: Provider perspectives on chronic pain in university student populations.
Padgett, J., Gossett, K., Mayer, R., Chien, W., & Turner, F. (2017). Improving patient safety through
high reliability organizations. Qualitative Report, 22(2), 410–425. Retrieved from http://nsuworks.
nova.edu/tqr/vol22/iss2/4
Weaver, S. H., & Lindgren, T. G. (2017). Getting safely through the shift: A qualitative exploration of
the administrative supervisor role. Journal of Nursing Management, 2017(25), 430–437.

Sciences
UNDERSTANDING QUANTITATIVE
RESEARCH: EXPERIMENTAL/
QUASI-EXPERIMENTAL DESIGNS
INTRODUCTION
7
CHAPTER GOALS:
This chapter discusses research designs that fall under the categories of
experimental and quasi-experimental. There are a number of research
designs that fall under the heading of experimental and quasi-experimental
research; however, this chapter will concentrate on an in-depth discussion • To introduce the defining features
of the following experimental designs (pretest-posttest control group, that are associated with each exper-
Solomon four-group, posttest-only control group, and within subjects) imental and quasi-experimental
and quasi-experimental designs (nonequivalent [pretest and posttest] research design discussed.
control group, single group time-series, and control group interrupted • To provide sufficient information
designs. This chapter will provide a description of each aforementioned to connect sampling, data collec-
design with examples, as well as how a researcher decides which research tion, and data analysis methods to
methods (sampling, data collection, and data analysis) to use while plan- each design.
ning an experimental or quasi-experimental research study (Table 7.1). • To provide students with working
Lastly, validity and threats to validity, as they pertain to each design definitions of the terms in italics.
presented in this chapter, will be discussed.
• To provide students with an under-
standing of the strengths and
EXPERIMENTAL AND QUASI- weaknesses inherent in each design.
EXPERIMENTAL DESIGNS—DEFINED
Experimental designs are the most rigorous of all quantitative designs
and have the greatest degree of internal validity and external validity. LEARNING OBJECTIVES:
As noted in Chapter 5, validity refers to the authenticity of the results of
a study, and internal validity refers to the extent to which the results of BY THE COMPLETION OF THIS CHAPTER,
a study are true. When applying this concept to a research study, a study THE STUDENT WILL BE ABLE TO:
has internal validity when the experimental results are directly related
to the intervention(s)/treatment(s) used in the study, rather than a con- • Define the terms: bias, confounding
founding variable. A confounding variable is one that the researcher is variable, dependent variable, error,
unaware is having an influence on the study, as such; it is not measured external validity, independent variable,
or observed. It exists, but its influence on a variable (i.e., independent, internal validity, and nonequivalent;
dependent) is not easily assessed. When a study has internal validity, a • Recognize the character-
researcher can draw accurate conclusions about cause and effect and istics of experimental and
other relationships within the data. We will come back to the concepts quasi-experimental research designs;
of internal validity and confounding variables later in this chapter. • Compare appropriate research
Quasi-experimental designs share many similarities with experimental methods utilized for each non-
designs. Some of these similarities may include a control group as well experimental research design; and
as an intervention. The main difference between these two designs is • Demonstrate the ability to choose
the most appropriate design for
sample research ideas.
139

Sciences
that a quasi-experimental design does not use probability sampling methods and does not
randomly assign study participants to an experimental or control group.
Stages Action Steps
EXPERIMENTAL RESEARCH
Experiments involve highly controlled and systematic procedures in an effort to minimize
error and bias, which also increases the level of confidence that the outcome was a direct
result of the intervention or treatment. (Refer to Chapter 5 to review type I and type II errors
and the concept of selection bias.) All of these designs use participants who were randomly
selected from the population and randomly assigned to the experimental or control groups.
In Health Science, the words interventions and treatments are often used interchangeably
but only in certain circumstances. When the health science researcher is doing medical
research (i.e., testing a medical treatment plan, testing a new drug treatment, testing a new
device, etc.), the researcher will use the words intervention and treatment interchangeably.
If the researcher is testing a complex program that has been designed to change health
behaviors, attitudes, or environmental and social conditions that impact health outcomes,
this is only referred to as an intervention (Bowen et al., 2009; Craig et al., 2008). For exam-
ple, testing the effectiveness of a new smoking cessation program (only called intervention)
versus testing the efficacy of a new medication (can be called an intervention, a treatment, a
medical intervention). For the purpose of this book, we will use the word treatment to refer
to clinical studies of drugs and other clinical medical treatments. The word intervention
will be used to refer to complex programs designed to change behaviors, attitudes, envi-
ronmental and social conditions.
Experimental designs have the following four elements in common:
Manipulation of variables – something that is purposely changed by the researcher in the

study. In health science research, it typically is the treatment or the new intervention
that is being tested. Two commonly used categories of variables are the dependent
and independent variable. An independent variable is presumed to have an effect
on a dependent variable, and a dependent variable “depends” on an independent
variable. We will come back to these terms later in the chapter.
Control – used to prevent outside factors from influencing the outcome of the study. For
instance, let’s say we create two groups (experimental and control) that are “equiv-
alent” to one another, with the exception that one group gets the intervention or
treatment (experimental) and the other group (control) does not get the intervention
or treatment. If there are differences in the outcome between these two groups,

Sciences
Chapter 7 Understanding Quantitative Research: Experimental/Quasi-Experimental Designs | 141
then the differences must be due to the only element that differed between the two
groups: the intervention/treatment.
Random sample – As noted in Chapter 5, using one of the probability sampling methods
requires the researcher to randomly select participants, which is thought to yield a group
of individuals representative of the population at large. This relates to another concept
discussed in Chapter 5, external validity, which means the results of the study can be
generalized to similar groups of people in different settings or other groups of people.
Random assignment – Once a random sample has been selected, random assignment
means that everyone in the random sample has an equal chance of being assigned to
the experimental or control group. In addition, confounding variables (both known
and unknown) are equally distributed between the experimental and control groups
(Harris et al., 2006). Random assignment ensures that the sample is representative
of the population. “It ensures that alternative causes are not confounded with a unit’s
treatment condition …” and “It reduces the plausibility of threats to validity by dis-
tributing them randomly over conditions” (Shadish, Cook, & Campbell, 2002, p. 248).
Research Notations
Research notations are a shorthand used by researchers to illustrate a specific research
design. The research notations noted below (O and X) were developed by Campbell and
Stanley (1963, p. 6). Each discipline has slight variations on this notation; for the purpose
of this chapter, the notation provided will suffice. Review the notations below, explain what
each of these notations mean, and then use an example to explain how these components
interact with one another in a research experiment. Figure 7.1 illustrates the placement of
the letters (R, O, X), and the numbers in subscript describe elements of the study.
O = an observation (also referred to as a dependent variable); O represents the outcome

of the influence of the independent variable (X)
X = the treatment, or intervention, the experimental group is exposed to (also referred
to as an independent variable), the effects of which are measured; the experimental
group receives the treatment or intervention, the control group does not. In an
experimental design, the researcher manipulates at least one variable. The absence
of X indicates the control group.
R = indicates that participants in the group have been randomly assigned to the exper-
imental or control group
Time: the left-to-right dimension sequential
order of procedures in the experiment;
FIGURE 7.1 An explanation of the placement of letters in
this is sometimes noted by an arrow
scientific notation.
Subscripts: subscript numbers show differ-
ence. For example, when the number
Time
follows the R, it shows there are two
X’s and O’s in the same row apply to
groups. When the number follows the O, R1 O X O the same group.
it shows that the study has several differ- R2 O O Row R1 has 2 O’s and 1 X
Row R2 has 2 O’s but no X
ent observations (data collection points).

Sciences
The notation below illustrates that there are two groups (R1 and R 2) as indicated by the
subscript numbers. In addition, because there are no numbers following the O, there is
one observation.
R1 O X O
R2 O O
The difference between the notation below and the previous one is that there are numbers
(1, 2, 3) following the O, which represent three different observations or data collection points.
R1 O1 O2 O3 X O1 O2 O3
R O1 O2 O3 O1 O2 O3
Remember, scientific notation is simply a shorthand that researchers use to explain the
design of the study. A researcher reviewing the above examples would not know what the
observation(s) were; however, seeing these two examples, the researcher would immediately
realize the first study had a single data collection measurement (observation), while the
second example had three different data collection measurements (observations), collected
over time, both prior to and after the intervention.
Consider the example below, presented earlier in the chapter, to explain the interactions of the
components listed in the study above in which there was one observation (see also Figure 7.2).
Researchers working in a large primary care clinic want to test the effectiveness of a new
smoking cessation intervention that includes education, six weeks of support group meetings,
FIGURE 7.2 An illustration of scientific notation applied to a narrative description of study.

Source: https://www.socialresearchmethods.net/kb/desintro.php.
Time: Elements on the left occur

before elements on the right
X in first row refers to new

R1 O X O smoking cessation
intervention
Random assignment of
randomly selected
R2 O O Lack of X in second row
population to experimental
refers to control group
or control group
which gets standard
program
O indicates a data collection point.

The O on the left indicates a pretest
measurement was taken and the O
on the right indicates a posttest
measurement was taken.

Sciences
and nicotine replacement. They plan to test the intervention by comparing it to the smoking
cessation program currently used in the clinic. The researchers use a probability sampling
method that allows them to select a random sample from the population (patients in the
clinic who are current smokers). Participants that consent to participate in the study are
randomly assigned to the experimental (R1) or control group (R 2). Researchers administer
the same survey (O) to both groups, in which participants are asked to answer questions
related to their smoking behaviors. In this type of design, this first observation is called a
pretest. Participants in both groups are then given an intervention: administration of the
new smoking cessation intervention for the R1-experimental group and the current smoking
cessation program for the R 2-control group. At this point, the only difference between the
experimental and control group is the intervention (X). Participants are then given the
same survey (O) at the conclusion of the study. This observation is called a posttest. After
data analysis has been performed, the researchers look to see if there is a difference in the
results of the observations (surveys) between the experimental and control group.
EXPERIMENTAL RESEARCH DESIGNS

The following four experimental research designs noted below will be discussed in
this chapter.
1. Pretest-Posttest Control Group

2. Posttest-Only Control Group
3. Solomon Four-Group
4. Within Subjects
These designs do not encompass all experimental research designs; they simply repre-
sent a sample. Before proceeding with the specifics of each of these designs, as a review,
remember that for each of these designs, a random sample was collected using one of the
probability methods discussed in Chapter 5. Participants in the random sample were then
randomly assigned to the experimental or control group. Everyone in the random sample
had an equal chance of being assigned to the experimental or control group. This practice
greatly increases the external validity of a study.
Pretest-Posttest Control Group Design

The pretest-posttest control group design is made up of two groups that have been randomly
sampled and then randomly assigned to either R1 or R 2, both of which receive an observation
prior to and at the same time afterward. The only difference between the two groups is that
the R1 (experimental) group receives an intervention and the R 2 group (control) does not.
The research notation for this particular design is presented below.
R1 O X O
R2 O O

Sciences
The term double-blind is defined in Chapter 5. Clinical trial is defined by the National
Institutes of Health (n.d.) as “A research study in which one or more human subjects are pro-
spectively assigned to one or more interventions (which may include placebo or other
control) to evaluate the effects of those interventions on health-related biomedical or
behavioral outcomes” (para. 4). These types of clinical research studies often use random-
ized and a double-blind control group. Wijesinghe et al. (2017) conducted a clinical study
that included randomization and a double-blind control group on a group of patients with
rheumatoid arthritis who were resistant to “… first-line non-biologic Disease Modifying Anti
Rheumatic Drugs” (p. 1). The researchers compared the “efficacy and safety of low dose
rituximab-methotrexate combination compared to leflunomide-methotrexate combination”
(p. 3). Results of the study showed that low-dose rituximab and leflunomide were equally
effective when added to methotrexate in controlling the disease.
Because participants in this design are randomly sampled, they should be a representative
sample of the population at large. Because these randomly sampled participants are then
randomly assigned to the experimental or control group, any differences in the posttest
results should be due to the intervention rather than any possible differences between the
two groups. Neither the participants nor the researchers knew who was in the experimental
or control group, which increases the internal validity of this study. A possible threat to the
external validity of this design is the pretest effect.
Posttest-Only Control Group

The posttest-only control group is one of the simplest experimental designs. It is made up
of two randomly assigned groups; one group receives a treatment, the other group does not.
Sometimes it is not possible to include a pretest in an experiment for reasons including, but
not limited to, the fact that a pretest may not exist and/or the experiment may be study-
ing the effects of a life event (i.e., natural disaster) in which it was not possible to pretest
individuals prior to the event. In this design, the researcher is interested in determining
if there is a difference between the two groups in the observations made after the inter-
vention is given to the experimental group. This is a relatively simple design for assessing
cause-effect relationships. An example of this particular design is presented using the
research notation below.
R1 X O
R2 O
A researcher is interested in studying the effectiveness of an intervention for post-

traumatic stress disorder (PTSD) following a devastating hurricane. A random sample of
individuals who were members of a community that experienced a devastating hurricane
and had subsequently been diagnosed with PTSD were selected to participate in the study.
Participants were randomly assigned to the experimental or control group. The experi-
mental group (R1) was exposed to the PTSD intervention; the control group was not. Both
groups were then given a PTSD checklist to complete. Following statistical analysis of the
data, the researcher then ascertains if there is a difference in the observations (O) between
the two groups.

Sciences
Solomon Four-Group Design

The Solomon four-group design is one of the most rigorous designs in quantitative research.
This design has the highest level of controls for threats to internal validity by controlling for
pretest effects. The Solomon four-group design combines the pretest-posttest design with
the posttest-only design into one research design. The research notation for the Solomon
four-group design is as follows:
R1 O X O
R2 O O
R3 X O
R4 O
In the Solomon four-group design, randomly selected subjects are randomly assigned to
one of four groups (Solomon, 1949):
1st row (R 1): Participants receive a pretest observation, an intervention, and a

posttest observation.
2nd row (R 2): Participants receive a pretest and posttest observation; this group does not
receive an intervention.
3rd row (R3): Participants receive an intervention and a posttreatment observation; this
group does not receive a pretest observation.
4 row (R 4): Participants receive a posttest observation; this group does not receive an
th
intervention or pretest observation.
Narrative example: Consider the following narrative description of the scientific

notation. A researcher is interested in studying the effect of computer-assisted cognitive
training on memory loss in a population of adults between the ages of 50 and 60. Par-
ticipants are randomly selected and then randomly assigned to groups R1, R2, R3, and R4.
Table 7.2 depicts the observations (memory tests) and interventions (computer-assisted
cognitive training) received by each group.
What Are We Looking for, and How Are the Results Interpreted?
As this is the most rigorous design with the greatest degree of external and internal controls,
it is important to provide an explanation of what purpose each of the groups in this design
serve and how the results are interpreted.
Comparison of the results of the four groups in the Solomon four-group design allows the
researcher to determine if the results are valid—meaning, has the study been influenced by
pretesting. Table 7.3 illustrates how pretest effects are controlled for in this design and how
researchers combine analyses among the four groups (R 1, R 2 , R3 , R 4) to determine results
of the study. The information in this table has been adapted from McGahee and Tingen
(2009); Morgan, Gliner, and Harmon (2000b); Phan and Ngu (2017); and Solomon (1949).

Sciences
TABLE 7.2 Solomon Four-Group Design: Narrative of Research Notation
Pretest observation Treatment Posttest observation

Memory test Computer-assisted cognitive training Memory test
R1 yes yes yes
R2 yes no yes
R3 no yes yes
R4 no no yes
Adapted from W. M. K. Trochim (2006), https://www.socialresearchmethods.net/kb/desintro.php.
This particular design controls for many threats to internal validity, such as threats of
history and maturation, because it has a control group. If an outside event occurred during
the course of the study that could impact the results or, if during the course of the study par-
ticipants matured, these changes would be seen in both the experimental and control groups.
TABLE 7.3. Solomon Four-Group Design: How Results Are Determined
Compare Results are the same: Results are different:
Posttest results of R1/R2 with Pretesting did not influence R1/R2 and R3/R4 results are different. Pretesting influenced the
the posttest results of R3/R4 results of the study. posttest results (O) of the experiment.
ɓɓ Meaning the results are most likely due to the pretest, not
the experiment’s intervention.
Posttest results of R1 and Pretesting did not influence R1 and R3 results are different. Pretesting influenced the inter-
posttest results of R3 the intervention. vention (X).
ɓɓ Meaning the pretesting most likely changed the impact or
the effectiveness of the experiment’s intervention.
Pretest result of R2 and Threats to internal validity R2 and R4 results are different. Threats to internal validity in-
posttest results of R4 (history or maturation) were fluenced the study’s results.
controlled. ɓɓ Meaning that history or maturation effects are present in
the two control groups. Therefore, the researcher must
assume these threats to internal validity could have influ-
enced the results of the study.
The use of the Solomon four-group design in nursing research. Southern Online Journal of Nursing Research, 9(1), e1–e7; Randomized experi-
mental designs. Journal of the American Academy of Child and Adolescent Psychiatry, 39, no 8, 1062–1063; Undertaking experiments in social
sciences: Sequential, multiple time series designs for consideration. Educational Psychology Review, 29(4), 847–867; An extension of control group
design. Psychological Bulletin, 46(2), 137–150.
Adapted from Martyn Shuttleworth, “Solomon Four-Group Design: How Results are Determined,”
https://explorable.com/solomon-four-group-design. Copyright © 2009 by Explorable.com.

Sciences
If the researcher wanted to add another layer of control to the study’s internal validity,
a placebo could be added by having R 2 and R 4 groups engage in a non-intervention activity
(e.g., jigsaw puzzle making), while the R1 and R3 groups are given the cognitive intervention).
As stated in Chapter 5, controlling for attrition is a very complicated issue; the researchers
could increase the size of the sample to compensate for losses to follow-up.
Within-Subjects Design (Repeated Measures Design)

A within-subjects design refers to a design in which the subjects participating in the research
are exposed to each intervention, otherwise known as the independent variable. The same
group of subjects are exposed to more than one intervention. Each subject’s performance
is repeatedly measured; hence the within-subjects design is sometimes called the “repeated
measures design.” An example of this design using the research notation below will help to
illustrate how the research is conducted.
R1 X1 O1
X2 O2
Example of Within-Subjects (Repeated Measures) Design

A simplified summary of Fisher, Rolls, and Birch’s (2003) within-subjects study is presented
as an example. The purpose of the study was to determine whether exposure to different
serving sizes (allowing children to self-serve their quantity of food versus serving children
an oversized portion of food) was related to the amount of food a child would eat during
lunchtime. Thirty randomly selected preschool children served as the sample (R1). The chil-
dren were allowed to self-serve a portion of macaroni and cheese in one intervention (X1).
In the other intervention, the same group of children were served an oversized portion of
macaroni and cheese (X 2). Data were collected by the research teams’ observation of calories
consumed during lunch in both interventions (O1 and O2). Researchers found that children
consumed fewer calories when they were allowed to serve themselves versus being given
an oversized serving of the macaroni and cheese. The researcher concluded serving size is
an environmental factor associated with calorie consumption.
The strength of this particular design is that it does not require a large number of study
participants, as the same participants receive multiple interventions. The use of one group
of subjects helps control for the internal threats to validity of history and maturation. Since
all participants are exposed to the same intervention, subjects serve as their own control
for individual differences between subjects.
Weaknesses of this design include carry-over effect, fatigue, and practice effects. The
carry-over effect is displayed when exposure to one intervention impacts the performance of
study participants for the next intervention. Fatigue can be seen as boredom, exhaustion, or
participant lack of interest after taking part in multiple treatments, tests, etc. Lastly, practice
effect refers to an improvement in one’s performance on a test due to repeated exposure to
the same test materials. This effect can make it difficult to ascertain if the results observed
are due to the treatments/intervention or are a result of practice.
Within-subjects design can also fall under the quasi-experimental study designs if
non-probability sampling is used to obtain the participants for the study, just like the control

Sciences
FIGURE 7.3 A list of designs that can fall under experimental

or quasi-experimental designs.
group time-series designs that are presented below.
If a control group time-series study used probability
sampling and random assignment to experimental
Experimental Research Designs and control groups, it would be an experimental
Probability sampling & Non-probability sampling &
random assignment to groups no random assignment to groups
design. Review Figure 7.3, as any design listed
in this figure can fall under the experimental or
Experimental Quasi-experimental quasi-experimental categories based on how par-
Pretest/Posttest Pretest/Posttest ticipants are sampled and assigned to experiment
Posttest only Posttest only
Within-subjects Within-subjects
or control groups.
Control group time-series Control group time-series
QUASI-EXPERIMENTAL RESEARCH DESIGNS

The following three quasi-experimental research designs noted below will be discussed in
this chapter.
1. Nonequivalent (pretest and posttest) control group

2. Simple time-series
3. Control group time-series
Quasi-experimental research can be defined as resembling (but not being true) exper-
imental research. Although attempts can be made to make both groups very similar
(i.e., similar communities, similar classrooms within the same school), the partic-
ipants are not randomly assigned to the experimental and control group, and thus
these groups are considered to be nonequivalent.
SIDE BOX 7.1 How the researcher sampled is the way to determine whether a study is a true experi-
ment or a quasi-experiment.
Experimental designs must use one of the probability sampling methods and random assignment to the
experimental and control groups, while quasi-experimental designs use one of the non-probability sam-
pling methods and/or do not randomly assign participants to experimental or control groups.
This introduces an important limitation to this particular type of study; without ran-
domization, there is no guarantee that the differences between the groups are due to the
intervention, but rather the results might be due to chance. It must be noted that the previous
designs presented under experimental research designs can also be used (i.e., within-subjects,
posttest-only designs) in quasi-experimental studies. The primary factor separating exper-
imental from quasi-experimental studies is how the participants are sampled and then
assigned to groups. In experimental studies probability sampling must be used, and then the
participants are randomly assigned to experimental or control groups. In quasi-experimental
studies, the researcher uses one of the non-probability sampling methods, and then makes

Sciences
one group the experimental group and the other the control group (meaning no random
assignment). It must be noted that all quasi-experimental studies have less external valid-
ity than studies using an experimental design. Chapter 5 explained external validity as the
extent to which the results of a study will be true for different groups of people or similar
people in different settings. Since quasi-experimental studies do not sample using a proba-
bility sampling method or randomly assign participants to groups, this greatly reduces the
statistical ability of researchers to make the statement that the results of the study would
apply back to a whole population.
Nonequivalent (Pretest and Posttest) Control Group Design

In this design, also called quasi-experimental pretest/posttest design, there are two non-
equivalent groups, meaning the participants in this study were not randomly assigned to
the experimental or control groups. Both groups are given a pretest and posttest, but only
group 1, the experimental group, receives an intervention.
G1 O X O
___ ____ ___ ___ ___ ___ ___ ____ ___ ___
G2 O O
SIDE BOX 7.2 The dotted lines indicate the groups were not randomly assigned. As stated earlier,
scientific notation varies across disciplines. Some disciplines label groups with the letters A for exper-
imental and B for the control group, or in the case of the Solomon four-group, A, B, C, D. Others write
the word Group or use the letters NR to show that there was not random assignment to the experimen-
tal and control groups. For the purpose of this text, we will use the letter G to indicate that the study is
quasi-experimental.
Example of a nonequivalent (pretest and posttest) control group design: Researchers

are interested in studying the effectiveness of a smoking cessation program on high
school students’ perceptions of smoking behaviors. Students in two 10th grade class-
rooms at a local high school were selected to participate in the study; the researchers
selected one classroom to be the experimental group (Group 1) and the other to be the
control group (Group 2). Students in both classrooms were given a pretest survey that
measured their attitudes and beliefs toward smoking (pretest-O), after which students
in one group participated in a two-hour smoking cessation program (X); the other group
did not participate in the smoking cessation program (control group). Students in both
classrooms were then given a posttest survey (posttest-O). Data analysis of the posttest
for both groups indicated a difference between the attitudes and beliefs of students who
participated in the smoking cessation program and those who did not.
Because the two groups that participated in this experiment were not equivalent to
one another (equivalency is obtained by randomization), it is possible that the difference
between the two groups could be due to a factor other than the intervention. As discussed in

Sciences
Chapter 5, there are threats to a study’s internal validity. The results of this particular study
could have been influenced by the internal validity threat of selection bias; e.g., there was
something different between the two 10th grade classes selected to be in this study. Another
possible threat to the study’s internal validity is history; e.g., an outside event that might
have occurred to one of the classes but not the other. Utilizing non-probability sampling
also reduces the external validity of the study.
Single Group Time-Series Design

In the single group time-series design, there is one group, and several observations of the
dependent variable (O) are taken over time, both prior to and after the intervention. This
research design is utilized when the researcher wants to determine what impact, if any,
an intervention had on a group over time. If the intervention had an impact, observations
after the intervention would be different from observations prior to the intervention. The
example research notation below includes five observations prior to the intervention and
five observations after the intervention.
Research Notation
Time
O O O O O X O O O O O
Baseline data
Observations prior to the intervention are notated as baseline data. Repeated measurements
over time enable researchers to observe whether there is a pattern of change over time in
the dependent variable being measured. Once there is a repeated measure of the dependent
variable prior to the intervention, post-intervention measurements allow the researcher to
assess what impact, if any, the intervention had on the variable.
This design can only be a quasi-experimental study since there is no control group.
Example of a single group interrupted time-series design: Researchers are interested in
studying the daily sales of food in a college cafeteria before and after the implementation of
an intervention. Prior to the intervention, the standard in that cafeteria was to post nutri-
tional food information for all the food served in one location in the cafeteria. Researchers
observed the food selections of consumers in the cafeteria once every 2 weeks for a period of
10 weeks. Then at week 10, the intervention was implemented. The original nutritional sign
was taken down. The intervention consisted of providing nutritional information in the form
of a brochure that consumers could pick up when they entered the cafeteria, as well as plac-
ards with nutritional information that were posted at eye level for each station. Once again,
researchers observed the food selections of consumers in the cafeteria once every 2 weeks
for a period of 10 weeks. Data analysis would then be performed in order to determine if
the dietary selections made by consumers after the intervention reflected an increase in the
amount of healthy foods selected as compared to foods selected prior to the intervention.
It is important to keep in mind the internal threats to validity when interpreting results.
As mentioned in the previous example, it is possible that the results of this particular study

Sciences
could have been influenced by history and maturation (threats to internal validity) by not
having a control group. Perhaps some consumers might have seen a dietitian during the
time this experiment took place or read a book on healthy eating, and therefore some of
the results observed may not have been entirely because of the change in food labeling in
the cafeteria.
Other single group time-series studies include interruption. For example, the interrupted
time-series design could involve studying the change in a variable that the researcher did
not manipulate. An example from Shadish, Cook, and Campbell’s (2002) book illustrates
this concept. A researcher could collect numerous observations (data) on traffic fatalities
in a particular state before and after a seat belt law was passed. It is called an interrupted
time-series design, since the data collection was interrupted by the change in the law (p. 172).
Control Group Time-Series Design

The only difference between this design and the previous design (single group time-series
design or interrupted time-series) is the inclusion of a control group, which in turn increases
the internal validity of the experiment. There are numerous variations on how time-series
studies are structured. Some are very similar to the previously presented study designs (e.g.,
pretest-posttest design or posttest-only design) where one group receives an intervention
or treatment and the control group does not receive the intervention or treatment. Another
type of interrupted time-series design staggers the intervention with the removal of the
intervention and a reintroduction of the intervention. The one thing all these variations of
the design have in common is multiple observations at posttesting (Cook & Campbell, 1979;
Morgan, Gliner, & Harmon, 2000a; Shadish, Cook, & Campbell, 2002).
This section will present one of the most common forms of the time-series designs, where
the control group (Group 2) has a parallel series of observations to Group 1. If change is seen
in the experimental group (Group 1) after the intervention and a corresponding change is
not seen in the control group (Group 2), the change is most likely due to the intervention.
Additionally, by collecting data at baseline and then at several times at the end of the study,
it is possible to determine whether the effects of the intervention lasted over time.
Research Notation
Time
Baseline data
G1 O O O O O X O O O O O
----------------------------------------------------------------------------------------------
G2 O O O O O O O O O O
Example of a control group time-series design: Researchers are interested in studying

the daily sales of food in a college cafeteria before and after the implementation of a new
marketing campaign to label the nutritional content of food. In this particular design,
there are two groups participating in the research study. Participants in this study were
not randomly selected and not randomly assigned to Group 1 or 2. Nutritional food
information is provided in the form of signs posted above the salad bar, soup station,

Sciences
etc., in the cafeteria. Researchers observed the food selections (O) of participants in
both groups once every 2 weeks for a period of 10 weeks (time component). At this
time, the intervention (original nutritional signs were taken down and nutritional
information was provided in the form of a brochure that consumers could pick up
when they entered the cafeteria, as well as placards with nutritional information that
were posted at eye level for each station) was implemented for Group 1 (experimental
group), and researchers observed the food selections of consumers in the cafeteria
once every 2 weeks for a period of 10 weeks (time component). Group 2 (control
group) did not receive the intervention, and their food selections were observed,
once again, every 2 weeks for a period of 10 weeks. At this point, data analysis would
be performed and reviewed to see if there was a difference in consumer selection of
food between the two groups.
This particular design controls for more threats to internal validity than the single
group time-series design discussed—such as history and maturation—because it has a
control group. However, it does not control for as many threats to internal validity as an
experimental study design would. Utilizing non-probability sampling also reduces the
external validity of the study.
As stated earlier in the chapter, a control group time-series study can also be a true
experiment. An example of a randomized control group time-series is the Fogel et al.
(2015) study. The inclusion and exclusion criteria used for this study was presented
in Chapter 5. The researchers want to test the efficacy of a new “evidence-based HIV
behavioral intervention [that] could reduce incidents of nonviral STIs, reduce sexual
risk behaviors, and increase risk reduction practices among newly incarcerated women
within the North Carolina Department of Corrections” (p. 802). Women were randomly
assigned to the behavioral intervention group or the control group. The new behavioral
intervention was delivered to the women while they were incarcerated. The control
group received the standard HIV/STI education taught by the prison’s nursing staff.
The researchers collected posttest data from the women at 3 and 6 months post release
from incarceration. Collecting data at two-time intervals allowed the researchers to
measure how long the effects of the intervention lasted.
METHODS
How Many Participants Are Needed for Experimental

and Quasi-Experimental Research Designs?
A detailed description of this topic can be found in Chapter 5 under the heading Quanti-
tative Sample Size. It must be noted that sample size estimations and power analysis are
vital when planning experimental research, especially clinical drug trials. The sample has a
direct impact on the quality of the results of the study. A summary of the important points
regarding this topic are: inclusion and exclusion criteria, especially in clinical drug trials,
as they directly impact the study’s internal and external validity; the size of the sample
is determined by the size of the population, the confidence and interval levels set by the

Sciences
researcher; and the power analysis, which takes into consideration the alpha level, power
level, and effect sizes.
What Types of Data Collection Are Used for Experimental

The data collection tools/instruments are dependent on the nature of the study. Review the
very detailed description of this topic provided in Chapter 5 in the section titled Quantita-
tive Data Collection.
What Types of Data Analysis Methods Are Used for Experimental

Experimental researchers use a combination of descriptive and inferential statistics. The
specific statistical test used will depend on the type of data collected, the purpose and
design of the study, and how the research questions are written. In general, the researcher
will use descriptive statistics to describe the sample. For example, in a research article,
often one will see a chart that presents demographic information on the participants.
This provides the reader with an opportunity to see, for instance, the percentages of
individuals who participated in the study based on gender, age, or the mean scores for a
particular data set.
The use of inferential statistics allows the researcher to draw inferences (conclusions
and correlations) by the analysis of parametric data (interval and ratio). In experimental
research, the analysis determines whether the null hypothesis can be rejected.
The main difference between experimental and quasi-experimental research studies is
the inferences that can be stated at the conclusion of the analysis. True experiments can
determine cause-and-effect relationships between variables, and the results of the study can
be generalized back to the population. Quasi-experimental studies can also draw inferen-
tial conclusions about the study. However, those conclusions are limited. For example, the
results of a quasi-experimental study cannot determine causality or generalize the results
back to the whole population, since non-probability sampling/nonrandom assignment to
groups were used.

This chapter presented a summary of the characteristics of experimental and
quasi-experimental designs, as well as a discussion of the use of research notations in
health science research. Examples of experimental (pretest-posttest control group, posttest-
only control group, Solomon four-group, and within-subjects) and quasi-experimental
(nonequivalent [pretest and posttest] control group, simple time-series, and control group
time-series) were presented with an emphasis on a description of the design, the research
notation used, an example of research using the design, and threats to validity (internal and
external). The information presented was not designed to be an extensive discussion of the
topic, but rather a concise description of the subject matter with specific examples provided.
Table 7.4 provides a summary of the purpose, special features, and methods specific to each
design covered in this chapter.

Sciences
TABLE 7.4. Experimental/Quasi-Experimental Designs: Purpose, Special Feature, and Methods
Special Feature of the Design (SF)

Methods: sampling (S), data collection (DC), & data analysis
(DA) specific to the design
Design Purpose Experimental Quasi-experimental
Pretest-posttest con- Test the efficacy (S)Probability methods (S)Non-probability methods

trol group of a treatment or (DC) Tools/instruments (DC) Tools/instruments
intervention (DA)Inferential statistics (DA)Inferential statistics
R1 O X O G1 O X O
R2 O O G2 O O
Posttest only control Test the efficacy (S) Probability methods (S)Non-probability methods
group of a treatment or (DC) Tools/instruments (DC) Tools/instruments
intervention (DA)Inferential statistics (DA)Inferential statistics
R1 X O G1 X O
R2 O G2 O
Solomon Four- Group Test the efficacy of a (S) Probability methods

treatment or interven- (DC) Tools/instruments
tion while controlling for (DA)Inferential statistics
pretesting effects
R1 O X O n/a
R2 O O
R3 X O
R4 O
Within-Subjects Compare the efficacy (S) Probability methods (S)Non-probability methods

between two treatments (DC) Tools/instruments (DC) Tools/instruments
or interventions (DA)Inferential statistics (DA)Inferential statistics
R1 X1 O G1 X1 O
X2 O X2 O
Single Group Study impact of an Time

Time-Series intervention on a single
group over time n/a OOOxOOO
Control Group Study impact of an inter- (S)Probability methods (S)Non-probability methods

Time-Series vention over time (DC) Tools/instruments are used to (DC) Tools/instruments are
collect data over time used to collect data over time
(DA)Inferential statistics (DA)Inferential statistics
Time Time
R1 O O O X O O O G1 O O O X O O O
R2 O O O O O O G2 O O O O O O

Sciences
STUDY PRACTICE
At this point, students should be familiar with the basic characteristics of experimental and
quasi-experimental designs as well as the symbols used in research notations. At this time,
given a research notation, students should be able to decipher the meaning of each notation
as well as the design of the study. Given a case scenario, students should also be able to
determine the research design being used. Students should also know the characteristics
of each of the designs presented in this chapter as well as the strengths and weaknesses of
each design.
ɠɠ Internal and external validity ɠɠ Research notations
ɠɠ Confounding variable ɝɝ O, X, and R
ɠɠ Error ɝɝ Time
ɠɠ Bias ɝɝ Subscripts
ɠɠ Experimental designs ɠɠ Quasi-experimental
ɝɝ Manipulation of variables ɠɠ Nonequivalent
ɝɝ Control
ɝɝ Random sample
ɝɝ Random assignment
SIDE BOX 7.3 STUDY TIP For each sample test question, write out the research notation for each of
the possible answers BEFORE selecting the correct answer.
1. The design below is an example of:
R1 O1 O2 X O1 O2
R2 O1 O2 O1 O2
a. Experimental pretest-posttest control group design

b. Quasi-experimental multiple baseline design
c. Experimental posttest-only control group design
d. Time-series design
2. Dr. Manneto sits on the board of a Service Dog Foundation program. She wants
to know if the level of consumer satisfaction with their dog would increase if the
training program was customized for the needs of the consumer.
Fifty consumers who had received their service dogs within the past month were ran-
domly selected to participate in the study. The consumers were randomly assigned to
one of two groups. One group received dogs who had completed the regular training
program; the other group received dogs who had completed the customized training

Sciences
program. Consumers were given a standardized survey to complete regarding their level
of satisfaction with how their dog was able to respond to their needs.
This study employed a(n) ______ design.
a. Solomon four-design
b. quasi-experimental posttest only control group design
c. experimental posttest only control group design
d. time-series design posttest only design
3. A researcher wants to determine which drug treatment is more effective for
treating high blood pressure. This researcher seeks to prove that the new drug
is more effective at reducing high blood pressure than the drug that is currently
being used.
This study should use which of the following research designs?
a. Cohort
b. Longitudinal
c. Correlational
d. Experimental
4. A researcher wants to examine the impact of a new course in gerontological nursing
on the perceptions and career intentions of nurse practitioners working with older
adults as compared to the current course. Seventy first-year nursing students were
randomly selected to participate in the study. Thirty-five students were randomly
assigned to the group that took the new course, and the remaining 35 students
took the current course. At the completion of the course, each group was given a
standardized test to assess their perceptions and career intentions with regard to
working with older adults.
This is an example of which of the following research designs?
a. Nonequivalent (pretest and posttest) control group
b. Experimental pretest/posttest control group
c. Quasi-experimental posttest only control group
d. Experimental posttest only control group
5. Which of the following best describes a problem associated with the
pretest-posttest design?
a. Possible effects of the pretest on posttest performance
b. The validity of the independent variable
c. The validity of the dependent variable
d. Possible carryover effects of the intervention on posttest performance
6. Which of the following statements describes a within-subjects design?
a. The same group of participants are given more than one treatment
b. Multiple groups of participants are given the same treatment
c. The performance of multiple groups is measured multiple times
d. The performance of multiple groups is measured once

Sciences
7. In the Solomon four-group design, how would a researcher determine if pre-
testing influenced the results of the study?
a. Compare the posttest results of R1 and R3
b. Compare the pretest results of R1 and R 2
c. Compare the pretest results of R1 and posttest results of R 4
d. Compare the pretest results of R 2 and pretest results of R 4
8. In a research notation, X refers to the _________ and O refers to the ___________.
a. independent variable; dependent variable
b. dependent variable; independent variable
c. pre-intervention results; post-intervention results
d. post-intervention results; pre-intervention results
9. What is the difference between an experimental and quasi-experimental
research design?
a. An experimental design uses a control group; a quasi-experimental design does not
b. An experimental design has no limits on the number of individuals participating
in the study; a quasi-experimental design has limits on the number of individuals
participating in the study
c. An experimental design randomly assigns individuals to an experimental or
control group; a quasi-experimental design does not
d. An experimental design can have multiple observations; a quasi-experimental
design does not
10. The purpose of this study is to compare face-to-face nutrition lectures woven into
guided grocery store tours to lecture-only nutrition classes. A physician in one of
the large clinics within a 16-hospital health system uses her patients with very high
cholesterol for this study. She has the group of patients go through the traditional
lecture-only nutrition class and then tests cholesterol 3 months later. She waits a
month and then has the group go through the face-to-face nutrition lectures woven
into guided tours in grocery stores and then tests cholesterol 3 months later.
What type of design is used in this study?
a. Longitudinal developmental
b. Within-subjects
c. Simple time-series
d. Correlational
STUDY ACTIVITIES
Take all of the information you have learned thus far and put it together as part of the
big picture.
1. Write the meaning of each research notation (O, X, R) discussed in this chapter
2. List the research designs discussed in this lecture, each on the front of a separate notecard

Sciences
3. For each research design, list the following information on the back of the notecard
a. Research notation
b. Characteristics of each design
c. Sampling method used for each design
Based on the information you now know about experimental and quasi-experimental
designs, you can, as you have in previous chapters, revise your notes/flash cards with this
new information.
V: add this information to your diagrams and flow charts.

A: make sure you read out loud to yourself when studying and writing flash cards.
R: turn the charts in the chapter into paragraphs.
K: take your revised flash cards and on the floor or a large table, make outlines with the
flash cards … see how things are related to one another.
Using the information you have learned from this chapter, read the scenarios below and
answer the questions.
1. What design would be best for this study?

The purpose of this study is to evaluate the efficacy of an intervention to reduce sexual
risk behaviors, sexually transmitted diseases (STDs), and unplanned pregnancy in female
teens. The researcher obtained a list of all patients who received services within the past
year from local health clinics. The researchers identify the sexually active females (ages
14–18) receiving services from the list and then randomly select 600. The young women
were randomly assigned to either a support group that emphasized sexual health: HIV
knowledge, communication, condom use skills, and healthy relationships, or to a group
that discussed exercise and nutrition. All young women completed a questionnaire,
demonstrated condom application skills, and provided specimens for HIV and STD test-
ing before and after the intervention, then again at a 6-month and 12-month follow-up.
2. What design would be best for this study?
A group of physical therapists working in a home care agency wanted to determine
whether a home-based walking exercise program that included a cognitive behavioral
intervention would improve the patients’ pain tolerance and functional walking perfor-
mance (confidence level 95%, alpha level .05). The research team recruited 100 elderly
patients (aged 75 or older) from their caseloads and assigned patients either to the new
home-based cognitive behavioral walking intervention or the standard physical therapy
treatment offered by the home care agency. Data were collected after the completion
of the program using the Walking Impairment Questionnaire (WIQ), Pain Composite
Scores from the 12-item short-form, and a physical therapy assessment of walking perfor-
mance. The WIQ and 12-item short-form were selected due to the instruments’ validity
and reliability scores. The researchers developed the rating tool to assess walking per-
formance, and prior to using it in the study, the research team rated 20 similar patients
using the assessment tool. The researchers found that scores were equivalent across the

Sciences
20 pilot patients. The results of data analysis showed a statistically significant increase
in walking performance (p < .01) and pain tolerance (p < .05) in the intervention group.
Scenario adapted from Rejeski et al. (2014).
3. What design was most likely used in this study?
Gout is difficult to accurately diagnose; traditionally this disorder has been diagnosed
by physician review of clinical signs (testing of joint fluid) and symptoms. It is unknown
whether diagnostic imaging tests are reliable diagnostic tools for gout. The purpose of
the pilot study is to compare the accuracy of different diagnostic imaging tests (dual
emission computerized tomography and plain X-rays) in 50 patients, 18 or older, who
have a confirmed diagnosis of gout currently being treated at Stony Brook University
Hospital. The 50 patients will first have an X-ray and then the dual emission comput-
erized tomography test.
REFERENCES
Bowen, D. J., Kreuter, M., Spring, B., Cofta-Woerpel, L., Linnan, L., Weiner, D., … Fernandez, M.
(2009). How we design feasibility studies. American Journal of Preventive Medicine, 36(5), 452–457.
doi:10.1016/j.amepre.2009.02.002
Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Boston,
MA: Houghton Mifflin Company.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation, design & analysis issues for field settings.
Boston, MA: Houghton Mifflin Company.
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and
evaluating complex interventions: The new Medical Research Council guidance. BMJ, 337, e1–e6.
Fisher, J. O., Rolls, B. J., & Birch, L. L. (2003). Children’s bite size and intake of an entree are greater
with large portions than with age appropriate or self-selected portions. American Journal of Clinical
Nutrition, 77(5), 1164–1170.
Fogel, C. I., Crandell, J. L., Neevel, A. M., Parker, S. D., Carry, M., White, B. L., … Gelaude, D. J. (2015).
Efficacy of an adapted HIV and sexually transmitted infection prevention intervention for incar-
cerated women: A randomized controlled trial. American Journal of Public Health, 105(4), 802–809.
doi:10.2105/AJPH.2014.302105
Harris, A. D., McGregor, J. C, Perencevich, E. N., Furuno, J. P., Zhu, J., Peterson, D. E., & Finkelstein, J.
(2006). The use and interpretation of quasi-experimental studies in medical informatics. Journal
of American Medical Informatics Association, 13(1), 16–23.
McGahee, T. W., & Tingen, M. S. (2009). The use of the Solomon four-group design in nursing
research. Southern Online Journal of Nursing Research, 9(1), e1–e7.
Morgan, G. A., Gliner, J. A., & Harmon, R. J. (2000a). Quasi-experimental designs. Journal of the
American Academy of Child and Adolescent Psychiatry, 39(6), 794–796.
Morgan, G. A., Gliner, J. A., & Harmon, R. J. (2000b). Randomized experimental designs. Journal of
the American Academy of Child and Adolescent Psychiatry, 39(8), 1062–1063.
National Institutes of Health. (n.d.). NIH’s definition of a clinical trial. Retrieved from https://grants.
nih.gov/policy/clinical-trials/definition.htm
Phan, H. P., & Ngu, B. H. (2017). Undertaking experiments in social sciences: Sequential, multiple
time series designs for consideration. Educational Psychology Review, 29(4), 847–867. doi:10.1007/
s10648-016-9368-0
Rejeski, W., Spring, B., Domanchuk, K., Tao, H., Tian, L., Zhao, L., & McDermott, M. M. (2014). A
group-mediated, home-based physical activity intervention for patients with peripheral artery

Sciences
disease: Effects on social and psychological function. Journal of Translational Medicine, 12(29),
e1–e8. doi:10.1186/1479-5876-12-29
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs
for generalized causal inference. Belmont, MA: Wadsworth Cengage Learning.
Solomon, R. L. (1949). An extension of control group design. Psychological Bulletin, 46(2), 137–150.
Wijesinghe, H., Galappatthy, P., deSilva, R., Seneviratne, S. L., Saravanamuttu, U., Udagama, P.,
Hart, M., & Wijayaratne, L. S. (2017). Leflunomide is equally efficacious and safe compared to
low dose rituximab in refractory rheumatoid arthritis given in combination with methotrexate:
Results from a randomized double blind controlled clinical trial. BMC Musculoskeletal Disorders,
18(1), 310–320. doi:10.1186/s12891-017-1673-3

Sciences
UNDERSTANDING
QUANTITATIVE RESEARCH:
NON-EXPERIMENTAL DESIGNS
CHAPTER GOALS:
8
INTRODUCTION • To introduce the defining features
that correlate with each non-experi-
This chapter discusses a number of research designs that fall under the
mental research design discussed.
category of non-experimental and explains the purpose for using each one.
• To provide students with sufficient
The classification of non-experimental designs often varies depending
information to connect sampling,
on the field of research. This book will focus on the non-experimental
data collection, and data analysis
designs most common in health science research: developmental, which
methods to each design.
incorporates longitudinal and cross-sectional; observational, cohort,
case-control, correlational, and survey. Each non-experimental design • To provide students with working
will be discussed with examples and will provide an explanation for why definitions of the terms correlational
a researcher would use the design during the planning stage (Table 8.1). coefficient, odds ratio, line of regression,
The second part of the chapter will present new information on meth- prospective, and retrospective.
ods (sampling, data collection, and data analysis) as they are utilized in • To furnish students with the tools
non-experimental research. In addition, it will discuss validity and threats necessary to identify biases in non-
to validity as they pertain to this type of research. Survey research, also experimental research and recognize
a non-experimental research design, will be covered in Chapter 9. ways researchers control for them.
• To present students with an under-
standing of the strengths and
NON-EXPERIMENTAL RESEARCH—DEFINED weaknesses inherent in each design.
Lobmeier (2012) describes non-experimental designs as “research designs
in which an experimenter simply either describes a group or examines
relationships between existing groups” or variables (p. 2). The term LEARNING OBJECTIVES:
non-experimental research design encompasses a variety of descriptive
research designs that all look to describe the problem through observing THE STUDENT WILL BE ABLE TO:
changes to variables that occur without incorporating any manipula- • Define at least seven terms related to
tion on the part of the researcher. Leedy and Ormrod (2016) describe non-experimental research design;
descriptive research as a category of research design that “involves either
• Recognize the characteristics of
identifying the characteristics of an observed phenomenon or exploring
non-experimental research designs;
possible associations among two or more phenomena” (p. 136). Whether
• Compare appropriate research
the research is looking to describe the problem through changes that
methods utilized for each non-
occur over time or an association between variables, the most import-
experimental research design;
ant feature to remember about non-experimental research is that the
• Demonstrate the ability to choose
researcher does not control or manipulate any variables; rather, the focus
the most appropriate design for
of the design is to simply describe what develops naturally. This common
sample research ideas; and
denominator of observing without manipulating variables is what sets
• Identify properties of biases inherent
in non-experimental design and ways
to control for each.
161

Sciences
Stages Action Steps
Design (Planning) Selecting the best research design and research methods (sampling, data collection, data anal-
ysis) that align with research question(s), submitting the research study proposal for IRB review
non-experimental designs apart from experimental designs, where the researcher manip-
ulates a variable in order to determine a causal or cause-and-effect relationship.
It is important to remember that while some descriptive research can determine that
an association exists between two or more variables; it cannot go so far as to determine
that a causal relationship exists between variables as explanatory research does. This type
of non-experimental research is called correlational. In other words, the researcher cannot
state that A caused B to occur.
Consider this example: When conducting non-experimental research, a researcher cannot con-
clude that exercising regularly caused lower obesity rates in young adults, but the researcher can
state that an association exists between regular exercise and lower obesity rates in young adults.
If a researcher wanted to state a causal effect between regular exercise and lower obesity rates,
an experimental design (as discussed in Chapter 7), could be employed to study whether regular
exercise causes obesity rates to decrease.
Why Choose Non-Experimental Research Design?

While non-experimental research cannot establish a cause-and-effect relationship between
variables, there are many instances where this type of design may be preferred.
• When it may be unethical or not feasible to conduct an experimental design.

• When the researcher is looking to study a rare outcome.
• When the research involves only one sample.
• When the researcher is looking to explore a topic before deciding whether or not
to examine it further with an experimental design.
• When the research is looking to establish past relationships or associations
between variables.
NON-EXPERIMENTAL RESEARCH DESIGNS

Non-experimental designs differ from experimental designs in three major ways:
1. Non-experimental designs typically involve one sample group.

• While some non-experimental designs like case control utilize two groups, most
will only employ one group.

Sciences
Chapter 8 Understanding Quantitative Research: Non-Experimental Designs | 163
2. Sampling methodologies in non-experimental design can be either probability or

non-probability.
• With the exception of two non-probability sampling methods: purposive and the-
oretical sampling methods.
3. Non-experimental designs do not manipulate variables. Rather than looking to estab-
lish a cause-and-effect relationship as experimental designs do, non-experimental
looks to discover relationships between variables.
• This means that non-experimental research can discover a relationship between vari-
ables, but it does not definitely prove that one variable causes a change in another.
Developmental Designs
Developmental research designs study characteristics or variables as they develop or change
over time. Most frequently associated with child psychology and educational research,
developmental research is becoming more commonplace in health care research now than
ever before where it is often used to study human development across a time span. Devel-
opmental research can be conducted as either a cross-sectional or a longitudinal study. It
must be noted that cross-sectional and longitudinal approaches to research can be used
with many other research designs; for example, a cross-sectional survey is a survey that is
sent once at a specific day and time, and whoever is present takes the survey (Hall, 2008).
Here the discussion will focus exclusively on using the cross-sectional and longitudinal
approaches with the developmental research design.
Developmental longitudinal design follows a group (sometimes called a cohort) over a
period of time. Dictionary.com defines a cohort as “a group of persons sharing a particular
statistical or demographic characteristic” (dictionary.com). In a developmental longitudinal
study, the focus is to record development over time; for example, a study can look to follow
the survival rates of children born in 1980. The group of children born in 1980 is considered
a cohort, as they share the characteristic of the year in which they were born. When the
term cohort is seen later in the chapter, it will refer to a research design.
Developmental longitudinal studies can take place over a few months, or they can last
for decades. A typical example of this type of longitudinal study can be found in the field
of psychology and is called the Genetic Studies of Genius, led by Professor Lewis Terman
from Stanford University. Findings from this study have been published continually since
1925 and continue to be published, even though Terman died more than 40 years ago and
the original subjects are well into their nineties (Gross, 2006). The process of following
one group over a period of time allows the researchers to set up numerous data collection
points throughout the life of the research. In addition, the researchers have the opportunity
to add variables as the study develops.
Now consider this example in the field of health sciences. Hammerlid, Silander, Hörnestam,
and Sullivan (2001) conducted a developmental longitudinal study where the developmental
issue they explored was quality of life during and after receiving treatment for head and
neck cancer. The research team followed participants, who had a diagnosis of head and neck
cancer, for several years and collected data using the Health-Related Quality of Life (HRQL)
questionnaire. The team found that quality of life was at its lowest points during treatment
but for most participants slowly returned to pretreatment value by 3 years post-treatment.
By using a longitudinal design, the researchers were able to assess the HRQL of participants

Sciences
at numerous points throughout the diagnosis and treatment process to watch their devel-
opment over time.
Unlike longitudinal design, which follows one group over a period of time, developmental
cross-sectional design looks at development over time by taking a snapshot of the stages
of development from different groups all at the same time. It can be considered a fast and
inexpensive alternative to a longitudinal study, as there is only one data collection point
for each respondent and the study does not look to follow participants over an extended
period of time. Instead, development is shown through the differences that exist between
groups, not from one group moving ahead through time. For example, if a researcher wanted
to study the eating habits of elementary school–aged children when their parents are not
present as they mature from first to fifth grade, a sample of students from each grade (1–5)
can be looked at to attempt to simulate the maturation process. In this case, the researcher
would collect data on one day from all the students sampled from each grade level (e.g.,
collect data by surveying them on their eating habits at school or observing them during
lunch period) and analyze the data between groups to see if their eating habits change over
time as the “students mature.”
One thing to keep in mind is that cross-sectional design does not utilize a control or
comparison group. While it samples different groups of people, when collecting data, the
groups are designed to represent one population that simulates growing older or time
passing. They are used to show snapshots of development, which, when analyzed together,
explain the development of change over time.
Cross-sectional design is useful in generating hypotheses and aiding in health care
planning. The design is often used when researchers are conducting a pilot study when
looking to determine if developmental changes exist in the snapshot groups before com-
mitting the time and money to a developmental longitudinal study. This draws back to the
relative low cost of employing this design and the short amount of time needed to execute
a cross-sectional research study.
What Are Some Issues with Longitudinal and Cross-Sectional Research Design?
Both types of developmental research designs have inherent pluses and minuses. While
developmental longitudinal studies allow researchers to look at numerous variables
over the course of the study, they are often expensive and require a team of researchers
to complete. Another issue is attrition (loss to follow-up), where participants leave the
study for various reasons before it is completed. In the Hammerlid et al. (2001) study on
HRQL during and after treatment for head and neck cancer discussed earlier, they note
that by the end of the study, 66% of the participants were alive and of those only 88%
completed the study.
In developmental cross-sectional studies, researchers can collect data quickly and inex-
pensively, but confounding variables can influence the outcome of the study (e.g., low
internal validity). For example, using the previous example of elementary students’ eating
habits at school, schools sometimes employ many lunch aides to cover all the lunch periods.
If grade 4 students eat lunch with a lunch aide who is extremely strict and grade 3 students
eat lunch with a lunch aide who is extremely lenient when it comes to enforcing lunchroom
rules (such as staying in your seat, finishing everything on your plate, or not throwing food),
it may appear that there is a high level of maturation in student eating habits between the

Sciences
third and fourth grade, when in reality the stark difference between third and fourth grade
lunchroom behavior may be correlated with the confounding variable of the lunch aide on
duty with each grade, in addition to or in place of their increased maturity.
Observational Design
In observational research, the researcher observes and records data without manipulating or
intervening in any way during the research process. It is non-experimental in that it observes
phenomena with no manipulation of variables and no intervention by the researcher. The
purpose of observational research is to record naturally occurring behavior (in a relatively
inexpensive way) to better understand what behavior is occurring. For example, a researcher
looking to understand if children will choose to wash their hands before eating lunch with-
out being directed to do so may use trained observers to rate instances of handwashing in
elementary school–age children during lunch.
Collecting data through observation is a method often associated with qualitative research
designs like ethnography, where a researcher observes behavior in a natural setting and
collects data by watching and observing norms and traditions associated with that cul-
ture. As is the case in other qualitative research designs, the themes and patterns that the
researcher is recording emerge from the data. Observational quantitative research, on the
other hand, is different on a number of levels. Chapter 1 presented an overview of paradigms
or the worldview that the researcher follows when looking to answer a question and how
this worldview is linked to the research methodology employed when conducting a study.
Qualitative observational research adheres closely to a constructivism paradigm where truth
is more fluid and rooted in social constructs. Quantitative observational research, on the
other hand, adheres more to either a positivism or post-positivism paradigm.
In quantitative observational research, there is a pre-specified focus to the research that
closely adheres to an underlying theoretical framework that has been determined through
the review of the literature. This pre-specified focus includes:
• An operational definition of what is being observed.

■■ This is important as it helps in the creation of a data collection tool
(rating scale). It provides strict guidelines as to what should be observed,
counted, and evaluated during the study.
• Data collection being divided into short, predetermined segments of time.
■■ This is important to ensure consistency of the data collection process.
• Codes that have been predetermined based on theory, meaning the codes are
developed based on established theory. Established theory is what is already
known about the relationship between the variables.
■■ Determines what variables are actually being observed and recorded.
ɝɝ The rating scale is created before the study begins.
■■ The evaluation of these predetermined observations is what is used in the
data analysis.
■■ A team of highly trained individuals independently rate (collect data on)
the participants. Inter-rater reliability, consistency between the raters, is
very important in this type of study.

Sciences
In summary, there is an important difference between the definition of the word code in
qualitative and quantitative observational research.
• In qualitative research, codes are determined after the data collection occurs.
Here the word code is defined as the interpretation of a meaning unit and is asso-
ciated with the data analysis processes.
• In quantitative research, codes are how the researcher quantifies the observa-
tion (collects data). The individuals who collect data, often called raters, are
made aware of what type of data they will be collecting before the study even
begins. The raters collect data using a rating scale that is made up of these
predetermined codes.
Cruz, Marques, Barbosa, Figueiredo, and Sousa (2013) conducted an observational pilot
study at a nursing home in Portugal when assessing a “multisensory and motor-based
group activity program for residents with dementia and … its impact on residents’ behav-
ior” (p. 137). By videotaping the activities of four residents, they were able to observe and
record participation in the program and assess its value using trained raters to analyze the
video recordings and rate participants’ engagement using predetermined scales. According
to Martin et al., “the observation of residents’ behavior is the best way to recognize when
a meaningful connection is occurring and, therefore, the only viable method to assess the
well-being of residents with advanced dementia” (as cited in Cruz et al., 2013, p. 139).
Cohort Design
SIDE BOX 8.1 Developmental Longitudinal Versus Prospective Cohort

Similar but very different concepts:
A developmental longitudinal study follows a group over time looking for developmental change
over time, while a prospective cohort study follows the group over time to study the development
of an outcome.
Cohort research designs can be used to determine if an exposure is linked to the progres-
sion of a disease or condition. It is important to note that a cohort study is used when the
relationship between the exposure and the outcome has not yet been determined. These
studies can be either prospective—looking forward—or retrospective—looking back into
existing data. As previously stated, a cohort is a group with a shared statistical or demo-
graphic characteristic. A cohort study can follow one cohort to see the effects an exposure
will have or has had on the development of an outcome (disease or condition). This would
then be compared to the rate of that outcome (disease/condition) in the general popula-
tion. It can also look to compare two cohorts, with one group having the exposure (being
exposed to something) and the other not. Whether the study employs one cohort or two
(see Figures 8.1 and 8.2), a defining feature of a cohort study is that no one in the observed
group or groups has developed the outcome of interest at the beginning of the study. In
other words, cohort studies start with an exposure and then either look back at records or

Sciences
FIGURE 8.1 An illustration of a cohort study compar- FIGURE 8.2 An illustration of a

ing two cohorts. cohort design with one cohort.
Outcome
Sample of interest
Sample
No
No outcome
Exposure
exposure
Outcome No Outcome No
of interest outcome of interest outcome
look forward (follow people for a while) to answer the question: How is this exposure related
with an outcome (disease or a condition)?
Cohort studies are often used when a randomized control trial (see Chapter 7) would
be unethical to put forth. For example, a researcher cannot deliberately expose people to
asbestos in order to record whether or not it can be associated with the development of
cancer (Mann, 2003, p. 54). Another example of when a cohort study would be an unethical
choice for a researcher to employ may be when looking to determine whether or not child-
hood exposure to secondhand smoke can be associated with the development of cancer in
adulthood. It would be unethical for a researcher to knowingly expose children to second-
hand smoke (since the health risks associated with secondhand smoke have already been
established) in order to follow them through to adulthood to look for the development of
cancer. In this instance, a researcher could only ethically follow a cohort who was exposed
to secondhand smoke as children to look for the development of cancer as adults.
Cohorts—groups of people who share a common feature, e.g., exposure to secondhand
smoke or asbestos—are chosen to be part of the research study based on that exposure
status. In these instances, the use of cohorts “permits calculation of the effect of each vari-
able on the probability of developing the outcome of interest” (Mann, 2003, p. 55). When
conducting a cohort study, the first important task is putting together the sample. The
researcher selects participants to be in the sample based on two factors: how representa-
tive of the general population the participant will be and the participant’s exposure status.
This is done so the researcher has the potential to study the development of the outcome of
interest (Mann, 2003, p. 55). Selecting participants based on exposure status and grouping
them by exposure/non-exposure means that cohort studies can be used to determine an
outcome, but because they do not include the elements of an experimental research study
(manipulate variables and probability sampling coupled with random assignment to exper-
imental and control groups), the results of this type of study cannot be used to determine
a cause-and-effect relationship.
Cohort studies are carried out in one of two ways: retrospective or prospective.
• Retrospective – The researcher looks back in time, either through medical

records or subject recall of self-reported data (asking the participants to remem-
ber past behaviors or events) about exposures that could possibly be linked to an

Sciences
outcome or a disease. At the time the study begins, outcome could or could not
have already occurred. While retrospective cohort studies can analyze data col-
lected over many years, the actual time needed to complete the study is only as
long as it takes the researcher to review the previously collected data. Therefore,
retrospective cohort studies can be relatively quick and inexpensive to conduct,
but since the data collection often involves the use of data that was originally
compiled or collected for another purpose (looking back at medical records) the
researcher understands when creating the study that the data they have access
to may not align perfectly with the factors being studied. Additionally, if the
researcher uses surveys that ask participants to remember past events or behav-
iors, the participants might not remember the events accurately. This is called
recall bias and will be covered in more detail later in the chapter.
• Prospective – The researcher knows exposure at the beginning of the study and
collects data going forward looking for development of the disease or outcome.
Prospective studies may take longer to complete, but the researcher has more
control over what is being observed. Prospective cohort studies can follow
■■ one group (a single cohort) who has had exposure to a variety of variables
and observe for the development of an outcome; or
■■ two groups (two cohorts), one who has had exposure to a variety of
variables and the other who has not and observe both groups for the
development of an outcome.
Many famous prospective cohort studies have been conducted and are still ongoing
in the United States and around the world. Some examples are the Framingham Heart
Study, the National Child Development Study, the British birth cohort study, the Dunedin
Longitudinal Study in New Zealand, the Birth to Twenty Study in Africa, and the Nurses’
Health Study in Boston, Massachusetts. All track cohorts over decades to look for natural
development of numerous diseases and outcomes and their associations with various risk
factors. Let’s take a look at one famous non-experimental cohort research study that has
been ongoing for decades, generating data that has influenced the way we approach health
care in myriad ways. The Nurses’ Health Study, a longitudinal cohort study, was established
in 1976 with 121,700 female nurses between the ages of 30–55 from various hospitals and
medical communities throughout the Boston area to assess risk factors associated with
cancer and cardiovascular disease. Today the study is in its third generation cohort and
has played an important role in determining many public health initiatives throughout the
decades (www.nurseshealthstudy.org).
Data from these prospective cohort studies provides a wealth of information that is
available to many researchers who wish to study any one of these risk factors (exposure)
in depth and determine associated risk factors for many various cancers and diseases,
including melanoma. For example, Han, Colditz, and Hunter (2006) used previously
collected data to study the relationship between sun exposure and the development of
melanoma by reviewing the self-reported survey data collected throughout the course of
the Nurses’ Health Study. Utilizing previously collected data can be an inexpensive and
fast way for someone to conduct research without having to sample from a population
all over again.

Sciences
When looking for a relationship between sun exposure and the development of mela-
noma, recruiting subjects from the general population, grouping them into two groups,
and forcing one group to spend many unprotected hours in the sun to see if they develop
melanoma would be unethical to say the least. In addition, it may take years for subjects to
develop melanoma. By using a non-experimental cohort design, researchers can examine
behavior that has already occurred naturally (exposure) and, as with this example, look for
an association between length of time and type of exposure with development of melanoma
without violating the ethical code of research concerned with ensuring that the benefits of
a study outweigh the risks. This concept, beneficence, was presented in Chapter 2.
In summary, it is important to understand the differences between developmental (lon-
gitudinal) studies and retrospective cohort studies. In developmental (longitudinal) studies,
one group of participants is being followed over time to study a developmental issue (e.g.,
quality of life). Conversely, a prospective cohort study is designed to study whether exposure
to a risk factor(s) yields the development of an outcome (e.g., disease) based on exposure
or non-exposure status, often using exposure and non-exposure groups for comparison.
So, while cohort studies are often longitudinal in that they follow a group that has shared
characteristics over a period of time, not all longitudinal studies are cohort studies.
Issues with Cohort Studies

There are issues that are inherent in the design and execution of a cohort study. Some that
may affect the outcome or validity of the study are:
• Attrition (loss to follow-up) – In a prospective cohort study that possibly spans a

significant amount of time (months, years, decades), there is always the risk that
subjects will lose interest, move away, or possibly die during the course of the
data collection process. This is called loss to follow-up and can have a significant
effect on the outcome of the study and the calculated incidence of the findings.
• Randomization – The absence of randomization can affect the outcome in a
cohort study by contributing to an imbalance in participant characteristics.
• Recall bias – Retrospective cohort studies often ask subjects to recall incidence
of exposure to risk factors. As is the case in any data collection where subjects
are asked to recall something, many times subjects may have selective recall of
exposures, placing more emphasis on something they consider to be important
and failing to mention something they consider unimportant. Participants may
also minimize, exaggerate, or completely leave out information depending on the
accuracy of their ability to remember certain incidents.
• Unrelated data – While retrospective cohort studies are often cheaper to conduct
because the data has already been collected, a researcher runs the risk of having to
utilize data that was collected for different purposes and therefore may be unre-
lated to the current study or fall short of addressing all the variables of interest.
• Confounding and extraneous variables – In a non-experimental study, researchers
do not have an opportunity to control the environment of the study. There-
fore, variables that are not being studied, e.g., extraneous variables, or variables
that are related to the independent variable and therefore affect the dependent

Sciences
variable (confounding variables, for instance) may have an effect on outcome.

As a result, it is sometimes difficult for a researcher who is conducting a cohort
study to determine if an outcome occurs as a result of the exposure being fol-
lowed or something unrelated to the study.
A summary of key strengths and limitations of cohort studies is found in Table 8.2.
TABLE 8.2 Key Points of a Cohort Study Design
Key Points of a Cohort Study Design
ɓɓ Best for determining incidence or natural history of an outcome or disease.
ɓɓ Can be used when a randomized control trial would be unethical or impossible to carry out.
ɓɓ Does not determine cause and effect.
ɓɓ Retrospective cohort studies are cheaper but are subject to recall bias and incomplete data.
ɓɓ Prospective cohort studies are very expensive and subject to loss to follow-up.
ɓɓ Prospective cohort studies can determine relative risk but are not useful when an outcome is rare.
Adapted from C. J. Mann, “Observational Research Methods. Research Design II: Cohort, Cross Sectional, Case-control Studies,” Emer-
gency Medicine Journal, p. 56. Copyright © 2003 by BMJ Publishing Group.
Case-Control Design
While cohort design looks for development of an outcome (e.g., disease) based on expo-
sure to a risk factor, case-control research design starts from a place where the researcher
knows the outcome (e.g., disease) and looks to identify the exposure (risk factor). In this
non-experimental design, the researcher begins the research after the development of the
disease or outcome and looks back in time to determine possible predictors of an outcome.
That is, the researcher looks to identify an exposure (risk factor) that explains why this
group (case) has the particular disease or condition compared to a similar group without
the disease or condition (control).
Case-control studies are very useful for studying rare outcomes due to the fact that the
outcome is already present in the case group. As is the case with most non-experimental
studies, case-control studies can also be used to develop hypotheses, which can then be
studied more in depth using an experimental design.
Case-control studies (Figure 8.3) should not be confused with retrospective cohort stud-
ies that also look back to collect data. The main difference between the two is that in case
control, the researchers match a group that has developed an outcome or a disease (case)
with a group without the outcome or disease (control) and look back in time either through
medical records or from self-reported data collected in the present to determine the level
of exposure to risk factors or variables in both groups (Mann, 2003, p. 57). It is important

Sciences
FIGURE 8.3 An illustration of a case-control study.
Exposure present
Control
outcome absent
Exposure absent
Population
Exposure present
Case
outcome present
Exposure absent
Process
Time
to note, however, that the terms retrospective and prospective apply only to describing
the different cohort research designs. The process of looking back is an inherent part of a
case-control study, and therefore it is not labeled as retrospective.
When selecting controls, it is important for a researcher to attempt to select controls that
are as closely matched to the cases as possible. Controls are often participants who would be
considered cases except they do not have the outcome or disease of interest. For example, let’s
say that researchers are conducting a study looking to prevent falls in an elderly population. The
study sought to identify factors that contribute to falls in elderly patients admitted to a rehabili-
tation center following a fall during hospitalization. They sample 370 admitted patients who fell
during hospitalization (cases) and a group of 370 admitted patients, with similar characteristics
and demographic factors, who did not fall during hospitalization (controls). By going through
the medical records of both the cases and controls, they found in the group that fell an admis-
sion diagnosis of stroke, incontinence in the week prior to the fall, and use of anticonvulsants.
(Note: The previous example was adapted from Mayo and Goldberg, 2009). The researchers then
determined that these exposures contributed to the increased chance that an elderly patient may
fall during hospitalization. While a convenience sample is easiest to find, matching controls to
cases, attempting to find characteristics in the control that are similar to those present in the
cases, can help to eliminate some instances of sampling bias (Mann, 2003, p. 58).
Issues with Case Control

The first most prevalent issue with case-control studies is sampling bias if, for example,
all the cases (research participants) are being taken from one establishment or area. When
possible, the researcher will attempt to sample cases from a variety of health care organiza-
tions; this will help increase the study’s external validity. In terms of selecting the control

Sciences
cases, Carlson and Morrison (2009) explain that matching is often used, “which refers to
selecting controls so that they are similar to cases in specific characteristics (e.g., race, age,
gender, socioeconomic status)” (p.80).
Another issue with case-control studies is observation and recall bias by the researcher
and/or the participants. This issue was presented earlier in retrospective cohort studies:
anytime a participant is asked to remember past behaviors, there is always the possibility
that participants will recall information they consider most relevant with more detail and
accuracy than information they do not consider to be relevant. This phenomenon is not
exclusive to research participants and may also happen with researchers who are collect-
ing data through observation. They may pay closer attention when they feel something is
an important behavior to observe and less attention to areas they do not consider to be as
important. As with all bias issues associated with any research that is being conducted, they
often influence the outcome or the validity of the study.
How Is Data Analysis Conducted in Cohort and Case-Control Studies?

In Chapter 5 we discussed how data in quantitative studies is analyzed using both descriptive
and inferential statistics. While it is not necessary to reiterate the information from that
chapter here, it is important to note that cohort and case-control studies utilize unique data
analysis methods when looking to establish a relationship or an association between variables.
As in most non-experimental designs, an advantage of a cohort design is that researchers
can evaluate a number of outcomes within the same study (Mann, 2003, p. 5). The data
analysis of a cohort study centers on determining relative risk (RR) as long as the outcome
is not rare. Relative risk is defined as the probability of an outcome of interest developing
as a result of the exposure being followed. In cases where the outcome of interest is rare,
a researcher would most likely opt to utilize a case-control design as opposed to a cohort.
The statistical test used in a case-control design is the odds ratio (OR), or the measure-
ment of association between an exposure and an outcome. “The OR represents the odds
that an outcome will occur given a particular exposure, compared to the odds of the out-
come occurring in the absence of the exposure” (Szumilas, 2010, p. 227). Like the Pearson
correlational coefficient, which will be discussed in the section on correlational design,
the odds ratio can determine the degree of association or the strength of the relationship
between an exposure and an outcome. This aspect of case-control research is helpful in two
areas of clinical research: it can help determine with some level of certainty whether or not
an individual should undergo a procedure or try a new drug, and, as case-control design is
often employed when evaluating risk factors for a
rare outcome, the odds ratio can assist in clinical
TABLE 8.3 Calculating the Odds Ratio
decision making when an experimental research
Calculating the Odds Ratio (OR)
design is not feasible (McHugh, 2009, p. 120).
When interpreting the odds ratio, according
OR = 1 Exposure does not affect odds of outcome to Szumilas (2010), a researcher records the odds
of an outcome occurring according to the calcu-
OR > 1 Exposure associated with higher odds of outcome lations in Table 8.3.
It is important to remember that the odds ratio
OR < 1 Exposure associated with lower odds of outcome can determine the strength and degree of asso-
ciation between variables. This being said, if the

Sciences
researcher finds statistical evidence of the association between disease and exposure, it does
not point to a causal relationship between variables. In other words, it cannot determine
that the exposure caused the disease.
In summary, both cohort and case-control studies examine exposure and disease or
conditions; however, the purpose and methods are very different between the two study
designs. Table 8.4 provides a detailed comparison of the two research designs. The infor-
mation found in Table 8.4 has been adapted from Mann (2003, p. 59).
TABLE 8.4 Comparing Cohort and Case-Control Design
Cohort Case Control
Starts with exposure, looks for disease Starts with disease, looks for exposure
Cohort studies either look forward in time or look Case-control studies look back at previously collected data and can
back in time to collect data collect data in the here and now
Participants are selected based on exposure status be- Subjects are selected because they have the outcome of interest
fore the outcome of interest (disease) is observed (disease) and matched to similar group without disease
Establish the sequence of events Establish a reason why one group has the disease and the other
Best way to establish the incidence of a disease does not
Best way to find risk factors associated with a disease
Numerous outcomes can be studied Only one outcome is studied
Statistical test used in data analysis: relative risk Statistical test used in data analysis: odds ratios
If prospective, they are expensive and often take a Prone to sampling bias and retrospective analysis bias
long time for sufficient outcome events to occur to
produce meaningful results
Adapted from C. J. Mann, “Observational Research Methods. Research Design II: Cohort, Cross Sectional, Case-control Studies,” Emer-
gency Medicine Journal, p. 59. Copyright © 2003 by BMJ Publishing Group.
Correlational Design
While it is well established that causal influence carries more weight than observation.
There are times in research when an experimental design is just not a valid or ethical option
(Russo, 2011, p. 86). In these instances, or when a researcher is looking to test a hypothesis
regarding possible relationships between variables before moving to an experimental study,
a correlational study is often used. Correlational research designs attempt to identify if the
characteristics of one or more variables are associated with the characteristics of another
variable. This allows a researcher to predict with some level of accuracy that if one variable
increases, the other will either increase or decrease (Leedy & Ormrod, 2016, p. 137). Correla-
tional research can determine the strength and degree of association between variables,
but it stops short of determining a cause-and-effect relationship. In other words, the results

Sciences
of a correlational study can tell a researcher that as X increases, so will Y, but it does not go
so far as to say X causes Y to increase.
Correlational research has the advantage of allowing the researcher to study multiple
variables, both individually and together, within one study. For example, let’s say a researcher
is looking to determine a relationship between cell phone use and incidence of migraines in
teenagers. By conducting a correlational research study and collecting data on hours spent
using a cell phone and incidence of migraines, a researcher can determine if a relationship
exists between these two variables. In addition, the study can also look at multiple variables
like watching movies on cell phones, texting, reading e-books, day/night cell phone use, and
searching websites and the relationships of each individually or whether some combination
of two or more variables is associated with incidence of migraines.
How Is Data Analysis Conducted in Correlational Studies?

Data analysis in a correlational study looks for association. Typically the data is arranged on
a scatterplot to determine association. By looking at what is referred to as the line of regres-
sion, the Pearson correlational coefficient, also known as the Pearson r, and the proximity
of the scatter points to that line, a researcher can determine if there is a correlation between
variables. The closeness of the points to the line indicates the strength of the relationship,
or the r value: the closer to the line, the stronger the correlation. If all points fall exactly on
the line, it would indicate a perfect correlation, or an r value of 1 or -1.
For example, university professors often tell students that reading the text will help
them to do better on tests, but can a researcher determine if there is a correlation between
reading the text and performing well on tests in class?
If a researcher wanted to study this research question, the researcher could take a
convenience sample of all the students in a research methods course and collect data on
self-reported reading of the text and recorded test scores after each exam. In order to pro-
vide a visual interpretation of the data, the researcher would plot it on a scatterplot with
the Y axis representing time spent reading the text and the X axis representing test grades.
If the hypothesis were true, indicating that students who spent more time reading the text
achieved higher grades on the exams, the dots on the scatterplot would slowly rise as the
researcher plotted them from left to right. Imagine now
FIGURE 8.4 An illustration of a positive correlation that all the dots (one for each student) are plotted on the
between variables. scatterplot graph and the results look something like the
example in Figure 8.4. The dots follow a path similar to
3.5 the line that is drawn diagonally across the graph. This
Y-Value 1 line is called the Pearson r, or the line of regression, and
Hours spent reading the text
3 Linear (Y-Value 1)
it represents a perfect correlation between the variables.
2.5
Notice how the line gradually rises from left to right;
2
the researcher now knows that the time spent reading
1.5 the text and exam grades have a positive correlation. If the
1 chart were to represent a perfect positive correlation or
0.5 r = + 1, the points would all fall exactly on the line. As it
0
stands, the points follow the line but do not fall exactly
0 20 40 60 80 100 120 on it, meaning that there is a strong correlation between
Test grades time spent reading the text and test grades for students

Sciences
FIGURE 8.5 An illustration of a negative correlation FIGURE 8.6 An illustration of no correlation between
between variables. variables.
3.5 3.5
Y-Value 1 Y-Value 1

3 Linear (Y-Value 1) 3 Linear (Y-Value 1)
2.5 2.5
2 2
1.5 1.5
1 1
0.5 0.5
0 0
0 20 40 60 80 100 0 20 40 60 80 100
Test grades Test grades
taking the course, but that not all students who read the text will see a high test grade.
Putting all this information together, the researcher can say that there is a strong positive
correlation between reading the text and exam grades for students in this class.
However, what if the researcher found the exact opposite was true, that students in the
course who read the text actually did worse on exams? Now obviously, this is not true,
but if it were, the researcher would be able to see this correlation plotted on a scatterplot
graph as well. Review the line of regression, or Pearson r, in Figure 8.5. Notice how as
the line follows along the path of the dots, it gradually declines from left to right. This is
indicative of a negative correlation. In other words, what this graph is describing is that
the more time students spend reading the text, the lower their exam grades are in the
class. Because the researcher sees the line gradually falling from left to right, it is known
that there is a negative correlation between time spent reading the text and exam grades.
If the chart were to represent a perfect negative correlation, or r = - 1, the points would
all fall exactly on the line. Putting all this information together, the researcher can say
that there is a strong negative correlation between reading the text and exam grades for
students in this class.
Now what if the researcher found that reading the text had no impact at all on exam
grades in the course? If this were the case (obviously, we know it is not), then it could be
demonstrated in a scatterplot graph that resembles Figure 8.6. As illustrated in this diagram,
the line of regression, or Pearson r, runs straight across the graph with no dots following any
sort of pattern. This lack of incline or decline shows the absence of a correlation between
time spent reading the text and exam grades. In other words, what this graph is describing
is that time spent reading the text has no association with exam grades at all. Because we
see the line continue straight across from left to right and the points falling all over the
chart in no up or down pattern, we know that there is no correlation between time spent
reading the text and test grades, or r = 0.
Correlations are determined through statistical measurement of the relationship between
two variables. Relationships are determined through a Pearson r, which is an inferential
statistical scale. Correlational strength is measured from -1 to +1, with a -1 indicating a

Sciences
TABLE 8.5 The following guidelines may help you remember strength of relationship
between variables
Value of r Strength of relationship
-1.0 to -0.5 or 1.0 to 0.5 Strong
-0.5 to -0.3 or 0.3 to 0.5 Moderate
-0.3 to -0.1 or 0.1 to 0.3 Weak
strong negative correlation, a +1 indicating a strong positive correlation, and a 0 indicating

no correlation exists. While it is easy to see that the closer the r value, or correlational coef-
ficient, is to -1 or +1, the stronger the correlation exists, be careful not to mistake a positive
correlation with strong and a negative correlation with weak (Cherry, 2016). For example,
a correlation coefficient of r = - .76 between the amount of time spent reading the text and
a test grade shows a stronger correlation than a correlation coefficient of r = .25 between
the amount of time spent studying the PowerPoint slides and a test grade, even though the
second number represents a correlation that is positive and the first represents one that
is negative.
Issues with Correlational Design

Correlational research does not identify a causal relationship between variables. In other
words, it can identify a relationship between variables that is either positive or negative, but
it cannot prove that one variable caused the change in the other or vice versa.
METHODS: NON-EXPERIMENTAL RESEARCH

This section of the chapter will include a brief overview of sampling and data collection
methods as they pertain to non-experimental design. For a more detailed discussion of data
analysis, refer back to Chapter 5.
How Does a Researcher Sample in Non-Experimental Research?

Chapter 5 presented different ways a researcher can recruit/select individuals to take
part in a study. These methods fall under two categories, probability and non-probability.
Non-experimental researchers can sample using either probability or non-probability meth-
ods (with the exception of theoretical or purposive).
How Is Data Collected in Non-Experimental Research?

A number of quantitative data collection methods are used for non-experimental research.
The most common are observation, surveys, or archival records. Each data collection method
has advantages and disadvantages.

Sciences
• Observation – As discussed previously in this chapter, collecting data through

observing in quantitative research involves sorting the data into predetermined and
pre-specified sections. Observations are normally broken up into small segments of
time, e.g., 5 minutes or 15 minutes; raters are highly trained in what to look for and
how to rate each observation; and there is normally more than one independent
rater collecting data at a time in order to ensure validity of the measurement tool.
■■ While it is an advantage of non-experimental research design that
variables can be viewed in a natural setting, this aspect can actually
be counted as a disadvantage, too, as a natural setting means that the
researcher cannot control the environment.
ɝɝ This means that confounding or extraneous variables can impact
the internal validity of the study.
ɝɝ Another disadvantage of observational data collection can be the
time it takes to record data. Since the recorder is observing and not
controlling the setting, it is sometimes a significant amount of time
before observable behavior occurs.
■■ Data obtained from observational research relies on the behavior of the
sample being observed. While the Hawthorne effect was presented in
Chapter 5 as a threat to a study’s internal validity, it can also be seen as limit-
ing a study’s external validity, or the generalizability of the results of a study.
ɝɝ Gall, Borg, and Gall (1996) explain the relationship between the
Hawthorne effect and external validity: “… External validity of the
experiment is jeopardized because the findings might not generalize
to a situation in which researchers or others who were involved in
the research are not present” (p. 475).
ɝɝ Researchers may try to control for this effect by remaining as unob-
trusive as possible when collecting data or increasing the sample
size to an acceptable number of participants to offset the projection
of how many participants may not be truthful.
• Survey – Survey research is a design in and of itself that we will cover in
Chapter 9. However, surveys are also a very popular and effective data collection
tool. There are many advantages to using surveys to collect data, including the
ability to collect large amounts of data in a relatively short amount of time. Sur-
veys are also relatively inexpensive to use and can be employed in many different
ways. While surveying someone in person is a popular method, online survey tools
like SurveyMonkey® or qualtrics® make it even easier and less expensive to reach
large amounts of participants relatively quickly. Surveys are also a flexible data col-
lection tool, as a researcher can design them to fit the needs of each study.
■■ One major disadvantage of collecting data via surveys is that they rely on
what researchers refer to as self-reported data. Self-reported data is sub-
ject to many threats to validity including recall bias, where respondents
recall exposure or incidents inaccurately, and response bias, where a
respondent may embellish responses in an attempt to appear more favor-
able to the researcher.

Sciences
■■ There can also be a loss to follow-up when a study has multiple data col-
lection points. Subjects can die, lose interest, or become unreachable for
any number of reasons during a study. E-mail accounts are changed fre-
quently, and people often move without supplying a forwarding address.
For this reason, the ease of sampling through e-mail or snail mail is often
outweighed by the possible lack of subject participation or follow-up. This
issue must be incorporated into determining the size of the sample (i.e.,
selecting a larger sample to compensate) when designing a study. And
lastly, researchers who collect survey data, either via paper, online, or tele-
phone, run the risk of never being sure if they are speaking to/collecting
data from the actual person they sampled.
• Archival records – Researchers often use previously collected information as
data in a study. This may be in the form of medical records or data collected in a
longitudinal cohort study. The biggest advantage to this type of data collection is
that researchers have relatively inexpensive access to large amounts of informa-
tion to pull data from.
■■ However, the downside of using this method of data collection is that
since the data has been collected for a different purpose, it may not accu-
rately reflect what the researcher is looking to measure in this study. Since
the researcher is not the one collecting the data, he or she has no control
over the participants, the methods used, or the recorded data. This may
threaten the validity of the research.
Often in research, the way data is collected can be confused with a study design. For
example, in many non-experimental research studies, data is collected once across different
groups. While this can be confused with the developmental cross-sectional research design,
it is also a common way that data is collected. In addition, a survey is a very common data
collection tool that is used in many experimental and non-experimental research studies.
Chapter 9 will define the difference between survey as a research design and survey as a
data collection tool.
The most important thing to realize when looking to determine what research design to
employ is: “What is the research question?” Meaning, what is the researcher attempting to
accomplish by collecting and analyzing data? Is the researcher looking to determine
• whether there will be a relationship between variables (correlational) to explain

or predict an outcome?
• if an exposure is linked to a disease or condition (cohort)?
• if there is an exposure that explains why a group has a disease and a similar group
does not (case control)?
• if characteristics or variables develop or change over time (developmental)?
How Is Data Analyzed in Non-Experimental Research?

Non-experimental research design employs descriptive and inferential statistics to analyze
data. With that being said, a brief explanation for some design-specific data analysis methods

Sciences
are discussed within the sections of the chapter on cohort design, case-control design, and
correlational design.
THREATS TO VALIDITY IN NON-EXPERIMENTAL RESEARCH

Each research methodology and design comes with a variety of strengths and weaknesses.
One inherent weakness in non-experimental research design is the inability of the researcher
to control for many threats to internal and external validity. While non-experimental research
can often be the only practicable method to answer a research question—especially in
instances where randomized control trials may be unethical or impractical, or if the outcome
a researcher is looking to study is rare—this type of research often has higher threats to
internal and lower levels of external validity. This is due to issues of measurement, selec-
tion bias, and recall bias when the study is retrospective; loss to follow-up in prospective
studies; confounding variables; justification for sample size; and aspects of recruitment bias
(Mann, 2003; Tooth, Ware, Bain, Purdie, & Dobson, 2005). Tooth et al. (2005) and Mann
(2003) recommend researchers use an established checklist when conducting and writing
research articles to ensure proper attention is paid to the reporting of these aspects when
conducting non-experimental research.
Threats to External Validity

External validity is the extent to which one can assume that the results of the study are
generalizable to other groups or the larger population. Non-experimental research tends to
have varying levels of external validity. Leedy & Ormrod (2016) describe the major factors
that can contribute to improving the external validity in non-experimental research.
• Real-life setting – Much of experimental research is conducted in a laboratory

or a controlled setting. This allows the researcher to oversee the variables that
are included in the study and ensure that little to nothing that has not been
accounted for affects the outcome. However, it does not provide a real-life setting
for the researcher to account for any outside, or confounding, variables; those
variables that may have an effect on the outcome of a study that a researcher may
not know to account for at the inception phase. Non-experimental research takes
place in a natural setting. In most designs, subjects are either observed in a situa-
tion or asked to recall experiences.
• Representative sample – It is important in all quantitative research to attempt
to draw a sample that is as close to representation of an entire population as
possible. In other words, non-experimental studies that are able to use one of
the probability sampling methods will have higher levels of external validity than
studies that cannot.
• Replicability – Because a researcher does not control the environment in a
non-experimental study, it may be difficult for another researcher to replicate the
same study in another setting. The more another researcher can replicate a study
in a different setting and have similar results, the more valid the results of the
first study are considered.

Sciences
Threats to Internal Validity

Internal validity, on the other hand, is the extent to which the results of the study are true
and not a result of some confounding variable or bias that has influenced the outcome and
results of the study. A brief overview of threats to internal validity was discussed in Chapter 5.
In this chapter we will review those threats that are most common in non-experimental
research design.
Because non-experimental research does not control for all variables through creating the
study in a laboratory setting or manipulating variables, there are threats to internal validity
that can possibly influence the results of the study. The following are the most common
threats to internal validity that can occur with the non-experimental designs.
• Self-selection – Non-experimental research is often conducted on groups that

share a commonality like an occupation, lifestyle, geographic location, recreation
preference, or a number of various other commonalities that can define a group.
■■ This commonality may be influenced by various factors such as education,
age, upbringing, or physical ability to name just a few. These factors can be
considered confounding variables that threaten internal validity simply by
the influence they have over why a participant is part of that group in the
first place.
• Response bias – Much of non-experimental research relies on self-reported data.
• Recall bias – Much like response bias, recall bias occurs when participants’
self-reported data have an incorrect or incomplete recollection of events or expo-
sure to risk factors being studied.
■■ It has been documented that respondents will recall with more accuracy
risk factors that they believe to have caused an outcome than those that
they do not consider to be important.
• Researcher bias – Because non-experimental research cannot always randomly
sample or use double-blind placebo controls, there is always the possibility that a
researcher’s subjectivity (personal opinions and beliefs) may accidentally play a
part in the analysis of data.
• History and maturation – Non-experimental studies like cohort or cross-sectional
often look at the natural progression or changes over time. Threats to validity
can occur if, for example, an uncontrolled historic event like a war occurs that
may drastically alter people’s perception or behavior for years to follow. Similarly,
maturation can occur at different rates within a select group of people. When
evaluating change over time in a developmental study, the maturation rate of the
subjects may be a threat to the internal validity of the study.

Non-experimental research design has many important features that allow a researcher to
conduct research that describes changes over time or shows an association between vari-
ables. While it is not used to show a cause-and-effect relationship as experimental research
does, it is a valuable research design that helps us to understand natural development and

Sciences
allows a researcher an opportunity to test hypotheses and conduct research in areas where
it may be unethical or not feasible to create an experimental research study.
In this chapter students were introduced to the defining features of several
non-experimental research designs, including observational, developmental, cohort, case
control, and correlational. To summarize some of the defining features, we know that
non-experimental designs typically involve one group, with the exception of case control,
which utilizes two groups. In addition, sampling methodologies used in non-experimental
designs can be either probability or non-probability. Most importantly, these designs do
not manipulate variables so therefore cannot establish a cause-and-effect relationship as
experimental designs do. This means that non-experimental research can discover a rela-
tionship between variables, but it does not definitely prove that one variable causes a change
in another. Table 8.6 provides a summary of the purpose, special features, and methods
specific to each design covered in this chapter.
TABLE 8.6 Non-Experimental Designs: Purpose, Special Features, and Methods
Special Feature of the Design (SF)

Methods: sampling (S), data collection
(DC), & data analysis (DA) specific to the
Design Purpose design
Developmental Study characteristics or variables as they (SF) Cross-sectional – measures development

develop or change over time from different groups (simulates growing older or
time passing)
(S) Usually quota
(DC) Snapshot – one specific day
(SF) Longitudinal – follow one group over time to

see the development of change
(DC) Has multiple data collection points
Observational Observes and records variables with the goal of (DC) Rating scale developed from theory
describing incidents, rates of behavior, etc.
Cohort To determine if an exposure is linked to the pro- (SF) Retrospective

gression of a disease or condition (DC) Using previously collected data
(DA) Relative Risk
(SF) Prospective
(DA) Relative Risk
Case Control To find an exposure (risk factor) that explains (S) Involves matching controls to the cases
why one group has a disease or conditions and a (DA) Odds Ratio
very similar group does not
Correlational To identify if the characteristics of one or more (DA) Pearson r

variables are associated with the characteristics
of another variable (strength, direction, and de-
gree of association among variables)

Sciences
Despite having greater threats to internal validity, non-experimental research designs

are a valuable way of conducting research, especially when it may be unethical or not
feasible to utilize experimental designs. For example, when the researcher is looking to
study a rare outcome, or when the researcher is looking to explore a topic, or deciding
whether or not to examine it further with an experimental design. While there may
be some threats to validity inherent in non-experimental designs, all in all the results
from these types of experiments work to advance new knowledge within the field of
health science.
STUDY PRACTICE
At this point you should be able to apply concepts previously presented in the book. For
example, in the following questions, you will be asked about which design was used but also
to identify what sampling method was used (when appropriate). As in previous chapters,
select the best or most correct answer.
ɠɠ Developmental research designs ɠɠ Relative risk (RR)
ɝɝ Cross-sectional ɠɠ Odds ratio (OR)
ɝɝ Longitudinal ɠɠ Correlational research
ɠɠ Observational research ɝɝ Strength
ɝɝ Rating scale ɝɝ Degree of association
ɠɠ Cohort research ɝɝ Line of regression
ɝɝ Prospective ɝɝ Pearson Correlational Coefficient
ɝɝ Retrospective (Pearson r)
ɓɓ Subject recall ɝɝ R value
ɓɓ Self-reported data ɠɠ Non-experimental research
ɝɝ Outcome ɝɝ Observation
ɠɠ Case-control research ɝɝ Survey
ɝɝ Archival records

1. W hich statement is true about experimental designs and non-
experimental designs?
a. Both must manipulate variables
b. Both must use probability sampling methods
c. One seeks to determine causation and one seeks to identify relationships
d. One uses many subjects and one uses very few carefully selected subjects
2. To date no study has focused on the food color groups (red, blue, orange, green, and
white) of fruits and vegetables and colon cancer. A researcher recruits 500 people
with colon cancer from oncology surgical units of regional hospitals and samples
another 500 very similar people (e.g., age, socioeconomic status) who had surgery but
not for cancer, from the general surgery units of regional hospitals. The participants
were asked to recall the types of fruits and vegetables they ate on a regular basis;

Sciences
the researcher was looking to compare food color groups (fruits and vegetables)
between those with colon cancer and those without colon cancer. Scenario adapted
from Lee, Shin, Oh, and Kim (2017).
Which study design was used?
a. Retrospective cohort
b. Prospective cohort
c. Correlational
d. Case control
3. “Hand hygiene is now regarded as one of the most important element[s] of infection
control activities” (Mathur, 2011, p. 611). The purpose of this study is to explore
the incidences of proper hand hygiene being used in a local hospital. Using a rating
scale developed from guidelines put forth from infection control experts on proper
handwashing techniques, numerous highly trained research assistants were placed
in patients’ rooms to rate the completeness of the handwashing practices of every
staff member who entered the room to provide services to the patient.
a. Prospective cohort
b. Observational
c. Developmental: cross-sectional
d. Developmental: longitudinal
4. The Zika virus is spread by mosquitoes. The CDC documents that many people
might not even realize they have been infected with the virus. A group of health
science students are invited to participate in an international internship in an
area of the world with a high incidence of the Zika virus. The students agree to
be followed for 6 months to see if they have any of the symptoms of the disease,
with special attention given to unusual symptoms and previously unreported
signs and symptoms.
c. Observational
d. Case control
5. The purpose of the study is to explore the eating habits of third graders when
there is no parental oversight. The researchers divide the school districts on Long
Island in socioeconomic groups by zip code (disadvantaged, middle income, and
wealthy), then randomly select 300 students from each group. They operationally
defined behaviors based on the guidelines put forth by the USDA, developed a
rating scale tool, and carefully trained their research assistants to use the tool in
order to determine:
• what types of food the children brought from home for lunch;
• what type of food is swapped between the children; and
• amount and type of food that is actually eaten by the children.

Sciences
Which study design should be used?

a. Prospective cohort
b. Observational
d. Case control
6. The researcher disseminates a survey to several local churches, temples, and mosques.
The purpose of the study is to determine the association between independent
variables (religious beliefs, age, gender) and the dependent variable being vaccinated
for influenza.
a. Correlational
b. Retrospective cohort
d. Prospective cohort
7. A study sought to investigate factors that influence women’s health. Female students
were identified and randomly selected (n = 200,000) from the national student loan
database. The women selected consented to be followed for 20 years. At the start
of the study, none of the women had been diagnosed with cancer, cardiovascular
disease, diabetes, or HIV. Annually, they completed questionnaires and allowed the
researchers access to their medical records. While cancer was the “primary focus,
the study has also produced landmark data on cardiovascular disease, diabetes, and
many other conditions. Most importantly, [this] study show[ed] that diet, physical
activity and other lifestyle factors can powerfully promote better health.” Example
adapted from http://www.channing.harvard.edu/nhs/
c. Correlational
d. Case control
8. A new allergy medication entered the market in 2000; the drug was removed from
the market in 2002 due to FDA concerns over life-threatening side effects (heart
attacks in a small percentage of people who took the drug). In 2015, a study was con-
ducted with 10,000 individuals identified from the Medicaid and Medicare databases
that had been prescribed the drug in 2000–2002 and used it daily for 6 months or
longer. The study participants consented to have their medical records from 2000
to 2015 reviewed to see if the researchers could find any increased incidents of other
health-related issues.
c. Correlational
d. Case control

Sciences
9. A questionnaire was sent to 1,500 randomly selected individuals who had been
admitted to a hospital for a length of 7 or more days (May 1, 2016 to June 30, 2017).
During data analysis, the researcher wanted to see which of the following variables
were associated with confidence in the medical team: patient demographic infor-
mation, physicians’ ethnicity, hospital location, amount of time physician spent
with patient, level of empathy from the medical team, health literacy of the patient,
visits from the chaplain service, and number of visitors the patient had. Data anal-
ysis revealed a strong positive relationship between empathy and confidence. The
analysis also revealed that the lower the patients’ health literacy, the lower the rates
of confidence in the medical team. All other variables revealed no association with
confidence. This study underscores the association between confidence in medical
care and the medical team’s ability to communicate medical information to patients
in an effective manner.
c. Correlational
d. Case control
10. The purpose of this study is to explore the impact of violence on the progression
of normal developmental milestones in a group of kindergarten children who sur-
vived a violent school-based incident. The researcher will collect many different
types of data twice yearly from school records and in-depth structured interviews
with the children, the children’s parents, and teachers now and until they graduate
high school.
d. Developmental: longitudinal
STUDY ACTIVITIES
Now that you are more than halfway finished learning the concepts and designs presented
in this textbook, it would be a good time to go back and create a study list of terms that
have different meanings when placed in various contexts. In addition, create an additional
list of concepts that appear to be similar but actually are distinct. When creating the lists,
try to answer the following questions:
Word alert: How do the definitions of words/concepts change based on the context?
• The variations of the word observe. This word can reference four different con-
cepts based on the context of the example. What are they?
• The variations of the word correlation, which can be used in two separate ways.
What are they?

Sciences
Even though they are similar concepts (collecting data from a research participant over
a timeframe of several years), when do you use the following words?
• Prospective
• Longitudinal
REFERENCES
Carlson, M. D., & Morrison, R. S. (2009). Study design, precision, validity in observational studies.
Journal of Palliative Medicine, 12(1), 77–82. doi:10.1089/jpm.2008.9690
Cherry, K. (2016, August). What is correlation? Verywell. Retrieved on June 7, 2017, from https://www.
verywell.com/what-is-correlation-2794986
Cruz, J., Marques, A., Barbosa, A., Figueiredo, D., & Sousa, L. X. (2013). Making sense(s) in dementia:
A multisensory and motor-based group activity program. American Journal of Alzheimer’s Disease
and Other Dementias, 28(2), 137–146. doi:10.1177/15 33317512473194.
Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction. White Plains, NY:
Longman.
Gross, M. (2006) Tips for parents: What we know from longitudinal studies of E/PG children. Retrieved
from http://www.davidson gifted.org/Search-Database/entry/A10424
Hall, J. (2008). Cross-sectional survey design. In P. J. Lavrakas (Ed.), Encyclopedia of Research Design.
Thousand Oaks, CA: SAGE Publications. doi:10.4135/9781412963947.n120
Hammerlid, E., Silander, E., Hörnestam, L., & Sullivan, M. (2001). Health related quality of life three
years after diagnosis of head and neck cancer—A longitudinal study. Head & Neck, 23(2), 113–125.
Han, J., Colditz, G. A., & Hunter, D. J. (2006). Risk factors for skin cancer: A nested case-control
study within the Nurses’ Health Study. International Journal of Epidemiology, 35(6), 1514–1521.
Lee, J., Shin, A., Oh, J. H., & Kim, J. (2017). Colors of vegetables and fruits and the risks of colorectal
cancer. World Journal of Gastroenterology, 23(14), 2527–2538. doi:10.3748/wjg.v23.i14.2527
Leedy, P., & Ormrod, J. (2016). Practical research: Planning and design (11th ed.). Boston, MA: Pearson
Education.
Lobmeier, J. H. (2012) Nonexperimental designs. In N. Salkind (Ed.), Encyclopedia of Research Design.
Thousand Oaks, CA: SAGE Publications.
Mann, C. J. (2003). Observational research methods. Research design II: Cohort, cross sectional,
case-control studies. Emergency Medical Journal, 20, 54–60.
Mathur, P. (2011). Hand hygiene: Back to the basics of infection control. Indian Journal of Medical
Research, 134(5), 611–620. doi:10.4103/0971-5916.90985
Mayo, N. E., & Goldberg, M. S. (2009). When is a case-control study not a case-control study? Journal
of Rehabilitation Medicine, 41(4), 209–216. doi:10.2340/16501977-0343
McHugh, M. L. (2009). The odds ratio: Calculation, usage, and interpretation. Biochemia Medica,
19(2), 120–126. doi:10.11613/BM.2009.011
Russo, F. (2011). Correlational data, causal hypotheses, and validity. Journal for General Philosophy of
Science, 42, 85–107. doi:10.1007/s10838-011-9157-x
Szumilas, M. (2010). Explaining odds ratio. Journal of the Canadian Academy of Child & Adolescent Psy-
chiatry, 19(3), 227–229. www.ncbi.nlm.nih.gov/pmc/articles /PMC2938757/pdf/ccap19_3p227.pdf
Tooth, L., Ware, R., Bain, C., Purdie, D., & Dobson, A. (2005). Quality of reporting of observational
longitudinal research. American Journal of Epidemiology, 161(3), 280–288. doi:10.1093/aje/kwi042

Sciences
UNDERSTANDING QUANTITATIVE
RESEARCH: SURVEY RESEARCH
INTRODUCTION CHAPTER GOALS:

9
This chapter focuses on surveys as a research design, which falls under
the category of non-experimental research, and presents features specific
to designing high-quality survey research. Table 9.1 lists the action steps • To introduce the defining features of
involved in the design stage of survey research. As stated in the previous survey research design.
chapter, non-experimental research encompasses those designs where • To provide students with enough
the researcher does not manipulate variables; rather, the researcher looks information to know when to use
to identify characteristics that occur naturally or attempts to consider survey design.
associations between two or more variables. Survey research looks to • To provide students with working
understand a population by collecting data about knowledge, attitudes definitions of the terms filter/screen-
and feelings, perception and beliefs, and behaviors for certain variables ing questions, Likert scale, Guttman
(Creswell, 2009). It collects this data through asking questions of respon- scale, rank order, and dichotomous.
dents, recording the responses, and analyzing them statistically. As with
• To furnish students with the tools
all non-experimental research designs discussed in Chapter 8, survey
necessary to critique survey ques-
research can correlate an association between variables, but it cannot
tions for a research project.
go so far as to determine a cause-and-effect relationship. This chapter
will provide a detailed explanation of survey research, including how • To provide students with an
surveys are constructed and the strengths and limitations of using sur- understanding of the strengths
veys to collect data. It will also clarify how the word survey has different and weaknesses inherent in
meanings based on whether it is referenced as a data collection tool or survey design.
a research design.
SURVEY RESEARCH—DEFINED LEARNING OBJECTIVES:

The word survey has two separate definitions based on the context in BY THE COMPLETION OF THIS CHAPTER, THE
which it is used. STUDENT WILL BE ABLE TO:
Survey research design (also known as survey research) is a research • Define at least seven terms related to
design used to develop an understanding of a population’s knowledge, survey research design;
attitudes and feelings, perception and beliefs, and/or behaviors about
• Recognize the characteristics of
specific issues (Creswell, 2009). In health science, surveys are widely
survey research design;
used to develop understandings of different populations’ health-related
• Compare appropriate research
knowledge, attitudes and feelings, perception and beliefs, and behaviors.
question styles utilized for survey
Survey research is conducted by asking a sample from the population
research design;
questions using a data collection instrument known as a survey (defined
• Demonstrate the ability to match the
below). The purpose of this research is to develop an understanding of
most appropriate survey question to
the sample, that will then be used to “predict attitudes and behaviors” or
collection type; and
“describe attributes” of the population (Teddlie & Tashakkori, 2009, p. 24).
• Identify properties of various biases
inherent in survey design and ways
187 to control for each.

Sciences
TABLE 9.1. Stages of Research: Action Steps Highlighted in This Chapter
Stages Action Steps
As seen in previous chapters, the individuals who consent to be in a research study have been
referred to as study participants or human subjects. Specific to survey research design, the
individuals who consent to be in the study are often referred to as respondents because they
responded to the researcher’s request to complete the survey (data collection tool).
Survey is a data collection instrument. There are two styles of surveys: questionnaire
and interview. While interviews are used in qualitative research, the difference here is the
structure and intention of the data collection process. When a survey is used to ask specific
questions, whether data is being collected through a questionnaire or an interview, there is
a level of standardization in data collection (Jacobsen, 2017; Teddlie & Tashakkori, 2009).
Standardization means that everyone is asked the same questions in roughly the same order
using the same terminology. These questions can be delivered in a variety of ways and look
to solicit a range of responses from simple yes/no to in-depth explanations of feelings or
attitudes. Additionally, the answers to those questions are analyzed numerically (e.g., 95% of
the individuals surveyed agreed that annual influenza vaccination is vital to staying healthy,
with a margin of error of +/-5). Survey as a data collection tool is very versatile and can be
used in any type of quantitative research study. Consider this example: Fujimori, Shirai,
Asai, Kubota, Katsumata, and Uchitomi (2014) conducted a study where they randomly
assigned physicians to either an experimental group (skills training to improve communi-
cating a cancer diagnosis) or a control group. Surveys were one of the ways the researchers
used to collect data. Just because the researchers collected data through the use of a survey,
the study’s research design did not change; it remained an experimental research design.
Survey research can be conducted on small groups, or it can collect data worldwide
to assess the global population’s opinions, attitudes, and behavior on various topics. One
example of survey research’s ability to assess data from large groups of people is the Pew
Research Center (www.pewresearch.org). The Pew Research Center utilizes telephone sur-
veys to collect data and “inform the public about the issues, attitudes, and trends shaping
America and the world” (www.pewresearch.org/about/). Topics range from abortion to
world elections, with a comprehensive alphabetical listing in between. The Pew Research
Center’s survey reports, demographic studies, and data-driven analyses are available for free
on the Internet for anyone who is interested in obtaining facts on public opinion regarding
a wide array of subjects.
SIDE BOX 9.1 Now is a good time to review the purpose of each research design discussed in previous
chapters. Remember, the research question—what the researcher is looking to find out—drives the meth-
odology and design of every research project.

Sciences
Chapter 9 Understanding Quantitative Research: Survey Research | 189
How Is Survey Research Conducted?

Survey research (design) is conducted by the dissemination of questionnaires (data col-
lection) and by conducting interviews (data collection). There are a variety of survey data
collection formats, each having their own strengths and weaknesses. The most common
strengths and weaknesses listed below are adapted from Owens’s (2002) SRL Fall Seminar
Series (PowerPoint slides 6–10).
Personal Interview
• Strengths
■■ Generally has high response rates
■■ Allows interviewer to elaborate on questions or ask for clarification
■■ Responses are usually easy to analyze
• Limitations
■■ Costly due to large number of interviewers needed
■■ Data collection is slow, and overall study requires a lot of time
■■ Difficult to control for interviewer bias
Telephone Interview
• Strengths
■■ Less costly because no field work is required
■■ Random digital dialing (RDD) allows researchers to reach a large repre-
sentative sample
■■ Data collection takes less time than in-person interviews
■■ Allows interviewer to elaborate on questions or ask for clarification
■■ Generally has a better response rate than mailed surveys
• Limitations
■■ Can only reach households that have telephones
■■ Higher non-response rate than in-person interviews
■■ Subject listens and responds with no visual clues
■■ Hard to control for question confusion when answering
Mail Survey
• Strengths
■■ Cheaper than phone or in-person interviews
■■ The need for a smaller number of interviewers/staff contributes to
lower cost
■■ Provides access to a large representative sample
■■ Respondents can participate when it is convenient

Sciences
• Limitations
■■ Easy for respondent to not participate or forget to participate
■■ Low response rate
■■ Incentives may increase participation but also increase cost
■■ Longer waiting period for responses to be returned
■■ Reminders increase response rate but also increase cost
Online Survey
• Strengths
■■ Lowest cost
■■ Provides access to global population
■■ Timely
■■ Easy to collect only relevant data through online programs
■■ Provides access to an enormous representative sample
• Limitations
■■ Varying computer capabilities may not allow access to some households
■■ Easy for respondents to ignore or delete requests, which leads to very low
response rates
■■ May contain higher response rates from those interested in the topic,
resulting in bias data
When planning the research project, it is important to review the strengths and weak-
nesses of each data collection method. This will help a researcher decide which method
will best align with the scope and size of the sample needed. Remember, if the researchers
wish to obtain generalizable results, they must use one of the probability sampling meth-
ods, sample size estimator, and then balance the strengths and limitations of each survey
format presented above.
WRITING QUESTIONS FOR A SURVEY INSTRUMENT

There is a difference between the research question (Chapter 4) that is guiding the entire
survey research study and individual questions that comprise the survey instrument. How-
ever, since the survey instrument is how the researcher collects the data, these questions
are the most important aspect of survey research design. “Their performance ordinarily
has a more profound effect on the survey results than has any other single element of the
survey” (Alreck & Settle, 1995, p. 87). Therefore, it is extremely important that the questions
that comprise the survey instrument are well written, with particular attention paid to:
Focus – Each question should focus on a specific topic. Questions often become confusing
when the focus is unclear.

Sciences
For example, if the researcher asks the question, Which pain reliever do you use
most often? the respondent may be unsure whether to list the brand of over-the-
counter pain relief used for an occasional headache or a prescription narcotic
used after surgery. A more focused way to ask this question may be List the
brand of over-the-counter pain reliever you most often buy. The second question
is focused on a type of pain reliever and indicates that in the wording of the
question (Alreck & Settle, 1995, p. 88).
Clarity – The meaning of the question should be as clear as possible to avoid misinter-
pretation and incorrect answers.
For instance, it is much clearer to ask, Do you utilize the patient portal on your pri-
mary care physician’s website? than to ask, Do you use the computer for health-related
issues? The first question is clearly asking about a specific use of online services,
while the second is vague in describing what the researcher is actually looking
to find out. With the second question the respondent may answer yes, but the
reason may be because he or she often looks up symptoms on the web to diag-
nose a possible illness without visiting a doctor (Alreck & Settle, 1995 p. 89).
Brevity – Shorter questions are easier to answer and provide less risk of the respondent
answering only a portion of the question or skipping over it altogether.
For example, if a researcher is looking to determine how many times a month
someone eats fast food, it would be better to ask
Indicate the number of times you ate at each fast food restaurant in the past month
McDonald’s
Burger King
Wendy’s
Chick-fil-A
Taco Bell
Checkers
Other (Please specify )
as opposed to Can you tell me how often you eat fast food and what fast food restaurants you
have eaten at in the past month? The first question will result in an itemized list of where and
how often; the second option may not provide the level of detail needed in the response
(Alreck & Settle, 1995 pp. 88–89).
It is important to note that questions should only ask for the level of detail that is needed
in each case. If it is enough to know how often someone has eaten fast food in the past
month, there is no reason to ask for more details on which restaurants he or she has gone
to and how often at each one. The more details that a question requires from a respondent,
the more intrusive the questionnaire may feel and the greater the likelihood that someone
may not complete the survey.
Survey research design relies on questions as a way to collect information (data) from
respondents. “Answers are of interest not intrinsically but because of their relationship
to something they are supposed to measure” (Fowler, 2014, p. 75). When writing survey
questions, it is important to focus on reliability, “providing consistent measure in compa-
rable situations,” and validity, “answers correspond to what they are intended to measure”

Sciences
(Fowler, 2014, p. 75). Wording should be easy to understand and free from technical jargon
and overly difficult words and phrases. While the researcher who is designing the questions
is often well versed in academic language and topic-specific jargon, the average respondent
is not. Similar to the informed consent form discussed in Chapter 2, a good guideline is to
use the national reading level when constructing the survey—meaning the language in the
survey (data collection instrument) should not exceed an eighth-grade reading level. This
will ensure that most respondents will not misinterpret the meaning of a question or have
difficulty answering.
Survey questions can be open or closed, also referred to as structured or unstructured.
Open, or unstructured, questions allow the respondent some sense of freedom to answer
the question and give the opportunity to elaborate on the topic using his or her own words.
There is much less structure to the responses than there is with closed questions. The answers
to these questions usually involve some form of qualitative analysis. Closed, or structured,
questions limit the responses that can be given by requiring that each respondent indicate
agreement or disagreement with predetermined choices. With structured questions there is
no opportunity to deviate from the script. Closed or structured responses are easy to quantify
and are turned into numerical form for analysis. In semi-structured surveys the respondent is
asked predetermined questions with an occasional open-ended question when the researcher
is looking for clarification or an elaboration on a response. While open questions are often
easier to write, they are usually harder to quantify and analyze during the data analysis phase
of research. Conversely, closed questions require much more planning and design when they
are being written, but they tend to be much easier to quantify and analyze later in the research
process. The terms open and closed or structured and unstructured are used throughout survey
research to describe data collection techniques, survey tools, and question design.
How Questions Are Structured on Survey Instruments

There are several standard survey question structures; this section will provide examples
of the most common. Note that survey questions, based on how the question is structured,
can be written in question or statement formats. Also notice how data scales, presented in
Chapter 5, are related to the structure of the question, meaning the structure of the question
will yield specific numerical values. The types of numerical values (data scales) are related
to the type of statistical analysis that can be conducted on that question.
Writing Nominal Questions

Nominal questions are used to label, name, or group responses. With nominal questions,
the researcher does not assign a value to each response. For example:
My hair color is:

Brown
Black
Blond
Gray
Other

Sciences
What is your religious affiliation?

Christian
Jewish
Buddhist
Muslim
Other
None
When collecting data through the use of nominal questions, the responses are not rank
ordered or compared, and they often do not identify a measurement. Black hair is not twice
as better than brown hair, for example, just as a person who is one religion does not have
more religion than someone who is another. When nominal responses are assigned numbers,
it is solely for the purpose of data analysis, where the numbers simply name a difference. For
example, data collected with nominal questions can describe variable characteristics such as
demographic information. The researcher would report these demographic variables in per-
centages (e.g., 47% of the respondents had brown hair or 30% of the respondents were Muslim).
Writing Ordinal Questions

Ordinal questions assign meaning to responses by ranking them in order from lowest to
highest or vice versa. For example:
What is your annual income?

1. Less than $20,000
2. $20,000 to $29,000
3. $30,000 to $39,000
4. $40,000 to $49,000
5. Above $50,000
or
How much does your pain affect your everyday life?

Not at all
Somewhat
A lot
With ordinal questions the responses can be rank ordered, meaning that one response
is better/larger/more than another response. Someone who makes $47,000 makes more
than a person who makes $32,000, just as pain that affects a person’s everyday life is a lot
more disruptive than pain that does not affect everyday life at all. It is important to be able
to order responses when analyzing data. While ordinal questions allow the researcher this
ability, it is often difficult or impossible to determine the distance between the responses
in ordinal questions. For example, there may be a small difference between “Not at all” and
“Somewhat” when determining the effect of pain, but there may be a much larger distance

Sciences
between somewhat of an effect and a lot. Additionally, as for the difference between response
4 and response 5 on the income question, one could be $1,000, or it could be $10,000; the
research has no way of determining the interval with an ordinal scale.
Ordinal questions also allow respondents to rank answers in whichever order they want.
For example:
Rank the following exercises in order of preference from 1 to 4:

Cycling
Swimming
Running
Walking
When written correctly, this type of question allows the respondent to apply a value to
each possible response. Incorrectly written ordinal questions can sometimes create confusion
for the respondent and result in incorrectly answered questions that can no longer be used
when analyzing the survey data. Wording the question in a way that does not adequately
explain how the response should be recorded is one way that can create confusion.
Writing Interval Questions

Interval questions rank order responses like ordinal questions; however, unlike ordinal ques-
tions, where the distance between responses is unknown, the distance between responses in
interval questions is measured in standard increments. Interval scales allow the researcher
to utilize a wider range of data analysis, including inferential statistics and averages. The
three types of interval questions we will discuss here are Likert, semantic differential,
and Guttman.
Using a Likert Scale – A Likert Scale is a psychometric scale developed by Rensis Likert to
quantifiably measure abstract concepts like “attitudes, values, internal states, and judgments
about … behaviors in both research and clinical practice” (Mellor & Moore, 2014, p. 369).
When using a Likert scale, consecutive numerical values are assigned to corresponding
statements that indicate levels of agreement or disagreement with a statement (Likert, 1932,
p. 42). Likert scales offer respondents a spectrum of choice ranging from neutral to varying
levels of agreement at one end of the spectrum to varying levels of disagreement on the
other end. The most common ratings scales used are 5-point and 7-point (Alreck & Settle,
1995 p. 448; Dawes, 2008, p. 62). A typical Likert-style question begins with a statement and
allows a respondent to decide the level of agreement with the statement. It must be noted
that some researchers categorize Likert scale questions as ordinal questions, while others
categorize this question structure as interval. For the purpose of this chapter, Likert scale
questions are presented as interval.
Using the patient portal saves time when I am making appointments.
Strongly agree Agree Neutral Disagree Strongly disagree
1 2 3 4 5

Sciences
Doctors have an obligation to relieve pain and suffering by any means in end-of-life care.
Strongly Agree Agree Undecided Disagree Disagree Disagree

Agree Somewhat Somewhat Strongly
1 2 3 4 5 6 7
However, sometimes a researcher will use what is called a “forced choice” (Allen & Seaman,
2007) version of the scale in order to remove the neutral response and force respondents to
choose a side. However, this can negatively impact the validity of the study if respondents
truly do not have an opinion on a topic and therefore choose not to answer or pick a response
that is truly not indicative of his or her attitude or belief.
It is important to sleep at least 7 hours every night.
Strongly agree Agree Disagree Strongly disagree
1 2 3 4
Likert scale questions can be written at almost any reading level and are often used in
research studies that focus on children’s feelings and opinions. Many of the scales that have
been developed for children, such as the Children’s Impact of Traumatic Events Scale–
Revised, the Illness Attitudes Scale, and the Children’s Depression Inventory, utilize 3-point
scales or pictures (Image 9.1) in order to make the questions more easily understood by
children (Mellor & Moore, 2014, pp. 369–370).
I like going outside for recess.

IMAGE 9.1 An example of a visual image–based Likert scale.
Adapted from https://pixabay.com/en/emoticons-smilies-set-smi-
ley-blue-150528/
Using a Semantic Differential Scale – The semantic differential scale, developed by Charles
Osgood, is another way that researchers measure attitudes, values, and opinions by having
respondents rate their opinion or belief on a scale using bipolar adjectives. A semantic dif-
ferential scale can ask a number of different questions in a short amount of time and space,
making it a useful option for many researchers. Utilizing a neutral middle point between the
bipolar adjectives serves as a zero point, allowing the semantic differential to serve as an
interval scale much like a Likert scale. However, unlike a Likert question, where a respondent

Sciences
FIGURE 9.1. An example of a semantic differential

scale.
agrees or disagrees with a statement, a semantic differential
Adapted from: Colin Robson, from Real World Research: A
scale uses adjectives or phrases that are completely opposite
Resource for Social Scientists and Practitioner-Research- of each other to classify within a relatively short amount of
ers, p. 10. Copyright © 1993 by Blackwell Publications. time and space the respondent’s degree of feeling positively
or negatively toward any given concept (Grove et al., as cited
in Park & Shin, 2015, p. 526). Using this type of scale in their
Semantic Differential Scale
research study assessing elder Koreans’ attitudes toward sex-
For each pair of adjectives place a cross at the point between them which
reflects the extent to which you believe the adjectives describe policemen
uality, Park and Shin were able to create a scale that reflected
Korean cultural attitudes about sexuality and also about
clean : : : : : : : : dirty
elder family members. In addition, the design worked well
honest : : : : : : : : dishonest
with this population as opposed to other more well-known
kind : : : : : : : : cruel
scales such as the “Aging Sexual Knowledge and Attitude
helpful : : : : : : : : unhelpful Scale” (ASKAS) (White, 1982) because it utilizes single words
fair : : : : : : : : biassed at opposite ends of the spectrum rather than long statements
strong : : : : : : : : weak that respondents must take time to read and comprehend
foolish : : : : : : : : wise before indicating agreement or disagreement (Park & Shin,
energetic : : : : : : : : lazy 2015, p. 526). Consider the semantic differential example in
unreliable : : : : : : : : reliable Figure 9.1 about police officers.
(Robson, 1993) Using a Guttman Scale – The last interval scale dis-
cussed is the Guttman scale. Developed in the 1940s, the
Guttman scale is a cumulative rank scale often used to
determine an individual’s knowledge, or the existence or
degree of agreement with a concept or belief. With this type
of scale, statements are listed in ascending or descending
order, starting with the least extreme and moving forward with the most extreme statement
appearing last. Respondents are asked to check off the statement that he or she agrees with.
Agreement with a statement in the list can predict agreement with the previous statements.
Alvarelhão and Lopes used a Guttman scale to “assess the knowledge and awareness about
Sexually Transmitted Diseases for adults with Cerebral Palsy, in Portugal” (2016, p. 487).
While the structured questionnaire used in the study consisted of 18 questions about overall
sexual knowledge, 10 of the questions were identified as pertaining to Sexually Transmitted
Diseases (STDs) and of those 10, a Guttman scale was created incorporating eight questions
that assessed knowledge of STDs in rank order with the easiest questions first. In this case,
a wrong answer on one question predicted a wrong answer on all subsequent questions on
this topic (Alvarelhão & Lopes, 2016). In Table 3 of their research paper, the authors list
the eight Guttman Scale questions they created. An example of three cumulative ranked
questions are presented here: “If a condom is punctured or gets damaged during its open-
ing [sic] should not be used” was the easiest question on the scale; the mid-scale question
was “Sexually transmitted disease may result in more serious health conditions”; moving
up to the most extreme knowledge question, “Redness, warts or blisters on the genitals or
around may be signs of a sexually transmissible infection” (Alvarelhão & Lopes, 2016, p. 491).
Guttman scales are difficult to develop due to the necessity of having each question build
on the concept of the previous question in a logical order, but once created they are highly
reproducible (Page-Bucci, 2003).

Sciences
Writing Dichotomous Questions

Many survey questions are dichotomous, meaning they offer two possible answers for the
respondent to choose:
• yes/no;
• agree/disagree; or
• true/false.
Dichotomous questions are closed questions in that the question dictates exactly how the
respondent must answer with no room for variation in responses. They are also structured,
which makes them easy to quantify during data analysis. Dichotomous questions are often
used when collecting data on respondent characteristics.
Are you a US citizen? Yes No
They can also be used to gain information on a sample’s opinion regarding a topic.
I believe that marriage should only take place between a man and a woman.
Agree
Disagree
Another area where dichotomous questions are often used is when a researcher seeks
information regarding a respondent’s experience.
I have used the patient portal to access my health records from my primary care physician.
True
False
They are often used as filter/screening questions when a researcher is looking to weed
out a portion of the respondents to whom the survey does not relate. Figure 9.2 provides a
mapping example of how a filter question guides a respondent through the survey questions.
While dichotomous questions can collect data fairly easily and quickly, the data that can
be collected this way is limited. Also, as is the case with all structured questions, dichoto-
mous questions must be written correctly in order to make them easy for the respondent
to understand and answer quickly and with confidence. An example of a common dichot-
omous question that until more recently had been used on most surveys to collect data on
respondents’ characteristics is
Are you Male Female
With a growing number of the population becoming more open regarding the fluidity
of gender identity, this historically common dichotomous question may now pose prob-
lems for some respondents who do not identify as the gender they were born into or who
no longer identify as one specific gender. It is important for the researcher to determine
how important the question is to the research. If, for example, gender plays an important

Sciences
FIGURE 9.2 An example of how a filter question can be used in

a survey.
role in the analysis of the data, the researcher
may reframe the question as a nominal ques-
tion, for example the researcher may use the
Have you ever taken Human Rights Campaign recommendation on
prescription pain
medication?
how to word this question (https://www.hrc.org/
resources/collecting-transgender-inclusive-gen-
der-data-in-workplace-and-other-surveys):
No Yes
Thank you for What is your gender?

your time. The The last time I took prescription pain medication Female
survey is now it was (choose one):
complete. Male
Non-binary/ third gender
Prescribed for me by my Prefer to self-describe
doctor for management of
the pain I was currently Prefer not to say
having.
Given to me by a friend or Writing Open-Ended Questions

relative from a prescription
written for them. Open-ended questions are unstructured ques-
tions that allow a respondent to write a response
Prescribed to me by my in his or her own words.
doctor for previous pain
management unrelated to
this pain.
How do you feel about staying in a hospital overnight?
Open-ended questions are often easier to

Obtained illegally. write but are much harder to turn into quan-
tifiable data. Another issue with open-ended
questions is that they can often be misunder-
Other stood by a respondent, often resulting in answers
that are difficult to quantify or group together.
Poorly Constructed Survey

Instrument Questions
As stated earlier, the data collection tool (the
questions in the survey) is the most important aspect of how a survey research design study
answers the research question. Most issues in survey research occur from poorly constructed
questions (data collection tool). Some examples are:
Double-Barreled Questions
Asking two questions in one where the respondent may feel differently about the two con-
cepts and therefore is unable to answer the question.
Do you support the Affordable Care Act (ACA) and socialized medicine?
Yes

Sciences
No
A respondent who supports one concept but not the other would be unable to answer
this question with a yes/no response.
Bias/Loaded Questions
Framing the question in such a way that does not allow the respondent to disagree with the
question or that creates an assumption of the respondent’s feelings or beliefs.
Why are Physician Assistants better than Nurse Practitioners?
Sensitive Questions
It is often necessary in survey research to ask questions that deal with a sensitive topic.
Respondents may become uncomfortable and not want to answer these questions or may
answer untruthfully in order to look good to the researcher. When asking questions that
are sensitive in nature, it is important to set the tone or introduce the section in a way that
may make the respondent feel more comfortable.
Poorly constructed sensitive question:
How many times in a week do you eat in front of the TV?
Better question construction:
We’ve found that many people often eat dinner while watching television. During a typical
week, how many nights a week do you eat dinner while watching television?
0–2
3–5
5–7
It is important to make the respondent feel that the response given is socially accept-
able. Introducing a sensitive question with a disclaimer or a way to put the respondent
more at ease with their response can increase the likelihood that the answer given will
be truthful.
Question Content/Question Wording

Are all the questions that are included necessary, and are they worded in such a way that
they can be answered relatively quickly and easily? Look back to the question on sensitive
topics used in the previous section. The question asks the respondent to indicate how many
nights during a typical week. This will eliminate confusion on how to answer the question
if, for example, the respondent is answering the question during a particularly stressful
week where the answer may be 5–7. Overloading a survey with too many or too complex
questions can become frustrating to the respondent.
• How often have you visited a doctor during the past year?
• How many times have you been treated as a patient in an emergency department
during the past year?

Sciences
FIGURE 9.3 An example of how • During the past year, how many times were you admitted to
survey questions can be com- the hospital?
bined.
Combining these questions (Figure 9.3) under one heading simplifies the
Circle one answer for each survey and eliminates unnecessary wording and the feeling of answering
question. the same question more than once.
During the past year, how many
times have you: Some Things to Keep in Mind When Constructing Survey Questions
1. Visited a doctor?
1. None • Does the question make sense?
2. 1-2
3. 3-4 • Is the question useful?
4. More than 5 • Is the question necessary?
2. Been a patient in the
emergency department?
• Is more than one question needed?
1. None • Does the wording portray a clear meaning?
2. 1-2 • Will this survey take up too much time and cause the respondent
3. 3-4
4. More than 5 to lose interest and walk away?
3. Been admitted to the • Will the respondent answer truthfully?
hospital?
1. None
2. 1-2
It is important to remember when designing a survey that the goal is to create
3. 3-4 a tool that will generate data to answer the survey research study’s question.
4. More than 5 If the data collection tool is too long or confusing for a respondent to com-
plete in an acceptable timeframe or with little to no assistance, it increases
the likelihood that the survey will never be completed and returned.
METHODS: SURVEY RESEARCH

As presented in previous chapters, the research methods are sampling, data collection, and
data analysis. The methods a researcher selects must align with the research design of the
study. This section of the chapter will include a brief overview of sampling and data collec-
tion methods as they pertain to the survey research design. For a more detailed discussion
of quantitative research methods, refer back to Chapter 5.
How Does a Researcher Sample in Survey Research?

Though it lacks the rigor of experimental designs, survey research has the ability to have a
high level of external validity if it uses one of the probability sampling methods. Therefore,
using probability sampling methods will often produce results that are most representative
of a sample to the population. Of course, if the researcher uses one of the non-probability
sampling methods, the generalizability of the study’s results will be weakened.
In survey research, the term population is used to define “a set of elements; an element is
defined as the basic unit that comprises the population” (Frankel, 1983, p. 23). For example,
while we may think of the term population as defining all persons living within a demographic
area, when using the term population in research sampling, it can be more narrowly defined
through the use of elements. A population may consist of all males over the age of 25 living
in the Northeast who attended a four-year college or university, or all medical clinics in the

Sciences
state of Washington that administered the H1N1 virus vaccination from September 2011 to
March 2014. When determining elements that will define a population, it is important to
be as concise as possible.
As we discussed in Chapter 5, many factors go into determining the sample size for a
research project, including significance level, confidence levels, and type I and type II errors.
Response rate is another issue the researcher needs to take into account when sampling.
The response rate—the percentage of people who actually take part in a survey and return
their responses—often tends to be much lower than the sample size. It is not uncommon
for the researcher conducting this type of research to continue to sample until the optimal
sample size has been reached. Some issues that may influence response rate are:
• Length and topic of survey – A longer, more complex survey may result in partici-
pants losing interest and not returning a completed survey. In addition, response
rates will be higher from participants who are interested in the survey topic.
• Relationship between researcher and participants – A professor who surveys stu-
dents in a classroom may have a larger participation rate than a researcher who
randomly dials numbers for a phone interview.
• How data is being collected – For instance, phone, face to face, Internet, mailed
survey, all directly impact response rates. Face-to-face interviews often result in
a higher response rate than online or e-mail surveys. There are many factors that
may influence the varying percentage of response rates based on survey type. It
may be easier to distribute a survey to a large sample through an online service
like SurveyMonkey® or qualtrics®, but the passive receipt of a link to fill out a
survey may result in less incentive to actively participate than will answering the
door and talking face to face with a live person.
• Incentives – Giving incentives for returning a completed survey most often
increases response rates. It is important to make sure the incentive is not so large
that it impacts the respondent’s choice to participate (undue influence).
• Life – Many times members of a sample have the best of intentions to return a
survey, but life gets in the way and causes them to forget, or get distracted, or
miss a due date. Often, researchers will send out reminders to those who are in
the sample in hopes of increasing the response rate.
An important question to ask when determining how to sample in survey research is,
Can the population be enumerated? Merriam-Webster gives the formal definition of enu-
merate as “to ascertain the number of” (www.merriam-webster.com). In other words, can
the researcher establish the number of units in the population to be sampled? In some
cases the answer is yes; in others, this task becomes much more difficult. For example, if
a researcher is looking to draw a representative sample from the general population and
decides to access that population through voter registration logs or DMV records of people
who hold a driver’s license or nondriver identification, the researcher can access a list of
people from which to draw a representative sample. Identifying information from these
lists would make it easy to contact a sample through the mail, by phone or the Internet, or
even in person. This population can be enumerated in the sense that the researcher knows
the total population from which a representative sample will be drawn at the start of the

Sciences
research. From this point, the researcher can establish how large a representative sample
will be needed to ensure generalizable results (see Chapter 5).
However, if the researcher is looking to collect data from homeless persons, a list of people
from which to sample most likely does not exist, and if one does exist, it most likely is not
comprehensive and does not contain accurate contact information. This would make it much
more difficult to enumerate the population and determine how large a sample is needed in
order to ensure results that can be generalized from the sample to the larger population.
When looking to sample through e-mail or snail mail, it is important to have the most
current addresses available in order to reach the maximum number of potential respon-
dents. It is important to consider timing when collecting data as well. If the researcher was
attempting to survey heads of households on a topic, calling residential phone numbers
during typical work hours would result in a very low response rate. Likewise, knocking on
homeowners’ doors during the dinner hours may guarantee more people will be home, but
they may not be open to talking to the researcher if their dinner was just interrupted. In
addition, sending an e-mailed survey during a holiday season when people’s mailboxes tend
to fill up with advertisements and junk mail may result in a larger portion of the sample
deleting the link or putting it aside and forgetting about it rather than filling it out as they
might do if it was received during a less busy time.
How Is Data Analyzed in Survey Research?

Survey research employs both descriptive and inferential data analysis methods. Survey
questions are written with data analysis in mind. That is, how the question is structured
(data scales) determines what type of data analysis can be conducted. How the data will
be analyzed should be planned out, tested, and finalized before the data collection phase
is implemented. While most questions are structured and therefore precoded (assigned
a numeric value for each possible response during the design phase), some unstructured
questions or response choices (i.e., Other ___) will require the researchers to postcode
this data by assigning numerical values to each written-in acceptable response (Alreck &
Settle, 2007, p. 241). As previously stated, unstructured or open-ended questions require
qualitative analysis methods.
Determining a survey instrument’s validity and reliability is very important and may
require numerous rewrites and testing of questions. Chapter 5 discussed validity and reli-
ability in detail; here is a brief overview related to the survey instruments.
Instrument Validity
• Content Validity: Do the questions express the underlying concept they were
designed to reflect?
• Criterion Validity: Do the responses to the questions agree with the gold standard
for the underlying concepts?
• Construct Validity: Are the hypotheses concerning the relationships between the
underlying concepts conveyed in the responses?

Sciences
Instrument Reliability
• Test-retest: Does the same question have the same response over time or with a
different sample?
• Interrater: Do two interviewers with the same questionnaire get the
same response?
• Internal Consistency: Are questions designed to evaluate the same concept/
obtain equivalent responses?
The process of ensuring instrument validity and reliability may be time consuming, but
it safeguards against collecting unusable data. Other threats to internal validity common
in survey research include self-selection, response bias, recall bias, interview distortion,
and false respondents. Most of these terms have been clearly defined in previous chapters
(5 and 8); only the new terms are defined below.
1. Interviewer distortion – The researcher is often a data collection tool in survey
research. This being said, it is always a possibility that an interviewer can distort the
responses of a survey by not asking questions that may make them uncomfortable
or questions that they believe they know the answer to, based on a respondent’s
previous answers. In addition, there is always the possibility that a researcher’s
subjectivity (personal opinions and beliefs) may accidentally play a part in the data
collection process.
• False respondents – One issue that commonly occurs in survey research is false
respondents. The researcher often does not know whether the individual sampled
completed the survey. For example, a survey is sent to a randomly selected group
of physicians; however, the researcher would not know if the returned survey was
completed by the physician or if the physician asked the office assistant to com-
plete the survey instead.

In this chapter students were introduced to the defining features of survey research design.
To summarize, survey research:
• can be conducted in a variety of ways, including interviews and questionnaires;

• allows researchers to collect data on intangible concepts such as attitudes,
beliefs, perceptions, values, and feelings for a wide array of topics;
• is unique in that it can be used to collect large amounts of data in a relatively
short amount of time;
• can be an economically viable way to conduct research with a large number
of participants;
• relies on well-constructed surveys to collect the data.

Sciences
TABLE 9.2. Survey Design: Purpose, Special Features, and Methods
Design Purpose Special Feature of the Design (SF)

Methods: sampling (S), data collection (DC), &
data analysis(DA) specific to the design
Survey Develop a quantitative understanding of a (S) Either probability or non-probability meth-

population on a specified issue: A population’s ods can be used
knowledge, attitudes and feelings, perception
and beliefs, and/or behaviors about the issue (DC) Uses surveys: Survey can be either in per-
son, over the phone, electronic, or mailed
■■ Writing good survey questions is an important part of the design phase

of survey research. It may be a time-consuming task, but the time spent
perfecting the questions pays off in increased response rates as well as
increased instrument validity and reliability.
STUDY PRACTICE
Study tips for all VARK styles: Since the reducing notes from 3:1 recommendation cuts across
all learning styles, the first suggestion for studying is to start making flash cards for every
bolded/italicized term. The important terms/concepts presented in this chapter include:
ɠɠ survey research design ɠɠ filter/screening questions
ɠɠ survey ɠɠ double-barreled questions
ɠɠ standardization ɠɠ bias/loaded questions
ɠɠ focus, clarity, brevity ɠɠ sensitive questions
ɠɠ open ended (unstructured) ɠɠ question content/question wording
ɠɠ closed ended (structured) ɠɠ response rate
ɠɠ Likert ɠɠ enumerated
ɠɠ semantic differential ɠɠ interviewer distortion
ɠɠ Guttman ɠɠ false respondents
ɠɠ dichotomous

1. In an average week (Sunday-Saturday), how many days did you consume more than
5 alcoholic beverages over the course of an entire day?
a. 0
b. 1–2
c. 3–4
d. 5–7

Sciences
This survey question is an example of a(n):

a. bias/loaded question
b. interval question
c. correctly written survey question
d. filter question
2. Which of the following is a disadvantage of a structured question?
a. They are harder to tabulate and synthesize.
b. They cannot be precoded.
c. They allow the respondent to answer in his or her own words.
d. They take more time to construct.
3. The student government at a college is interested in determining how important inter-
collegiate sports are to the students there. Since all students have e-mail accounts,
they are able to send a survey to all the students at the college. About 30% of the
students respond.
Which is the most likely bias in this sample?
a. Bias in question wording that reflects student government’s feelings about sports.
b. Self-selection, or a higher rate of response by those who have strong feelings.
c. A bias from contacting only those whom student government could easily contact.
d. Response bias from leading questions.
4. In regard to survey questions, which of the choices below uses a cumulative
ranked scale, where the response statements are ranked from least extreme to
most extreme?
a. Likert
b. Guttman
c. Simple ordinal scale: Rank from best to worst
d. Filter question
5. I believe the federal government should pass a law legalizing physician-assisted sui-
cide for terminally ill patients and that nurse practitioners and physician assistants
should be allowed to help terminally ill patients die just like a physician.
This survey question is an example of a:
a. double-barreled question
b. sensitive question
c. correctly written survey question
d. filter question
The survey asks you to rank order a group of four statements, from the one you agree
with the most to the one you agree with the least. The following are the choices.
I am unhappy with my current health care provider.
I think abortion is morally wrong.
I think marijuana should be legal.
I am unhappy with the current political climate in this country.

Sciences
6. Select the answer that best describes what type of survey question this is.
a. Guttman scale
b. Biased rank ordered
c. Correctly written rank ordered
d. Likert scale
I believe that the influenza vaccination causes you to get the flu.
1 2 3 4 5
Strongly Agree Neutral Disagree Strongly
Agree Disagree
7. What type of question is the example above?
a. Likert-type scale
b. Semantic differential scale
c. Guttman
d. Contingency
8. The survey question in the above example is a:
a. double-barreled question
b. biased question
c. correctly written question
d. filter question
9. Which of the following is not one of the three major characteristics that most
surveys possess?
a. Information is collected from a group of people in order to describe some aspect
or characteristic of the population of which that group is a part.
b. The main way in which the information is collected is through asking questions;
the answers to these questions by the members of the group constitute the data
of the study.
c. The questions on the surveys must be very complex in order to get accurate data.
d. Information is collected from a sample rather than from every member of
the population.
10. The purpose of this study is to gain a generalizable understanding of the attitudes
and behaviors of preteens and early teens with regard to physical activity, nutritional
education, and participation in the school lunch program. The researcher randomly
selects 500 students from grades 5 through 8 from a list of all enrolled students in
the New York City public school system to fill out a questionnaire.
Survey in this example is?
a. The data collection tool only.
b. Both the research design and the data collection tool.
c. The research design only.
d. None of the above.

Sciences
STUDY ACTIVITIES
At this point, you should be able to begin writing practice survey questions on your own.
Using the following research purpose statement by Silliman, Rodas-Fortier, and Neyman
(2004), write survey questions in the formats listed below.
A Survey of Dietary and Exercise Habits and Perceived Barriers to Following

a Healthy Lifestyle in a College Population
“The purpose of this survey was to assess the diet and exercise habits and perceived
barriers to following a healthy lifestyle of college students and to determine if dif-
ferences exist by gender” (p. 10).
Hint: Before you begin to write questions for your survey (data collection tool), research
the healthy lifestyle recommendations for the amount/frequency of physical activity (https://
health.gov/paguidelines/) and for dietary choices required for optimal health (https://www.
choosemyplate.gov/vegetables).
1. Write five survey questions that directly relate to your research question, one in each
of the following categories:
a. Dichotomous
b. Likert scale
c. Filter/screening
d. Nominal question
e. Ordinal question
REFERENCES
Allen, E. I., & Seaman, C. A. (2007, July). Statistics roundtable: Likert scales and data analyses.
Quality Progress, 64–65. Retrieved from http://asq.org/quality-progress/2007/07/ statistics/likert-
scales-and-data-analyses.html
Alreck, P. L., & Settle, R. B. (1995). The survey research handbook (2nd ed.). Boston, MA: Irwin
McGraw-Hill.
Alvarelhão, J., & Lopes, D. (2016). A Guttman scale to assess knowledge about sexually transmit-
ted diseases in adults with cerebral palsy. Sexuality and Disability 34(4), 485–493. doi:10.1007/
s11195-016-9454-8
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd
ed.). Thousand Oaks, CA: SAGE Publications.
Dawes, J. (2008). Do data characteristics change according to the number of scale points used? An
experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research,
50(1), 61–78.
Fowler, F. J. (2014). Survey research methods (5th ed.). Thousand Oaks, CA: SAGE Publications.
Frankel, M. (1983). Sampling theory. In P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook
of survey research (pp. 21–66). New York, NY: Academic Press.
Fujimori, M., Shirai, Y., Asai, M., Kubota, K., Katsumata, N., & Uchitomi, Y. (2014). Effect of commu-
nication skills training program for oncologists based on patient preferences for communication

Sciences
when receiving bad news: A randomized controlled trial. Journal of Clinical Oncology, 32(20),
2166–2172. doi:10.1200/JCO.2013.51.2756.
Human Rights Campaign. (2018). Collecting transgender-inclusive gender data in workplace and other surveys.
Retrieved from https://www.hrc.org/resources/collecting-transgender-inclusive-gender-data-in-work-
place-and-other-surveys
Jacobsen, K. J. (2017). Health research methods. A practical guide. Burlington, MA: Jones & Bartlett
Learning.
Likert, R. (1932). A technique for the measurement of attitudes. (Doctoral dissertation). Retrieved from
ProQuest Dissertations & Theses Global. (Order No. 0130969).
Mellor, D., & Moore, K. A. (2014). The use of Likert scales with children. Journal of Pediatric Psychology,
39(3) 369–379. doi:10.1093/jpepsy/jst079
Owens, L. K. (2002). Introduction to survey research design [Presentation slides]. Retrieved from https://
www.researchgate.net/publication/253282490_INTRODUCTION_TO_SURVEY_RESEARCH_
DESIGN
Page-Bucci, H. (2003, February). The value of Likert scale in measuring attitudes of online learners.
Retrieved from http://www.hkadesigns.co.uk/websites/msc/reme/likert.htm
Park, H., & Shin, S. (2015). Development and psychometric testing of a semantic differential scale
of sexual attitude for the older person. Nursing and Health Sciences, 17(4), 526–532. doi:10.1111/
nhs12230
Robson, C. (1993). Real world research: A resource for social scientists and practitioner-researchers. Oxford,
UK: Blackwell.
Silliman, K., Rodas-Fortier, K., & Neyman, M. (2004). A survey of dietary and exercise habits and
perceived barriers to following a healthy lifestyle in a college population. California Journal of
Health Promotion, 2(2), 10–19.
SAGE Publications.
White, C. B. (1982). A scale for the assessment of attitude and knowledge regarding sexuality in the
aged. Archives of Sexual Behavior, 11(6), 491–502.

Sciences
UNDERSTANDING MIXED
METHODS RESEARCH: MIXED
METHODS DESIGNS
INTRODUCTION
10
CHAPTER GOALS:
Previous chapters have provided a detailed overview of qualitative and THE GOALS OF THIS CHAPTER ARE:
quantitative methodologies, qualitative and quantitative designs, and
appropriate methods used in quantitative and qualitative research. The • To provide an understanding
purpose of this chapter is to explain how and why health science research- of the term mixed methods as a
ers combine and integrate qualitative and quantitative approaches to research methodology.
conduct research. This integration is called mixed methods research. • To explore three of the most com-
The term mixed methods research denotes a research methodology, monly utilized mixed methods
and this chapter will explore three of the numerous research designs designs in health science research.
that fall under this methodology: convergent, explanatory, and explor- • To introduce the divergent purposes
atory. Prior to studying mixed methods research and mixed methods and practices across the convergent,
research designs, one must not only have a firm understanding of both explanatory, and exploratory designs.
qualitative and quantitative research methodologies, but also a strong • To provide an understanding of
understanding of the strengths and limitations of each qualitative and the strengths and challenges of
quantitative research design/methods. The researcher must undertake using mixed methods in health
the action steps in the design stage of research twice (Table 10.1). When science research.
conducting mixed methods research, the researcher plans both a quali-
tative and quantitative study.
As a methodology, mixed methods research formally emerged approx-
imately 40 years ago, yielding a new approach to conducting research LEARNING OBJECTIVES:
(McKim, 2017; Teddlie & Tashakkori, 2009). Historically, mixed meth-
ods research has been used almost exclusively in the social sciences, BY THE COMPLETION OF THIS CHAPTER, THE
but within the past 20 years it has begun to be used more widely in STUDENT WILL BE ABLE TO:
the health sciences. Guetterman, Fetters, and Creswell (2015) provide • Compare the purpose and structure
a clear explanation of the relevance of this methodology in the field of of three mixed methods designs;
health science. • Compare the strengths and challenges
inherent in mixed methods research;
Mixed methods research increasingly is being used as a methodology in
the health sciences to gain a more complete understanding of issues and • Identify the most commonly used
hear the voices of participants. Researchers have used the mixed meth- mixed methods design, qualitative,
ods approach to examine nuanced topics, such as electronic personal and quantitative research designs
health records, knowledge resources, patient physician communication, used in health science research;
and insight about intervention feasibility and implementation practices. • Select a mixed methods research
Mixed methods research is the collection and analysis of both qualitative design that aligns with the purpose
and quantitative data and its integration, drawing on the strengths of both of the research study; and
approaches (p. 554). • Identify the appropriate qualitative
and quantitative research design and
the associated methods best suited
209 to the mixed methods design.

Sciences
Stages Action Steps
Conceptual (Thinking) Having an idea (research problem), systematically reviewing the literature to verify the problem has
the potential to generate new knowledge, writing a problem statement, a research purpose state-
ment that explicitly states the purpose of mixing methods, and research questions
Design (Planning Selecting the best qualitative research design and research methods (sampling, data collection,
of two studies linked by data analysis) that align with research question(s)
a research purpose)
Selecting the best quantitative research design and research methods (sampling, data collec-
tion, data analysis) that align with research question(s)
Submitting the research study proposal for IRB review
MIXED METHODS RESEARCH—DEFINED

Mixed methods research utilizes rigorous and systematic investigations that intentionally
combine qualitative and quantitative methodologies into one single or multiphase study
(Teddlie & Tashakkori, 2009).
Best Practices for Mixed Methods Research in the Health Sciences is a guide developed for the
National Institutes of Health’s Office of Behavioral and Social Sciences Research by Cre-
swell, Klassen, Plano Clark, and Smith (2011). It includes some important recommendations
regarding the circumstances under which a researcher would use mixed methods in a study.
Mixed methods research typically involves a research purpose that is multilayered and is
exploring a real-life issue that is often rooted in social constructs (e.g., cultural experiences,
normative beliefs, oppression, geographical issues, etc.). In mixed methods research, the
focus of the “quantitative research [is on] assessing magnitude and frequency of constructs
and … [the focus of the] qualitative research [is on] exploring the meaning and understanding
of constructs” (Creswell et al., 2011, p. 4).
If the goal of mixed methods research is to develop a deeper, richer more fully developed
understanding of the research topic, then the conceptualization and design phase of a mixed
methods research study must include the development of a research purpose and question(s)
that require the integration of two studies. This is a critically important point: if the research
question can be answered by either conducting a qualitative or quantitative study, it does
not warrant the use of mixed methods. For example, simply adding one or two open-ended
questions to the end of a survey does not make that study a mixed methods study. Mixed
methods research (as the definition implies) needs a complex research question(s) that cannot
be answered by using one methodology (qualitative or quantitative) alone (McKim, 2017).
Consider the following example in which the purpose of the study requires mixed methods:
Wittink, Barg, and Gallo’s (2006) study focused on patients’ discussion of depression symp-
toms with a primary care provider. The researchers chose a mixed methods design so they could
simultaneously test a hypothesis (quantitative phase of the study) and generate a new hypothesis

Sciences
Chapter 10 Understanding Mixed Methods Research: Mixed Methods Designs | 211
(qualitative phase of the study). This combined focus of the study allowed the authors to conclude
that “physicians may signal to patients, wittingly or unwittingly, how emotional problems will be
addressed, influencing how patients perceive their interactions with physicians regarding emo-
tional problems” (p. 302).
Without a clear purpose and rationale for linking the two studies, the researcher is just
conducting two separate and unrelated studies or is simply collecting minimal qualitative
data in a quantitative study. The next section of this chapter provides a more detailed dis-
cussion of mixed methods designs, research purpose statements, research questions, and
factors to consider when planning/designing a mixed methods study.
MIXED METHODS RESEARCH—DESIGNS

This chapter will explore three of the most common mixed methods designs used in health
science research. The relationship, importance, and purpose of each mixed methods design
have been summarized in Table 10.2 (Creswell et al., 2011; Halcomb & Andrew, 2009; Onwueg-
buzie & Leech, 2006; Tashakkori & Teddlie, 1998; Teddlie & Tashakkori, 2009; Wisdom &
Creswell, 2013).
TABLE 10.2 Mixed Methods Designs: Relationship, Importance, and Purpose
Design How (structure) Why (purpose)
Convergent Relationship: Quantitative(QUAN) and To merge the analysis of the two data sets
qualitative (QUAL) research studies (verbal/image and numerical) for comparison
conducted at same time when looking for:
ɓɓ areas of agreement and areas of disagreement;
Importance: One study is not more important ɓɓ how do the results/finding relate to one another; and
than the other; QUAN and QUAL are equal ɓɓ how do the merged results offer deeper insight into
the research topic.
Explanatory Relationship: QUAN then QUAL To better understand an interesting, unusual, or key
QUAN result or variable, a follow-up QUAL study is
Importance: QUAN has the most weight conducted.
Exploratory Relationship: QUAL then QUAN QUAL findings are transformed (i.e., into variables,
survey questions, key points in an intervention, etc.) so
the researcher can develop a QUAN tool, instrument, or
Importance: QUAL has the most weight
intervention.
Convergent Design
This design has also been referred to as the parallel or concurrent design, since both stud-
ies can be done simultaneously and the timing of the data collection for each study is not
dependent on the results of the other study. The primary reason a researcher would select

Sciences
this design is to see how the results of a quantitative study and the findings of a qualitative
study, when merged for comparison, can enhance the understanding of the research topic.
For example, Andrew, Salamonson, Everett, Halcomb, and Davidson’s (2011) mixed
methods study merged the results from a patient satisfaction survey with the findings
from in-depth patient interviews. The researchers found that “despite scoring all of
the quantitative items highly, the qualitative responses indicated some dissatisfac-
tion with areas of the participants’ care” (p. 60). In this study, the results/findings
of the two studies did not agree. However, by merging the analysis for comparison,
the researchers were able to develop several conclusions that more fully explained
the phenomenon. These conclusions included, but were not limited to: the survey
had a ceiling effect (meaning the survey was not sensitive enough to capture the
nuanced experiences of patients living with chronic illness) and patients who require
long-term care at a specific health care facility might be hesitant to give negative
scores that reflect their actual experiences.
Explanatory Design
This is a sequential design where the study is conducted in two phases. The results of the
first study (quantitative) are directly related to why the second study (qualitative) is con-
ducted. The primary reason a researcher would select this design is to gain an in-depth
understanding of an interesting, unusual, or key quantitative result by following up with
a qualitative study. The qualitative findings provide further explanation or clarification of
the quantitative results. The examples below illustrate this concept.
1. Fox et al. (2013) studied the use of complementary and alternative medicine (CAM) by
surveying 406 women with breast cancer living in Ireland. The results of the survey
revealed that over half of the women used some form of CAM. Then researchers also
interviewed 20 oncology health care workers, 20 CAM providers, and a small group
of women living with breast cancer to explore one result from the survey. The quali-
tative study was centered on understanding the patients’ decision-making processes
when selecting a CAM treatment. The researchers found that the oncology staff had
a very influential role in the patients’ selection of specific CAM treatments (p. 495).
2. Mayoh, Bond, and Todres (2012) conducted a study of online health information
seeking behaviors of adults with chronic illness. The authors shared how the anal-
ysis of the first phase of the study (quantitative data analysis) determined the type
of data that would be needed in the second (qualitative) phase of the study. This
type of data collection strategy is very appropriate to an explanatory sequential
mixed methods design, where the purpose of the study was to first develop a general
understanding of frequency and scope (e.g., type of health information data, how
the data influences health decisions, etc.). The follow-up qualitative data collection
could then target certain quantitative results for further exploration. In this specific
study, the qualitative data collection centered on exploring six key results from the

Sciences
quantitative phase of the study in more depth, thus revealing a more fully developed
explanation of the research topic.
Exploratory Design
This is a sequential design where the study is conducted in two phases, and the results of
the first study (qualitative) are directly used to develop a tool, instrument, or intervention
for the second phase of the study (quantitative). The primary reason a researcher would
select this design is to explore an issue qualitatively in order to develop a better or more
accurate quantitative instrument or intervention. First, the researcher would conduct the
qualitative study. Then the findings of the study would be transformed (e.g., into variables,
survey questions, key points in an intervention, etc.) so the researcher can develop and
disseminate the quantitative instrument (i.e., survey) or intervention.
For example, Coyle and Williams (2000) first conducted a grounded theory study
to develop a conceptual framework of patients’ dissatisfaction with health care
experiences. Themes within this framework ultimately assisted in the develop-
ment of questions for a patient satisfaction survey, one that “would be sensitive to
patients’ subjective experiences of health care because it is grounded in people’s
actual accounts” (p. 1238).
Review Figure 10.1: This figure summarizes steps involved in determining the order,
importance, and relationship between the two studies.
FIGURE 10.1 An illustration of the planning stage when conducting mixed methods research.
Mixed Methods Research
Convergent, Explanatory,
or Exploratory
The mixed methods design

The purpose of study and the reveals the order,
importance, and relationship How the research question(s) is written
research question(s) must include the
between the informs which QUAN and QUAL
rationale for combining the two
QUAL/QUAN or designs/methods will be utlized in the
studies. This leads to selection of a
QUAN/QUAL studies. mixed methods study.
mixed methods research design.
Requires the design of two

studies within a larger single
or multi-phased study.
Researcher selects a QUAN research Researcher selects a QUAL research
design and QUAN methods: sampling, design and QUAL methods: sampling,
data collection, and data analysis. data collection, and data analysis.

Sciences
MIXED METHODS RESEARCH—PURPOSE

STATEMENT AND RESEARCH QUESTIONS
The mixed methods researcher follows the same five stages (conceptualization, design,
implementation, analysis, dissemination) as a researcher conducting a solo qualitative or
quantitative study. During the conceptualization stage, the mixed methods researcher must
review the literature, verify the problem has the potential to generate new knowledge, and
write a problem statement, purpose statement, and research question(s). This section simply
highlights variations in how mixed methods purpose statement and research questions are
written so the rationale for integration is clear.
Collins and O’Cathain’s (2009) article explained that researchers can take different
approaches to developing mixed methods purpose statement and research questions. For
example, Teddlie and Tashakkori (2009) recommend that the researcher develop one overar-
ching question for the study with sub-questions that delineate the qualitative or quantitative
components of the study. It is in these sub-questions that they reveal which qualitative
research design and which quantitative research design are best suited for the study.
Another way of approaching mixed methods purpose statement and research questions
are found in Chapter 4. It becomes clear that the rationale for integrating the studies can
be found at the end of the purpose statement. For example, “the reason for combining both
quantitative and qualitative data is to better understand this research problem by converging
both quantitative (broad numeric trends) and qualitative (detailed views) data” (Creswell,
2015, pp. 122–123). The rationale is clearly explained and aligns with the convergent mixed
methods design; the two studies will be conducted simultaneously so that the data analysis
can be merged for comparison. Another purpose statement example is, “the rationale for
using both qualitative and quantitative data was that a useful survey of student experience
could best be developed only after a preliminary exploration of student use” (Creswell, 2015,
pp. 122–123). It is clear from this purpose statement that the exploratory mixed methods
design would be used in this study. The study would be conducted sequentially, since the
qualitative study needs to be conducted first so that the qualitative data can be used to
develop quantitative questions for the survey. Creswell then proposes that once the purpose
statement has been written, that two separate research questions can be written; one for
the qualitative and one for the quantitative components of the study, with an overarching
summary research question that unites the two (Collins & O’Cathain, 2009).
MIXED METHODS RESEARCH—SELECTING THE QUALITATIVE

AND QUANTITATIVE DESIGNS AND METHODS
The mixed methods researcher follows the same design/planning stages as if they were
conducting a solo study. The challenge that the mixed methods researcher faces is selecting
the best research designs and methods (sampling, data collection, data analysis) to align
with the purpose of the mixed methods design. Said another way, the design phase of mixed
methods research includes planning two separate studies that are linked together by the
mixed methods design. The mixed methods design reveals the relationship and importance
of the qualitative and quantitative studies and the purpose for conducting the mixed methods
research. Creswell et al. (2011) explain there are some common qualitative and quantitative
designs used in health science mixed methods research.

Sciences
The qualitative designs that are most often used are “case studies, grounded theory, ethnography,
and phenomenology” (p. 4), and the quantitative designs that are most often used are “descriptive
surveys, observational studies, case-control studies, and randomized controlled trials” (p. 5).
Onwuegbuzie and Leech (2006) recommend that the purpose of the mixed methods deigns
can be revealed in how the research question is written. For example, “What side effects do
the new medication for depression cause among adolescents?” (p. 485). This question provides
information on the rationale for conducting the mixed methods study (design-convergent)
and provides sufficient information as to which qualitative and quantitative research designs
should be used. This mixed methods study would have to use one of the experimental designs
(to test the new drug) blended with a qualitative study (most likely a phenomenological
design) to understand the lived experiences of the adolescents taking the drug.
Sampling
The mixed methods researcher follows sampling guidelines presented in previous chapters.
For example, experimental designs require probability sampling methods, quasi-experimental
designs require non-probability sampling methods, qualitative designs require the use of
non-probability sampling methods, and non-experimental designs can use either probability
or non-probability sampling methods. Samples are typically large in quantitative studies so
the study has statistical power/rigor, while the samples in qualitative studies are typically
small. These rules do not change when conducting mixed methods research. That being
said, the mixed methods researcher has some latitude on how and at what point in the study
sampling takes place (Teddlie & Tashakkori, 2009). Based on the mixed methods design, the
sampling might occur prior to data collection or at the same time for both the quantitative
and qualitative studies; sampling for one of the studies might precede the other study, or
the results of one study could inform how sampling occurs in the other study.
Collins, Onwuegbuzie, and Jiao (2007) developed 24 mixed methods sampling strate-
gies that build on and expand the sampling methods presented earlier in Chapter 5. One
example of their mixed methods sampling strategies is the “Multistage purposeful random,”
where researchers conduct sampling twice but in two different stages. In this strategy the
researchers would first use one of the probability sampling methods, and once they have their
random sample, they would then use purposive sampling from that random sample (p. 272).
This would be a very good sampling strategy to use in Onwuegbuzie and Leech’s (2006)
example “What side effects does the new medication for depression cause among adoles-
cents?” (p. 485). Since the study requires an experimental design, one of the probability
sampling methods would be most appropriate. The researchers would use a probability sam-
pling method and then simple random assignment of the participants to the experimental
and control groups. The second stage of sampling would include purposive sampling for
the phenomenology part of the study, purposefully selecting participants from both the
control group and experimental group. This is where the latitude of mixed methods research
comes into play; while qualitative researchers do not sample with the goal of comparison,
in this case the purpose of the convergent mixed methods design which merges data for
comparison requires it.
A more detailed review of the 23 remaining strategies is beyond the scope of this book;
these sampling strategies go beyond sampling of participants and include ways of sampling

Sciences
elements of the social situation (Teddlie & Tashakkori, 2009). This information is only pro-
vided to demonstrate that planning mixed methods studies requires expertise. In summary,
the mixed methods researcher is sampling twice, once for the qualitative part of the study
and once for the quantitative part of the study. The sampling methods must be related to
the purpose of the mixed methods design, be in service of answering the research ques-
tion(s), and be appropriate for the qualitative and quantitative designs being used in the
mixed methods study.
Data Collection and Data Analysis

Similar to the section on sampling, the same practices and procedures presented in early
chapters regarding qualitative and quantitative data collection and data analysis applies to
mixed methods research. Qualitative data collection involves the collection of verbal/nar-
rative/image data, while quantitative data collection involves the collection of data that can
be converted into numbers. However, when conducting mixed methods research, the data
collection plan should intentionally and purposefully collect quantitative and qualitative
data that when combined into one study reduces the limitations inherent of both types of
data (Creswell et al., 2011; Zoellner & Harris, 2017).
This section will highlight some of the minor variations in data collection and analysis
practices that occur in mixed methods research. Just as in sampling, data collection and data
analysis strategies are determined by the mixed methods design being used in the study.
In the convergent design, the qualitative and quantitative data collection and data analysis
are conducted independently; the two data sets come together when the researcher merges
them for comparison (Zoellner & Harris, 2017). This mixed methods analysis compares the data
looking for areas of agreement and disagreement. For example, as Hesse-Biber (2010) explains,
there can be “contradictions found between the quantitative and qualitative results” (p. 466).
During analysis the mixed methods researcher, using the convergent design, is not looking to
determine whether one set of results/findings are right and the other wrong. Instead, when
contradictory evidence is found, the mixed methods researcher’s goal is to find meaning, deeper
understanding, and explanations that clarify the contradictions (Zoellner & Harris, 2017).
In the sequential mixed methods designs, data collection and data analysis of the two
phases of the study are conducted one after the other. In the explanatory design, the quan-
titative data is collected and analyzed first. Segments of the quantitative analysis that need
further exploration are what determines the type of qualitative data and analysis required to
explore components of the quantitative results. In the exploratory design, the qualitative data
must be collected and analyzed first, since it is being used to develop a quantitative instrument.
MIXED METHODS RESEARCH—STRENGTHS AND CHALLENGES
Strengths
The strength of quantitative research lies in its ability to statistically prove, explain, predict,
test, describe, or compare. The strength of qualitative research lies in exploring why some-
thing happens. When the two methodologies are intentionally combined in mixed methods
research, the researcher is using the strengths of both methodologies in one study (Creswell
et al., 2011); this practice reduces limitations inherent in the use of any one design or any one

Sciences
research method (Zoellner & Harris, 2017). Furthermore, when the researcher intentionally
and carefully combines qualitative and quantitative methods (sampling, data collection,
and data analysis), the study’s rigor, validity, reliability, and trustworthiness of the results
are enhanced (Hesse-Biber, 2010). Mixed methods research also has “great flexibility and
is adaptable to many study designs, such as observational studies and randomized trials, to
elucidate more information than can be obtained in only quantitative research” (Wisdom &
Creswell, 2013, p. 3). Wisdom and Creswell (2013) explain an additional strength of mixed
methods research is that it allows the researcher to “give a voice to study participants and
ensure that study findings are grounded in participants’ experiences” (p. 3) which is vitally
important in health sciences research.
Challenges
The challenges of mixed methods research are expertise and time. This type of research is
often undertaken in teams with some members of the team experts in qualitative research,
other experts in quantitative research, and still other experts in the best practices of mixed
methods research.
Mixed methods research takes more time and resources throughout all five stages of
research (conceptualization, design, implementation, analysis, dissemination) than con-
ducting a solo qualitative or quantitative study (Wisdom & Creswell, 2013). For example,
in the implementation and analysis phases, all qualitative and quantitative data must be
collected and analyzed before the study goes through the final analysis phase related to the
purpose of mixing methods. This is especially true for studies where mixed methods design
requires the completion of the data collection and analysis in the first study prior to the data
collection and analysis of the second study (Onwuegbuzie & Leech, 2006).

This chapter provided an overview of the rationale for mixing methods, explored the three
mixed methods research designs most commonly used in health science research, and sum-
marized the strengths and challenges inherent in conducting this type of research study.
Mixed methods research is purposefully the final chapter of this book, since a researcher
who undertakes this type of research requires a strong understanding of both qualitative
and quantitative problem statements, purpose statements, research questions, research
designs, and research methods. Review Table 10.2 for a summary of purpose of each of the
mixed methods designs presented in this chapter.
STUDY PRACTICE
At this point, you should begin to see how previously presented information on research
designs (Chapters 6–9) can be integrated when conducting mixed methods research. Make
your final additions/amendments to your flash cards, outlines, and notes.
For each scenario below, answer the multiple-choice question that asks you to identify
the mixed methods design and then the study activity question that requires you to identify
the qualitative and quantitative designs. The multiple-choice questions assess your under-
standing of the information presented in this chapter. The study activity questions (which

Sciences
qualitative and quantitative designs were/are used in this study) are designed to assess your
understanding of the designs presented in Chapters 6–9. Said another way, the questions
assess your ability to put it all together.
Attempt to answer these questions without looking at your notes to see if you have
developed a working definition of each design/method and the skills required to apply those
concepts to a research problem.
ɠɠ Mixed methods research
ɝɝ Convergent
ɝɝ Explanatory
ɝɝ Exploratory

COMBINED WITH STUDY ACTIVITIES
The purpose of this study was to explore youths’ (ages 16–18) social norms for alcohol use.
To date there has been no previous study on this research topic found in the literature. This
study sought to develop a theory that would explain the age group’s social norms for underage
alcohol use. The researcher purposefully selected 20–50 teens to conduct in-depth interviews
with, read the teens’ text messages with permission, review images on their social media
sites, and observe the teens in social situations. The researcher analyzed the data to identify
factors so that the conceptual model could be developed. The researcher’s intention was to
transform key elements from the conceptual model into survey question categories to aid in
the development of a questionnaire. The researcher has a master list of all school districts on
Long Island and will randomly select five districts to send the survey to once it is complete.
1. Which mixed methods design should be used in this study?

a. Convergent
b. Explanatory
c. Exploratory
Which qualitative and quantitative designs were/are used in this study?
Prior research has shown a correlation between high-intensity exercise and improve-
ments in the working memory of adults. It is well known that during adolescence the
brain is still developing and rates of physical activity have declined over the past decade.
To date no studies have focused on the role of high-intensity exercise and cognitive func-
tioning in adolescents. The researchers postulated that 12 weeks of high-intensity aerobic
activity four times a week would improve the cognitive functioning of sedentary middle
school–aged adolescents. The researchers asked teachers in the regional middle schools
to refer students and their parents who met the following criteria: students who did not
participate in sports and those who have been observed inside the classroom reading
or on a computer during recess as opposed to being outside on the playground. The
researchers interviewed 12 students about the meaning of exercise in their lives, and the

Sciences
researchers found three important themes (Exercise is not fun; I don’t like being outside;
I’m not as good as the other kids at sports) that helped the researchers tailor a 12-week
exercise program for sedentary adolescents. Adolescents who met the following criteria
were randomly selected from middle schools: those who engaged in less than 15 minutes
of aerobic exercise weekly and no medical history that would exclude the adolescent from
physical activity. Students were randomized to either the high-intensity exercise group
or a mild stretching group. The data was collected using the memory subtest from the
Wechsler Intelligence Scale for Children before the study and after the 12-week program
had ended. Scenario was adapted from Jeon and Ha’s (2017) research study.
a. Convergent
b. Explanatory
c. Exploratory
A researcher from a large health system explored the correlation of eight specific
variables to see if the presence of one or more of these variables would increase the like-
lihood of a medical error occurring. None of the identified variables were associated with
increased rates of medical errors; however, in the course of data analysis, the researcher
found one surgical team that had not had a single medical error reported in the past 2 years.
The researcher interviewed and observed the team and uncovered that 2 years ago the
team had developed a pre-surgical check list that they use before every surgery and that
they had begun intentionally using a unique set of error-reduction communication skills
during the surgical procedures. Using a theoretical framework from the patient safety
field, the researcher wrote an article explaining how the team made these changes in the
hopes that these changes could inform the practice of other surgical teams.
a. Convergent
b. Explanatory
c. Exploratory
Moseholm, Rydahl-Hansen, Lindhardt, and Fetters (2017) conducted a study to under-
stand the impact of patients seeking medical attention for evaluation of “symptoms possibly
attributable to cancer” (p. 993). The researchers measured changes in health-related qual-
ity of life (HRQoL) using a standardized HRQoL questionnaire in 680 patients, but the
researchers also selected 21 individuals to conduct in-depth interviews with so they could
merge the quantitative results and qualitative findings “with intent to obtain a more compre-
hensive understanding of their HRQoL experience during this stressful life event” (p. 993).
a. Convergent
b. Explanatory
c. Exploratory

Sciences
REFERENCES
Andrew, S., Salamonson, Y., Everett, B., Halcomb, E., & Davidson, P. M. (2011). Beyond the ceiling
effect: Using a mixed methods approach to measure patient satisfaction. International Journal of
Multiple Research Approaches, 5(1), 52–63.
Collins, K. M., & O’Cathain, A. (2009). Introduction: Ten points about mixed methods research
to be considered by the novice researcher. International Journal of Multiple Research Approaches,
3(1), 2–7.
Collins, K. M., Onwuegbuzie, A. J., & Jiao, Q. G. (2007). A mixed methods investigation of mixed
methods sampling designs in social and health science research. Journal of Mixed Methods Research,
1(3), 267–294.
Coyle, J., & Williams, B. (2000). An exploration of the epistemological intricacies of using qualitative
data to develop a quantitative measure of user views of health care. Journal of Advanced Nursing,
31(5), 1235–1243. doi:10.1046/j.1365-2648.2000.01381.x
Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and
qualitative research (5th ed.). Upper Saddle River, NJ: Pearson Education Inc.
Creswell, J. W., Klassen, A. C., Plano Clark, V. L., & Smith, K. C. (2011). Best practices for mixed meth-
ods research in the health sciences (for the Office of Behavioral and Social Sciences Research).
Retrieved from https://obssr.od.nih.gov/wp-content/uploads/2016/02/Best_Practices_for_Mixed_
Methods_Research.pdf
Fox, P., Butler, M., Coughlan, B., Murray, M., Boland, N., Hanan, T., … O’Sullivan, N. (2013). Using a
mixed methods research design to investigate complementary alternative medicine (CAM) use
among women with breast cancer in Ireland. European Journal of Oncology Nursing, 17(4), 490–497.
doi:10.1016/j.ejon.2012.10.008
Guetterman, T. C., Fetters, M. D., & Creswell, J. W. (2015). Integrating quantitative and qualitative
results in health science mixed methods research through joint displays. Annals of Family Medicine,
13(6), 554–561. doi:10.1370/afm.1865
Halcomb, E., & Andrew, S. (2009). Mixed methods research for nursing and the health sciences. Chich-
ester, UK: Wiley-Blackwell.
Hesse-Biber, S. (2010). Qualitative approaches to mixed methods research. Qualitative Inquiry, 16(6),
455–468.
Jeon, Y. K., & Ha, C. H. (2017). The effect of exercise intensity on brain derived neurotrophic factor
and memory in adolescents. Environmental Health and Preventive Medicine, 22(27), e1–e6. doi:10.1186/
s12199-017-0643-6
Mayoh, J., Bond, C. S., & Todres, L. (2012). An innovative mixed methods approach to studying the
online health information seeking experiences of adults with chronic health conditions. Journal
of Mixed Methods Research, 6(1), 21–33. doi:10.1177/1558689811416942
McKim, C. A. (2017). The value of mixed methods research: A mixed methods study. Journal of Mixed
Methods Research, 11(2), 202–222. doi:10.1177/1558689815607096
Moseholm, E., Rydahl-Hansen, S., Lindhardt, B. Ø., & Fetters, M. D. (2017). Health-related quality
of life in patients with serious non-specific symptoms undergoing evaluation for possible cancer
and their experience during the process: A mixed methods study. Quality of Life Research, 26(4),
993–1006.
Onwuegbuzie, A. J., & Leech, N. L. (2006). Linking research questions to mixed methods data anal-
ysis procedures. Qualitative Report, 11(3), 474–498. Retrieved from http://nsuworks.nova.edu/tqr/
vol11/iss3/3
Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative
approaches. Applied Social Research Methods Series, Volume 46. Thousand Oaks, CA: SAGE
Publications.

Sciences
SAGE Publications.
Wisdom, J., & Creswell, J. W. (2013, February). Mixed methods: Integrating quantitative and qualitative
data collection and analysis while studying patient-centered medical home models. Rockville, MD:
Agency for Healthcare Research and Quality. AHRQ Publication No. 13-0028-EF.
Wittink, M. N., Barg, F. K., & Gallo, J. J. (2006). Unwritten rules of talking to doctors about depres-
sion: Integrating qualitative and quantitative methods. Annals of Family Medicine, 4(4), 302–309.
doi:10.1370/afm.558.
Zoellner, J., & Harris, J. E. (2017). Mixed-methods research in nutrition and dietetics. Journal of the
Academy of Nutrition and Dietetics, 117, 683–697. doi:10.1016/j.jand.2017.01.018

Sciences
APPENDIX A
QUESTIONS AND ANSWERS
Question Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9
#1 G C D C D D A C C
#2 D D D A A C C D D
#3 B A B B B A D B B
#4 C B C B B B D B B
#5 D A B B B D A B A
#6 B D A B B C A A B
#7 A B D D A C A B A
#8 C B A A D B A A C
#9 A A A B A D C C C
#10 A B B B A A B D B
Question Chapter 10
#1 C; Grounded theory then Survey
#2 C; Phenomenological then Experimental post-test only
#3 B; Correlational then Case study
#4 A; Survey and Phenomenological
Chapter 1
• Quantitative only: compare two groups; impact; variables; sci-
entific method; predicting.
• Qualitative only: inductive reasoning.
• Common to both QUAN/QUAL: Scientific merit; sampling;
understand; research; paradigm; data collection; and rigor.
223

Sciences
Chapter 2
Case Study 1
Answer: This case study illustrates the violation of the Belmont Report principle of respect
for persons. The patient in this case, Dan Markingson, exhibited diminished autonomy as
exhibited by his involuntary commitment to a psychiatric ward after he made delusional
threats to his mother. Based on his diminished autonomy, Dan was not able to act as an
autonomous agent due to his mental disability and should not have been found to be com-
petent to participate in the trial.
Case Study 2
Answer: This case study illustrates the violation of the Belmont Report principle of respect
for persons and beneficence.
An example of showing a lack of respect to an autonomous person is by withholding infor-

mation for the purpose of interfering with the person’s ability to make a decision. In this
case, the consent form failed to reveal that there was a greater risk of dying in the low-oxygen
group, and a greater risk of severe eye damage in the high oxygen group. In addition, the
consent form did not explain that a participant whose clinical needs might have led doctors
to deliver a relatively high dose of oxygen, might have randomly been assigned to the group
that received a lower dose of oxygen.
Researchers also have an obligation to treat people in an ethical manner by demonstrating

beneficence, which is to maximize benefits and minimize the risk of harm to participants in
a research study. This principle was violated in this study due to the fact that there was a
greater risk of dying in the low oxygen group and a higher risk of severe eye damage in the
high oxygen group.
Chapter 4
Case 1
Problem Statement: Night shift nurses are critical to the delivery of health care. Despite
the fact that nurses on the night shift often express dissatisfaction with their work due to
a number of reasons (i.e., fatigue, absence of clinical leadership), there are few studies in
the literature that explore these issues.
Purpose Statement: The aim of this study is to understand how nurses in community hos-
pitals in the Northeast United States navigate the challenges of working the night shift.
Research Question: What is the theory that explains how nurses navigate the challenges
inherent in working the night shift in community hospitals in the Northeast United States?

Sciences
Appendix A | 225
Case 2
Problem Statement: Skin cancer is one of the most commonly diagnosed cancers in the
United States. The greatest risk for skin cancer is exposure to ultraviolet radiation (UVR)
and in order to reduce exposure to UVR, skin cancer organizations recommend the use of
sunscreen, sun protective clothing and limiting exposure to sun when UVR is at its highest
level. In spite of these recommendations, it has been documented that outdoor athletes
tend to not wear sunscreen because they are concerned it will interfere with their perfor-
mance and uniforms. To date, there is a deficiency in the literature regarding the efficacy
of educational interventions developed specifically for athletes.
Purpose Statement: The purpose of this study is to determine the impact of an educational
intervention targeting sun protection behavior change in NCAA student athletes enrolled
in universities in the northeast.
Research Question: What are the effects of an educational intervention on the use of sun
protection behaviors for NCAA student athletes enrolled in universities in the Northeast?
Chapter 5
First scenario
• Sampling
■■ Quantitative part of the study: one of the probability methods (but not
enough information is given to determine which one was used)
■■ Qualitative part of the study: purposive
• Data collection
■■ Quantitative part of the study: tool/instrument (survey)
■■ Qualitative part of the study: in-depth interviews
• Data analysis
■■ Quantitative part of the study: descriptive and inferential statistics
■■ Qualitative part of the study: coding looking for patterns/themes
• Rigor
■■ Quantitative part of the study: large sample size and the use of a data
collection tool that is known for its high scores on validity and reliability
measures.
■■ Qualitative part of the study: member checking was used to enhance the
credibility of the findings.
Second scenario
• Sampling: Convenience
• Data collection: Tools/instruments - WIQ, Pain Composite Scores from the
12-item short form, and a rating tool that assessed walking ability
• Data analysis: Inferential statistics (confidence level 95% and alpha level .05)

Sciences
• Rigor: The researchers used two data collection instruments’ that had high
validity and reliability scores. The walking assessment tools the researchers
created was tested for validity and reliability before it was used to collect
data. The use of a control group controlled for some threats to internal
validity.
Third scenario
• Sampling: Purposive and the researchers received permission to enter the area and
interact with participants (more detail on this concept will be found in Chapter 6).
• Data collection: Prolonged engagement and persistent observation (200 hours of
observations), interviews, and documents
• Data analysis: Nothing is mentioned…the scenario focuses on the sampling and
data collection only
• Rigor: Triangulation is being used to enhance the credibility of the study.
Chapter 8
• Observe.
1. Quantitative research design (observational design)
2. Quantitative data collection method (rating scale)
3. Qualitative data collection method (persistent observation)
4. Used in scientific notation: O means an observation (data collection point)
• Correlation.
1. Correlational research design
2. Correlational statistics (Pearson r)
• Prospective: Cohort design
• Longitudinal: Developmental design
Chapter 9
1. Write five survey questions that directly relate to your research question, one in each
of the following categories:
a. Dichotomous: Do you have a campus meal plan?
i. Yes
ii. No
b. Likert scale:
Breakfast is the most important meal of the day.
Strongly agree agree neutral disagree strongly disagree

1 2 3 4 5

Sciences
Appendix A | 227
c. Filter/screening: In an average week I exercise 30 minutes or more at least 4 times.

i. Yes
ii. No
d. Nominal question: What year of college are you in, are you a…
i. Freshmen
ii. Sophomore
iii. Junior
iv. Senior
e. Ordinal question: Rank the following exercises in order of preference from 1 to
5 (1 being that which you prefer to do most often):
i. Biking
ii. Walking
iii. Jogging
iv. Running
v. Swimming
vi. Weight- training
vii. Other (please specify)

Sciences
INDEX
convenience sampling, 101, 225

A
convergent design, 211, 216
abstract, 48, 59, 194
correlational design, 172, 173, 176
aim of the study, 66
credibility, 46, 60, 68, 95
alpha level, 64, 113, 114
criterion validity, 202
analysis/findings section, 67
critique of an article, 52, 53, 56
annotated bibliography, 52, 56, 72
current and relevant, 53, 60
applied/clinical research, 8
artifacts, 15, 117 D
assent, 35 data analysis, 7, 15, 16, 63, 64, 108–115
attrition/lost to follow-up, 94, 169 database, 32, 48, 50
audit trail, 96, 132 data collection, 7, 13, 105–108
autonomous person, 28, 224 data saturation, 102, 103
dependability, 94, 96, 105
B dependent variable, 77, 83, 104
background section, 65, 66 descriptive statistics, 63, 112, 153
basic research, 8, 9 design, 6, 12
Belmont Report, 28, 29, 30, 31 developmental design, 163
beneficence, 28, 29 dichotomous questions, 197
bias/loaded questions, 199 diminished autonomy, 28, 36, 224
directional hypothesis, 82
C discussion/conclusion section, 64, 68
case-control design, 170, 172, 173 double-barreled questions, 198
case study design, 127 double-blind, 93, 180
cited reference searching, 51
clinical significance, 115, 117 E
closed-ended (structured), 192 effect size, 103, 104
cluster sampling, 100 equivalent forms reliability, 107
coding, 108, 109 ethical research, 25, 28
coercion, 29, 34, 36 evidence-based practice, 2, 3
cohort design, 166, 169, 170, 172 exclusion criteria, 97, 98, 152
Common Rule, 30, 31, 32 exempt from review, 32
composition of the IRB board, 33 expedited review, 32
comprehension, 33, 95 explanatory design, 212, 216
confidence interval, 103, 104, 115 exploratory design, 213, 216
confidence level, 103, 104, 113 external validity, 67, 91, 97, 100
constructivism, 10, 11, 12
construct validity, 107, 202 F
content validity, 106, 202 false respondents, 203
control group time-series design, 147, 151 filter/screening questions, 197
229

Sciences
230 | Index
FINER, 78, 79 median, 112

focused ethnography design, 128 mediating variable, 84
full review, 32 member checking, 96, 109
methodology, 9, 10, 12
G
methods, 6, 7, 14
gap in the literature, 45, 53, 57
methods section, 62, 66
generalizable knowledge, 3, 25, 43
mixed methods, 12, 14
grounded theory design, 131, 132
mode, 112
Guttman scale, 196
moderating variable, 84
H N
hand searching, 51 National Research Act, 28, 36
Hawthorne effect, 94, 177 nominal scale, 110
history, 92, 146, 152 non-directional hypothesis, 82
human subjects, 4, 8, 13 null hypothesis, 82, 113, 114, 115
hypothesis, 13, 81, 82 numerical data, 12, 105, 108, 110
Nuremberg Code, 27
I
immersion, 108, 109 O
inclusion criteria, 97, 98 observational design, 165
independent variable, 83, 140, 141, 147 ordinal scale, 110, 111, 194
inferential statistics, 63, 111, 112, 113
information, 2, 4, 7, 33 P
informed consent, 31, 33, 34 paradigms, 9, 10
institutional review board (IRB), 8, 30, 31 persistent observation, 105, 127, 226
instrumentation, 93, 116 phenomenological design, 130, 215
internal consistency reliability, 107, 117 PICOT, 79, 80
internal validity, 67, 92, 95, 116 placebo-controlled, 93
interrater reliability, 107 placebo effect, 93
interviewer distortion, 203 popular journals, 47, 48
introduction section, 45, 59, 60, 65 population, 7, 13, 30, 96
population enumerated, 201
J positivism, 10, 12
Jewish Chronic Disease Hospital Study, 27 post-positivism, 10, 12
justice, 28, 30 posttest only control group design, 144
power analysis, 103, 105, 115
K pragmatism, 10, 11, 13
key word searches/ relevant term searches, 50 pretest-posttest control group design, 143
primary sources, 46, 47, 51
L private information, 4, 31
Likert scale, 194, 195 problem statement, 7, 9
literature review, 4, 44, 53, 54 professional journals, 47, 48
prolonged engagement, 105, 127, 134
M proportional stratified sampling, 99
maturation, 92, 164, 180 purpose statement, 7, 46, 54, 76
mean, 112 purposive sampling, 101, 102, 132, 215

Sciences
Index | 231
p value, 114 survey research design, 187, 188, 190, 191

systematic investigation, 3, 6, 14
Q
systematic literature review, 44
qualitative, 12, 13, 16
systematic random sampling, 99
qualitative methodology, 12, 14
quantitative, 10, 12, 13 T
quantitative methodology, 12 tertiary sources, 47
quasi-experimental designs, 133, 139 testing, 2, 12, 92
question content/question wording, 199 test-retest reliability, 107
quota sampling, 101 themes, 16, 52, 109
theoretical sampling, 102, 132, 133
R thick description, 95
range, 112 threats to internal validity, 92, 145, 180
reference mining, 50, 51 transferability, 68, 95, 105
reliability, 47, 62, 68, 91 triangulation of data, 95, 129
replicate the study, 62, 63, 66 trustworthiness, 67, 85, 91, 94
research, 1, 3 Tuskegee Study of Untreated Syphilis, 25, 30, 31
respect for persons, 28 type I errors, 103, 115, 140, 200
response rate, 200, 201, 202 type II errors, 103, 115, 140, 200
results section, 63, 64, 67 type of qualitative data, 105, 216
review of the literature scholarly article, 57
review section of a research article, 57 U
undue influence, 29, 34
S
sample size estimation, 103, 152 V
sampling, 7, 10, 16, 96–102 validity, 49, 91
scholarly journals, 31, 46, 47, 48, 51 voluntariness, 28, 33, 34
scientific literature, 4, 64 vulnerable population, 28
scientific merit, 7, 8, 46, 53
scientific method, 2, 3, 12 W
scientific principles, 1, 2 Willowbrook Hepatitis Study, 27
secondary sources, 46, 48, 60 within-subjects design, 147
selection bias, 94, 98, 100
semantic differential scale, 195
seminal research, 54
sensitive questions, 199
simple random sampling, 99, 215
single group time-series design, 150, 151, 152
snowball sampling, 102
Solomon four-group design, 145, 146
standard deviation, 112
standardization, 188
statistical regression, 93
statistical significance, 114
stratified random sampling, 99
survey, 32, 99, 177

Sciences

Han 251 Book

Uploaded by

Copyright:

Available Formats

Han 251 Book

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Han 251 Book

Uploaded by

Copyright:

Available Formats

What is the purpose and scope of the document?

What is the purpose and scope of the document?

What topics are covered in the document?

What topics are covered in the document?

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

Deborah Zelizer, Kathleen McGoldrick, and Deborah Firestone

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

Cover image copyright © 2017 iStockphoto LP/Filograph.

Printed in the United States of America.

ISBN: 978-1-5165-1866-1 (pbk) / 978-1-5165-1867-8 (br)

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

CHAPTER 1 Generating New Knowledge: Conceptualizing

CHAPTER 2 Ethical Issues When Conducting Research.......................................................................... 25

CHAPTER 3 Understanding the Role of Literature in the Research Process:

CHAPTER 4 Developing the Research Problem Statement, Purpose Statement,

CHAPTER 5 Selecting Methods to Increase Rigor: Planning a Study

CHAPTER 6 Qualitative Research: Qualitative Designs......................................................................... 125

CHAPTER 7 Understanding Quantitative Research: Experimental/

CHAPTER 8 Understanding Quantitative Research:

CHAPTER 9 Understanding Quantitative Research:

CHAPTER 10 Understanding Mixed Methods Research:

APPENDIX A ................................................................................................................................................................................ 223

INDEX ................................................................................................................................................................................................ 229

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

CHAPTER 1 Generating New Knowledge: Conceptualizing

CHAPTER 2 Ethical Issues When Conducting Research.......................................................................... 25

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

CHAPTER 3 Understanding the Role of Literature in the Research Process:

CHAPTER 4 Developing the Research Problem Statement, Purpose Statement,

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

CHAPTER 5 Selecting Methods to Increase Rigor: Planning a Study with Validity

CHAPTER 6 Qualitative Research: Qualitative Designs......................................................................... 125

CHAPTER 7 Understanding Quantitative Research: Experimental/

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

CHAPTER 8 Understanding Quantitative Research: Non-Experimental Designs........... 161

CHAPTER 9 Understanding Quantitative Research: Survey Research..................................... 187

CHAPTER 10 Understanding Mixed Methods Research: Mixed Methods Designs...... 209

APPENDIX A ................................................................................................................................................................................ 223

INDEX ................................................................................................................................................................................................ 229

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

TABLE I.1 Characteristics of the VARK Learning Styles

Characteristics of the learning styles/modes

Learning = textual based (e.g., books, handouts, notes, lists).

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

BY THE COMPLETION OF THIS CHAPTER,

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

health administration personnel;

SCIENTIFIC METHOD AND EVIDENCE-BASED PRACTICE—DEFINED

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

What Constitutes New Knowledge?

05/28/2020 - tp-cc6a6970-a120-11ea-be98-024 (temp temp) - Research Methods in the Health

• What are the health implications of smoking cigarettes on the lungs?

CHAPTER 1 Generating New Knowledge: Conceptualizing

CHAPTER 2 Ethical Issues When Conducting Research.......................................................................... 25

CHAPTER 3 Understanding the Role of Literature in the Research Process:

CHAPTER 4 Developing the Research Problem Statement, Purpose Statement,

CHAPTER 5 Selecting Methods to Increase Rigor: Planning a Study

CHAPTER 6 Qualitative Research: Qualitative Designs......................................................................... 125

CHAPTER 7 Understanding Quantitative Research: Experimental/

CHAPTER 8 Understanding Quantitative Research:

CHAPTER 9 Understanding Quantitative Research:

CHAPTER 10 Understanding Mixed Methods Research:

CHAPTER 1 Generating New Knowledge: Conceptualizing

CHAPTER 2 Ethical Issues When Conducting Research.......................................................................... 25

CHAPTER 3 Understanding the Role of Literature in the Research Process:

CHAPTER 4 Developing the Research Problem Statement, Purpose Statement,

CHAPTER 5 Selecting Methods to Increase Rigor: Planning a Study with Validity

CHAPTER 6 Qualitative Research: Qualitative Designs......................................................................... 125

CHAPTER 7 Understanding Quantitative Research: Experimental/

CHAPTER 8 Understanding Quantitative Research: Non-Experimental Designs........... 161

CHAPTER 9 Understanding Quantitative Research: Survey Research..................................... 187

CHAPTER 10 Understanding Mixed Methods Research: Mixed Methods Designs...... 209