American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

REVIEW
Developing Critical Thinking Skills in Pharmacy Students
Adam M. Persky, PhD,a,b Melissa S. Medina, EdD,c Ashley N. Castleberry, PharmD, MAEdd
a
Eshelman School of Pharmacy, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
b
Associate Editor, American Journal of Pharmaceutical Education, Arlington, Virginia
c
College of Pharmacy, University of Oklahoma Health Sciences Center, Oklahoma City, Oklahoma
d
College of Pharmacy, University of Arkansas for Medical Sciences, Little Rock, Arkansas
Submitted February 23, 2018; accepted May 21, 2018; published March 2019.

Objective. To review the importance of and barriers to critical thinking and provide evidence-based
recommendations to encourage development of these skills in pharmacy students.
Findings. Critical thinking (CT) is one of the most desired skills of a pharmacy graduate but there are
many challenges to students thinking critically including their own perceptions, poor metacognitive
skills, a fixed mindset, a non-automated skillset, heuristics, biases and the fact that thinking is effortful.
Though difficult, developing CT skills is not impossible. Research and practice suggest several factors
that can improve one’s thinking ability: a thoughtful learning environment, seeing or hearing what is
done to executive cognitive operations that students can emulate, and guidance and support of their
efforts until they can perform on their own.
Summary. Teaching CT requires coordination at the curricular level and further to the more discrete
level of a lesson and a course. Instructor training is imperative to this process since this intervention has
been found to be the most effective in developing CT skills.
Keywords: critical thinking, metacognition, course design, problem solving, clinical decision making

INTRODUCTION actions automatically).7,8 Interpretation includes under-

Critical Thinking (CT) is one of the most desired standing and communicating the meaning of information
skills of a pharmacy graduate because pharmacists need to others. Analysis includes connecting pieces of informa-
to think for themselves, question claims, use good judg- tion together to determine the intended meaning. Inference
ment, and make decisions.1,2 It is needed in almost every is recognizing elements of information one has and using
facet of pharmacy practice because pharmacy students those elements to reach reasonable conclusions or hypothe-
need to evaluate claims made in the literature, manage ses. Evaluation involves making a judgment about the cred-
and resolve patients’ medication problems, and assess ibility of a statement or information. Explanation includes
treatment outcomes.3 While pharmacy educators may adding clarity to information one shares so it can be fully
agree that CT is an essential skill for pharmacy students understood by another person. Self-regulation is the ability
to develop, it must be consistently defined because the to control one’s own thoughts, behavior and emotions.
definition determines how it is taught and assessed.4 While Besides the six core skills, CT is more than a stepwise
many definitions of CT exist,5 it is most commonly defined process. It is a summation of attitude, knowledge, and
as automatically questioning if the information presented knowledge of the CT process (Attitude 1 Knowledge 1
is factual, reliable, evidence-based, and unbiased.2 In sim- Thinking Skills 5 Critical Thinking).9 All three components
pler terms, it is reflecting on what to believe or do.6 are necessary. First, individuals need an attitude that aligns
To operationalize the CT definition, six core CT skills with CT. This attitude includes a willingness to plan, being
have been proposed: interpretation, analysis, evaluation, in- flexible, being persistent, willingness to self-correct, being
ference, explanation, and self-regulation (directing one’s mindful and a desire to reconcile information.9 If the atti-
tude is not there, it is unlikely that the individual will engage
Corresponding Author: Melissa S. Medina, Department of
in the actual process. Second, CT requires knowledge or
Pharmacy: Clinical and Administrative Sciences, College of something to think about. The more knowledge the individ-
Pharmacy, University of Oklahoma Health Sciences Center, ual has, the better their process and answer. Thus, acquiring
PO Box 26901, 1110 N. Stonewall, CPB 125, Oklahoma City, foundational, requisite knowledge is important in CT. The
OK 73190. Tel: 405-271-6484. Fax: 405-271-3830. E-mail: final part is the knowledge of the CT process. Knowing the
[email protected] steps and following them is key to success. Not following
161
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

the steps can lead to incorrect answers. Skipping steps is one ful. The first barrier is students’ perceptual problem –
of the barriers to CT. When these three components are students believe they know how to solve problems, so
present, CT can occur at a deep level. often, they do not understand why they are being re-taught
While CT is used often, it is important to differenti- this skill. Educators teach students how to monitor their
ate CT from other processes. Problem solving, clinical thinking and become better problem solvers by giving
reasoning and clinical decision-making are related them a framework to be more thoughtful thinkers.
higher-order CT skills and while the terms may be used The next challenge is students’ weak metacognitive
interchangeably, there are distinguishing features. Prob- skills. The relationship between CT and metacognitive skills
lem solving is a general skill that involves the application has been noted in the literature.15 Metacognition refers to an
of knowledge and skills to achieve certain goals. Problem individual’s ability to assess his/her own thinking and actual
solving can rely on CT but it does not have to.10,11 The level of skill or understanding in an area. Metacognition helps
steps of identifying a problem, defining the goals, explor- critical thinkers be more aware of and control their thinking
ing multiple solutions, anticipating outcomes and acting, processes.15 Students who are weak at metacognition jump to
looking at the effects, and learning from the experience conclusions without evaluating the evidence, thinking they
are all steps that can benefit from eliminating assumptions know the answer, which ultimately interferes with CT.
or guesses during the problem-solving process.12 In com- A third reason CT is difficult for students is that they
parison to general thinking skills, clinical reasoning and may have a fixed mindset or a belief that their intelligence
clinical decision-making depend on a CT mindset and are cannot change.16 If students believe CT is an innate skill-
domain-specific skills that are used within pharmacy and set that occurs naturally, they may not invest the effort to
other health sciences.4 Clinical reasoning is the ability to develop it because they believe that no matter how hard
consider if one’s evidence-based knowledge is relevant they try, they will never get it.
for a particular patient during the diagnosis, treatment, Heuristics can get in the way of CT. Heuristics are our
and management process.4,13 Clinical decision-making shortcuts to thinking – they are a strategy applied implicitly or
happens after the clinical reasoning process and is focused deliberately during decision-making where we use only part
on compiling data and constructing an argument for treat- of the information we might otherwise want or need. This
ment based on the interpretation of the facts/evidence results in decisions that are quicker and less effortful because
about the patient.14 Overall, the process of thinking like the individual may be using the best single piece of data to
an expert by considering the evidence and making correct make a more frugal approach.17-19 In a classic study, partic-
decisions about a patient to solve a patient’s problems is a ipants were asked, “If a ball and bat cost $1.10, and the bat is
skillset that students should practice so it becomes auto- $1 more than the ball, what was the cost of the ball?”20 The
matic. See Figure 1 for a visual representation. most popular answer is $0.10, which is incorrect (the correct
answer is the ball costs $0.05, the bat then is $1.05 or $1
Barriers to Critical Thinking more. If the ball was $0.10, the bat is only $0.90 more than
There are several challenges to students thinking the ball). We take cognitive shortcuts because thinking is
critically: perceptions, poor metacognitive skills, a fixed effortful and if we can get a quick response that fits our
mindset, heuristics, biases and because thinking is effort- current needs, we will do it. Kahneman referred to two
systems of thought: System 1 and System 2.19,21 System 1
is a fast decision-making system responsible for intuitive
decision-making based on emotions, vivid imagery, and as-
sociative memory. System 2 thought processes is a slow
system that observes System 1’s outputs, and intervenes
when “intuition” is insufficient.21
Another challenge that makes CT difficult for students is
their inherent biases. One major bias is confirmation bias or
the tendency to search for information in a way that confirms
our ideas or beliefs.22 Confirmation bias happens because of
an eagerness to arrive at a conclusion, so students may assume
Figure 1. Schematic of Critical Thinking and its Relationship they are questioning their assumptions when they are only
to Other Types of Thinking searching for enough information to confirm their beliefs.22
White boxes represent the thinking type while gray boxes When we want to think critically, we want the evidence
provide descriptions of each type and show how the skills against our view to better inform our decision. See Appendix
build upon each other 1 for a list of cognitive biases that may affect our thinking.
162
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

CT is difficult and does not develop automatically. It because it relies on discipline-specific knowledge. Re-
takes practice and effort. Experts think critically without search and practice suggest several factors that improve
conscious thought, which makes it effortless. However, thinking: a thoughtful learning environment (eg, integra-
developing expertise is estimated to take 10 years or tion), seeing or hearing what is actually done to executive
10,000 hours (or more) of deliberate practice, so it is a time cognitive operations one is trying to improve (eg, model
consuming activity.14,23 In a study of thinking using the behavior), guidance and support of one’s efforts until he
game Tetris, it was shown that initial game learning or she can perform on their own (eg, scaffolding);26 and
resulted in higher brain glucose consumption compared prompting to question what is thought to be known (eg,
to individuals with experience playing and those watching challenging assumptions).27 These are general, key points
someone play.24 Similar results are seen when comparing that instructors can do to help students develop CT skills.
experts to novices. Functional MRI studies show that ex- Creating a thoughtful learning environment is not
perts use less of their brain to solve a problem than novices, limited to just letting students make mistakes. Table 1
partly because a problem for a novice is not a problem for compares features of thoughtful classrooms to traditional
an expert.25 It is experience that has led to muscle memory classrooms that do not emphasize CT. The first piece of
and heuristics. Students do not have a lot of experience this thoughtful learning environment is helping students
thinking critically and therefore, do not want to do it be- to integrate their knowledge. Integration allows students
cause it is difficult and time consuming; they would rather to build on previous experiences, provide developmen-
do things that are automatic and effortless. tally appropriate opportunities for the individual to pro-
duce optimal performance, and lay a foundation for
Developing Critical Thinking Skills further development. By intentionally creating an envi-
Developing CT skills is difficult but not impossible. ronment that allows students to integrate previous and
CT is a teachable skill and is often discipline-specific current knowledge, they can begin to evaluate how the

Table 1. Major Features of Thoughtful and Traditional Classrooms26

Behavior or Condition Thoughtful Classroom Traditional Classroom
Student Ask and answer questions Answers questions
Remember and report information Remember, repeat and report information
Collect, analyze, evaluate and synthesize Collect and arrange information
information and ideas Reproduce information
Produce and construct knowledge Avoid thinking risks
Accept and take thinking risks Work individually
Work in groups and individually Engage in recitation and drill
Engage in sustained discussion, deliberation, Respond to the instructor
and inquiry
Interact with each other and the instructor

Instructor Guide student use of information Provide information to be learned

Seek clarification, elaboration, evaluation, Seek correct answers
and justification Direct and referee learning
Stimulate, encourage, facilitate, and support Model reporting, recording, remembering
learning Stand above students
Model complex thinking
Join with students in learning

Curriculum In-depth study of limited number of topics Superficial coverage of many topics
Incremental, conceptual, integrated learning Fragmentary, episodic, entity learning
Integrates learning with student experience Information learning an end of itself
Uses multiple sources of information Uses single source of information

Class Considerable student talk Considerable teacher talk

Considerable student-to-student interaction Limited to no student-to-student interaction
Builds knowledge as a group and individually Accumulate information individually
Requires considerable mental inquiry and effort Requires limited mental inquiry and effort

163
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

concepts are related and make decisions on how to apply point, it may be important to initiate the critical thinking
that knowledge to future, and likely different, situations. process by having students make predictions on outcomes
Integration can take many forms and does not necessarily and showing how their predictions may be correct or in-
mean courses need to be integrated or aligned in time. correct.
Integration can take the form of integrating the cumula- Developing CT requires a 4-step approach.9 The first
tive knowledge gained over the curriculum. step is explicitly learning the skills of CT. The second is
Modeling expert thinking is another way to help stu- developing the disposition for effortful thinking. The
dents see CT in action and begin to use these steps them- third step is directing the learner to activities to increase
selves. Instructors should verbalize their executive the probability of application and transfer of skills. The
cognitive operations for students to hear or see when final step is making the CT process visible by instructors
addressing a problem or issue that requires CT. No single making the metacognitive monitoring process explicit
step is too insignificant to point out. Learners are novice and overt. These four steps should be included both at
and assumptions should not be made that they understand the broad curricular level and down to the more discrete
or know how to perform a seemingly simple set in the level of a lesson and a course.
thinking process. By watching the experts process infor- Curriculum. College has shown to increase CT skills
mation, learners begin to form those thinking skills when CT is measured through standardized assessments
as well. of CT skills (four years of college 5 effect size of 0.6).31
Scaffolding is another general method that can facil- While part of this growth in college may be due to matu-
itate development of CT skills. Scaffolding is a temporary ration and increase in knowledge, developing CT skills
support mechanism. Students receive assistance early on requires curriculum-level coordination. Just like a mili-
to complete tasks, then as their proficiency increases, that tary action will fail if the individual units do not play their
support is gradually removed. In this way, the student role, CT development will fail if individual units do not
takes on more and more responsibility for his or her play their respective roles. One way to develop CT skills
own learning. To provide scaffolding, instructors should is to use a two-fold approach.1,32 The first step is to have a
provide clear directions and the purpose of the activity, course in the curriculum that teaches the general thinking
keep students on task, direct students to worthy sources, skill process and starts to develop the dispositions. The
and offer periodic assessments to clarify expectations. second step is to have individual courses reflect that pro-
This process helps to reduce uncertainty, surprise and cess within the context of the subject matter. Ideally
disappointment while creating momentum and efficiency courses have explicit learning objectives and make the
for the student. thinking process equally as explicit; this is called the in-
Thinking begins when our assumptions are violated. fusion method. Table 2 shows the effect sizes (difference
Driving to work requires little effort. We do it all the time in performance relative to the standard deviation) of these
and sometimes we may wonder how we got to work be- types of interventions. Typically effect sizes under 0.2 are
cause our thoughts were elsewhere. On a daily basis, you considered small, over 0.4 are considered educationally
assume your drive will be normal and unimpeded. Now significant, and over 0.7 are considered large.33,34 To
imagine there is traffic. You move from auto-pilot to note, these effect sizes come from a variety of study types,
thinking mode because your assumptions were violated. durations and outcome measures. For example, one study
When our assumptions are violated, we start to think and in nursing used a standardized assessment of CT (Califor-
we see this thought process as early as a few weeks from nia Critical Thinking Skills Test) to compare lecture to
birth.28 In the classroom, we must identify and challenge problem-based learning (PBL) in a pre/post design.35 Ex-
students’ assumptions. As an example from self-care in- amining pre-to-post changes, PBL showed an effect size
structors, when students are asked to recommend a prod- of 0.42 whereas lecture was 0.010. When comparing the
uct for cough associated with the common cold, any post-scores from PBL to lecture, the effect size was 0.44.
student pharmacist with community pharmacy experi- Alternatively, undergraduates were placed in dyads
ence may answer “dextromethorphan.” This may be what across four different conditions outlined in Table 2: gen-
they have seen in practice or what they received as a child eral, infusion, immersion and control.36 The outcome was
from their parents. They have experience in this context. a rubric developed by the instructor and research team.
However, this answer is not supported by the guide- Compared to control, the general (.46), infusion (1.1) and
lines,29,30 but the students will argue it is correct because immersion (.97) all showed positive and moderate-to-
of their experience. The cognitive dissonance – not expect- large effect sizes. Relatively, infusion was better than
ing something to happen that you thought would – starts the general (.60) as was immersion (.49) with very little dif-
cognitive thinking process. From an instructional stand- ference between infusion and immersion (.12). Although
164
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

Table 2. Effect Size and 95% Confidence Intervals for Types of Interventions to Develop Critical Thinking.1,32 (Effect sizes may
include: pre-post design, quasi experimental design, or true experimental design. Outcome measures may include standardized
critical thinking tests, instructor-developed critical thinking assessments, researcher-developed critical thinking assessments or
some combination thereof. Study durations range from short – 1 hour to 2 days – to greater than 1 semester.)
Intervention Definition Effect Size (g1)
General thinking skills CT abilities and dispositions taught separately from content .38 (.31, .45)
of existing subject matter.
Infusion Deep, thoughtful and well-understood subject matter instruction .54 (.49, .59)
and general CT principles are made explicit.
Immersion Subject matter instruction is thought provoking and students do .09 (.05, .13)
get immersed in the subject. However, general CT principles
are not made explicit.
Mixed General approach with either infusion or immersion: students .94 (.82, 1.05)
are involved in subject specific CT but also a separate thread
or course aimed at teaching general principles.
Authentic Instruction Effort to present students with genuine problems that make sense .25 (.05, .46)
to them, engage them and stimulate them to inquire.
Applied problem-solving .35
Role-playing .61
Dialogue Learning through discussion. .23 (.07, .39)
Teacher poses questions .42
Class discussion led by teacher .42
Small group discussion led by teacher .41
Authentic1Dialogue .32 (.17, 47)
Authentic1Dialogue1Mentoring Mentoring is one-on-one interaction between an expert (or someone .57 (.38, .77)
with more expertise) and a novice where the mentor models and
error corrects based on a critical analysis.

the effect sizes in Tables 2 and 3 should be interpreted with Lessons. Individual lessons should be designed with
some caution as the context varies, they represent effects CT in mind by intentionally providing learners opportu-
across a variety of disciplines, outcome measures and study nities to engage in complex thinking. Appendix 2 offers a
designs, thus suggesting a more generalizable effect. guide to developing these types of opportunities for stu-
Courses. Within a course structure, collaborative dents. The goals of the activities should be made clear and
learning (ie, peer teaching, cooperative learning) helps de- instructors should acknowledge that effortful thinking is
velop CT more than other activities. One meta-synthesis required while recognizing that the learning environment
that attempted to integrate results from different but in- allows students to make mistakes. Instructors should ex-
terrelated qualitative studies on critical thinking found plicitly model their expert thinking and actively monitor
an effect size of 0.41 for promoting CT skills when col- how students are learning. Adjustments to teaching
laborative learning was used.1,32 Collaborative learning should be made reactively as instructors notice trends in
provides feedback to learners and puts learners in a setting student thinking. Providing enough time to think and
that challenges their assumptions and engages them in learn during these activities is crucial. Expect novice stu-
deeper learning to solve a problem. However, if learners dents to take at least double the time it would take you as
receive minimal guidance, they may become lost and an expert to complete the activity. Appendix 3) provides a
frustrated or develop misunderstandings and alternative worksheet that students can use to develop their CT skills
understandings.32,36 Students’ CT improves most in envi- during an activity.
ronments where learning is mediated by someone who Instructors. While the curriculum structure can have
confronts their beliefs and alternative conceptions, en- a large effect, it relies heavily on the individual instructor.
courages them to reflect on their own thinking, creates Instructor training has been found to be the most effective
cognitive dissonance or puzzlement, and challenges and intervention in developing CT skills (Table 3). This train-
guides their thinking when they are actively involved in ing, however, must go beyond having students observe
problem solving. This guided participation role may be others think critically. This facilitation requires the ap-
implemented by learners in structured activities with the propriate material (eg, cases), facilitation skills and men-
guidance, support, and challenge of companions.26 toring skills.32 Appendix 4 provides a rubric to help
165
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

Table 3. Effect Size for Pedagogical Grounding of would enter the generation phase, find all the problems
Intervention.1 (Effect sizes may include: pre-post design, and facts (laboratory values, past medical history, etc.).
quasi experimental design, or true experimental design. Then the learner would define the problem(s) and gener-
Outcome measures may include standardized critical thinking ate ideas as to why the problems are occurring. For exam-
tests, instructor-developed critical thinking assessments, ple, the patient is complaining of fatigue and the learner
researcher-developed critical thinking assessments or some
would have to come up with reasons why fatigue might
combination thereof. Study durations range from short – 1
hour to 2 days – to greater than 1 semester.
occur (anemia, lack of sleep, pregnancy, poor diet). The
learner then uses the facts to evaluate each potential cause
Type of Intervention Effect Size (g1)
and consider what further tests may be necessary to exclude
Instructor training 1.0 (.92, 1.07) some of the potential causes. After selecting the cause, the
Extensive observations (observing .58 (.51, .65) learner formulates a plan and decides his or her next action.
others do it) Once the learner discovers the patient is anemic, the cycle
Detailed curriculum description .31 (.22, .40)
restarts with treatment options. This cycle can be used
Critical thinking among course .13 (.09, .17)
objectives
along with the Joint Commission of Pharmacy Practi-
tioners Pharmacists’ Patient Care Process.39

instructors assess students’ problem-solving skills on a CONCLUSION

problem-solving activity. Though difficult, instructors Critical thinking skills (interpretation, analysis, eval-
should often remain silent during the activity. When nec- uation, inference, explanation, and self-regulation) are
essary, instructors can ask probing questions that require important for health care providers, including pharma-
students to clarify, elaborate, explain in more depth or ask cists. While some students and instructors may think that
more questions, which are related to metacognition. In- CT skills are fixed, CT can be developed and augmented
structors can signal acceptance of the student’s assertions through a process of attitude alignment, absorption of
by paraphrasing, providing a friendly facial expression, or knowledge, and learning new thinking skills. CT is also
writing responses for all to see. The key is to facilitate developed when one learns to combat potentially hazard-
learning and not “do” the learning for the students. ous CT roadblocks such as bias, heuristics (thinking
shortcuts), and simply not wanting to go through the effort
Recommendations of thinking on a higher level. Pharmacy educators can
A common model for the process of CT should be used foster the development of CT skills in the wide scope of
in each pharmacy school curriculum. Ideally, a course should curricular design, in the narrowest interactions between
be required for all students early in the curriculum that ad- professor and student, and everywhere in between. It is
dresses the definition, common model, and dispositions of CT important to note that the methods described in this paper
and then provides an opportunity for students to actively prac- do not have to be added to an already compressed curric-
tice these skills on general subject matter content. As students’ ulum but rather can be used with existing materials to
knowledge of pharmacy specific content grows, courses need cover the content in a deeper and more meaningful way.
to explicitly use the process outlined in the general course By modeling expert thinking and using scaffolding tech-
with application to the subject specific content. The repetition niques to support students’ CT development, pharmacy
of these skills in multiple courses or course series will help educators can instill both the desire and the drive for
students practice this skill. Additionally, all instructors should students to begin thinking critically. Regardless, it is note-
learn the model taught to students and learn how to create and worthy to point out that teaching CT skills requires time
facilitate activities to encourage CT in their content areas. and effort at the potential expense of other skills. Thus,
While there may be many templates for CT, we pro- gains in critical thinking during a PharmD curriculum
pose a 4-step cycle: generation, conceptualization, opti- may be a function of our need to develop a multitude of
mization and implementation.37,38 In the generation other skills like teamwork, empathy, adaptability, com-
phase, learners identify the problem and find facts. This munication, and initiative.White boxes represent the
is followed by the conceptualization phase when learners thinking type while gray boxes provide descriptions of
define the problem and draft ideas that could explain the each type and show how the skills build upon each other.
defined problem. In the optimization phase, learners eval-
uate and select an idea then design a plan. Finally, the
REFERENCES
implementation phase involves accepting the plan and 1. Abrami PC, Bernard RM, Borokhovski E, et al. Instructional
taking action. The cycle restarts with finding a new prob- interventions affecting critical thinking skills and dispositions: a
lem. For example, during a patient encounter, a learner stage 1 meta-analysis. Rev Educ Res. 2008;78(4):1102-1134.

166
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

2. Alfaro-LeFevre R. Critical Thinking, Clinical Reasoning, and 22. Jonas E, Schulz-Hardt S, Frey D, Thelen N. Confirmation
Clinical Judgment: A Practical Approach. 6th ed. Philadelphia, PA: bias in sequential information search after preliminary decisions:
Elsevier; 2017. an expansion of dissonance theoretical research on selective
3. Oderda GM, Zavod RM, Carter JT, et al. An environmental scan exposure to information. J Pers Soc Psychol. 2001;80(4):
on the status of critical thinking and problem solving skills in 557-571.
colleges/schools of pharmacy: report of the 2009-2010 Academic 23. Persky AM, Robinson JD. Moving from novice to expertise and
Affairs Standing Committee. Am J Pharm Educ. 2010;74(10):Article its implications for instruction. Am J Pharm Educ. 2017;81(9):Article
S6. 6065.
4. Hughes R. Patient Safety and Quality: An Evidence-based 24. Haier RJ, Siegel BV, MacLachlan A, Soderling E, Lottenberg S,
Handbook for Nurses. Rockville, MD: Agency for Healthcare Buchsbaum MS. Regional glucose metabolic changes after learning a
Research and Quality, US Department of Health and Human complex visuospatial/motor task: a positron emission tomographic
Services; 2008. study. Brain Res. 1992;570(1):134-143.
5. Oyler DR, Romanelli F. The fact of ignorance: revisiting the 25. Jäncke L, Shah NJ, Peters M. Cortical activations in primary and
Socratic method as a tool for teaching critical thinking. Am J Pharm secondary motor areas for complex bimanual movements in
Educ. 2014;78(7):Article 144. professional pianists. Cog Brain Res. 2000;10(1-2):177-183.
6. Ennis RH. Critical thinking asssessment. Theory Pract. 1993; 26. Beyer BK. Improving Student Thinking: A Comprehensive
32(3):179-186. Approach. Boston, MA: Allyn and Bacon; 1997.
7. Erstad W. Six critical thinking skills you need to master now. 27. Johnson DW, Johnson RT. Energizing learning: the instructional
http://www.rasmussen.edu/student-life/blogs/college-life/critical- power of conflict. Educ Res. 2009;38(1):37-51.
thinking-skills-to-master-now/. Accessed February 21, 2018. 28. Wynn K. Addition and subtraction by human infants. Nature
8. Facione PA. Critical Thinking: A Statement of Expert Consensus 1992;358(6389):749-750.
for Purposes of Educational Assessment and Instruction. Millbrae, 29. Malesker MA, Callahan-Lyon P, Ireland B, Irwin RS. CHEST
CA: American Philosophical Association; 1990. expert panel. Pharmacologic and nonpharmacologic treatment for
9. Halpern DF. Thought and Knowledge: An Introduction to Critical acute cough associated with the common cold: CHEST expert panel
Thinking. 5th ed. New York, NY: Psychology Press; 2014. report. Chest 2017;152(5):1021-1037.
10. Bransford Jd, Stein BS. The Ideal Problem Solver: A Guide for 30. Pratter MR, Brightling CE, Boulet LP, Irwin RS. An empiric
Improving Thinking, Learning, and Creativity. 2nd ed. New York, integrative approach to the management of cough: ACCP evidence-
NY: W.H. Freeman; 1993. based clinical practice guidelines. Chest. 2006;129(1 Suppl):222S-
11. Medina MS, Plaza CM, Stowe CD, et al. Report of the 2012-2013 231S.
academic affairs standing committee: revising the Center for the 31. Huber CR, Kuncel NR. Does college teach critical thinking? A
Advancement of Pharmacy Education (CAPE) Educational meta-analysis. Rev Educ Res. 2016;86(2):431-468.
Outcomes 2013. Am J Pharm Educ. 2013;77(8):Article S8. 32. Abrami PC, Bernard RM, Borokhovski E, Waddington DI, Wade
12. Hartman D. Strategies for critical thinking and problem solving. CA, Persson T. Strategies for teaching students to think critically: a
http://smallbusiness.chron.com/strategies-critical-thinking-problem- meta-analysis. Rev Educ Res. 2015;85(2):275-314.
solving-12709.html. Accessed February 21, 2018. 33. Cohen J. A power primer. Psychol Bull 1992;112(1):155-159.
13. Linn A, Khaw C, Kildea H, Tonkin A. Clinical reasoning: a guide 34. Hattie J. Visible Learning: A Synthesis of Over 800 Meta-
to improving teaching and practice. Aust Fam Physician. 2012; analyses Relating to Achievement. London; New York, NY:
41(1-2):18-20. Routledge; 2009.
14. Ericsson KA. The Road to Excellence: The Acquisition of Expert 35. Gholami M, Moghadam PK, Mohammadipoor F, et al.
Performance in the Arts and Sciences, Sports, and Games. Mahwah, Comparing the effects of problem-based learning and the traditional
NJ: Lawrence Erlbaum Associates; 1996. lecture method on critical thinking skills and metacognitive
15. Medina MS, Castleberry AN, Persky AM. Strategies for awareness in nursing students in a critical care nursing course. Nurse
improving learner metacognition in health professional education. Educ Today. 2016;45:16-21.
Am J Pharm Educ. 2017;81(4):Article 78. 36. Angeli C, Valanides N. Instructional effects on critical thinking:
16. Dweck CS. Can personality be changed? The role of beliefs in performance on ill-defined issues. Learn Instruct. 2009;19(4):322-
personality and change. Curr Direct Psychol Sci. 2008;17(6):391-394. 334.
17. Gigerenzer G, Gaissmaier W. Heuristic decision making. Ann 37. Scheirer MA. Planning evaluation through the program life
Rev Psychol. 2011;62(1):451-482. cycle. Am J Eval. 2012;33(2):263-294.
18. Martignon L, Hoffrage U. Fast, frugal, and fit: simple heuristics 38. Romiszowski AJ. Designing Instructional Systems: Decision
for paired comparison. Theory Decis. 2002;52(1):29-71. Making in Course Planning and Curriculum Design. New York, NY:
19. Evans JS. Dual-processing accounts of reasoning, judgment, and Nichols Pub.; 1981.
social cognition. Ann Rev Psychol. 2008 2008;59:255-278. 39. Joint Commission of Pharmacy Practitioners. The pharmacists’
20. De Neys W, Rossi S, Houdé O. Bats, balls, and substitution patient care process. https://jcpp.net/patient-care-process. Accessed
sensitivity: cognitive misers are no happy fools. Psychon Bull Rev. February 21, 2018.
2013;20(2):269-273. 40. List of biases in judgment and decision making. http://psychology.
21. Kahneman D. Thinking, Fast and Slow. New York, NY: Farrar, wikia.com/wiki/List_of_biases_in_judgment_and_decision_making.
Straus and Giroux; 2013. Accessed February 23, 2018.

167
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

Appendix 1. List of Biases That May Impact Critical Thinking40

Bias Definition
Affective bias Occurs when emotions about the situation intervene with objective reasoning and decision-making.
Ambiguity effect Tendency to avoid options for which missing information makes the probability unknown.
Anchoring The over reliance on an initial single piece of information or experience to make subsequent
judgments. Once an anchor is set, other judgments are made by adjusting away from that
anchor, which can limitone’s ability to accurately interpret new, potentially relevant information.
Authority bias The tendency to attribute greater accuracy to the opinion of an authority figure (unrelated to
its content) and be more influenced by that opinion.
Availability bias Attributing cause based on what readily comes to mind either because the latter is common or was
recently encountered.
Availability heuristic People overestimate the importance of information that is available to them. Example: a
person might argue that smoking is not unhealthy because they know someone who
lived to 100 and smoked three packs a day.
Bandwagon effect The probability of one person adopting a belief increases based on the number of
people who hold that belief (a form of group think).
Base-rate fallacy Tendency to ignore the base rate information and focus on specific information.
Blind-spot bias Failing to recognize your own cognitive biases. People notice cognitive and motivational
biases much more in others than in themselves.
Choice-supportive bias When you choose something, you tend to feel positive about it even if that choice has flaws.
Clustering illusion Tendency to see patterns in random event.
Confirmation bias Occurs when decision makers seek out evidence that confirms their previously held beliefs,
while discounting or diminishing the impact of evidence in support of differing conclusions.
Conservatism bias People favor prior evidence over new evidence or information that has merged.
Focusing effect Tendency to place too much importance on one aspect of the event or problem.
Framing effect Drawing different conclusions to form the same information depending on how that
information is presented.
Gambler’s fallacy Expect past events to influence the future.
Halo effect An observer’s overall impression of a person, company, brand, or product. Overall impression
influences the observer’s feelings and thoughts about that entity’s overall character or properties.
It is the perception, for example, that if someone does well in a certain area, then they will
automatically perform well at something else regardless of whether those tasks are related.
Information bias Tendency to seek information when it does not affect the action.
Ostrich effect Decision to ignore dangerous or negative information by burying one’s head in the sand like
an ostrich.
Outcome bias Judging a decision based on the outcome rather than how exactly the decision was made
in the moment.
Overconfidence bias Occurs when a person overestimates the reliability of their judgments. This can include the
certainty one feels in her own ability, performance, level of control, or chance of success.
Experts are more prone to this bias than laypeople.
Pro-innovation bias When a proponent of an innovation tends to overvalue its usefulness and undervalue its limitations.
Recency Tendency to weigh the latest information more heavily than older data.
Salience Tendency to focus on the most easily recognizable feature of a person or concept.
Satisfaction of search Tendency to end a search after one has led to a finding, despite the lack of a thorough examination
of the factors in a particular case.
Selective perception Allowing one’s expectation to influence how he or she perceives the world.
Stereotyping Expecting a group or person to have certain qualities without having real information about the person.
Survivorship bias An error that comes from focusing only on surviving examples causing a misjudgment. Example:
we might think that being a doctor is easy because we have not heard of those who failed as a doctor.
Sutton’s slip Tendency to evaluate the obvious problem and address it immediately without a thorough
examination of other helpful information.
Zero-risk bias The love of certainty and elimination of risk.

168
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

Appendix 2. Generating Thoughtful Questions to Engage Students in Critical Thinking26

Select a topic
a. Rich enough detail, depth of detail, implications and interconnections and relationships inside and outside of area.
b. Open to diverse interpretation and methods of inquiry.
c. Capable of being entered at any variety of points.
d. Requires guidance of an instructor.
e. Is one that instructors are likely to spend lots of time on instead of rushing through it.
f. Contributes to the development of meaningful and significant key ideas, explanation, principles, concepts, and generalizations.
g. Can be learned about in the context of realistic problems.
h. Fits into the overall curriculum and course

Begin at the global level

i. Focuses on big picture.
j. Focuses on ill-defined rather than precisely delineated topics.
k. Requires students to pose and then answer numerous subordinate questions as they seek to define and probe the initial
question and implications.

Word the question provocatively

l. Helps invite student engagement; questions that present unusual, unanticipated, or unconventional points of view bother
people, agitate thinking, spark curiosity, and demand response.

Engage the students

m. Focus on the “non-present” – predicting or planning future conditions or events, reconstructing past events.
n. Have students reflect about questions before they attempt to answer them or before examining the answers they generate.

Appendix 3. Critical Thinking/Making an Argument Worksheet

Adapted from Halpern’s Thought and Knowledge: An Introduction to Critical Thinking.9

1. State your conclusion.

2. Give 3 reasons (or some other number) that support your conclusion. Rate each reason (weak, moderate, strong, very
strong).
3. Give 3 counterarguments (or some other number) that weaken your conclusion. Rate how much each counterargument
weakens the conclusion (little, moderate, much, very much).
4. List any qualifiers (limitations on the reasons for or against).
5. List any assumptions.
6. Are your reasons and counterarguments directly related to your conclusion?
7. What is the overall strength of your argument? Weak, moderate, strong, very strong

169
 American Journal of Pharmaceutical Education 2019; 83 (2) Article 7033.

Appendix 4. University of Oklahoma College of Pharmacy Problem Solving Rubric

Created by Dr. Melissa Medina
Part 1: Prioritizing Answers (Explicit ranking/numbering of multiple-choice (MC) options) Points Earned
Each MC answer was numbered in order of priority (15best answer to 45least desirable). 2
Only the best answer was numbered (eg, #1 or the best answer/first choice is). 1
More than one MC answer/option was numbered but not all four answers were numbered. 1
The order of priority for the MC answers/options was implied but not explicitly stated. 0.5
None of the MC answers/options were numbered in order of priority. 0

The “best” answer is selected (as determined by the problem’s answer key). 1
An acceptable answer is selected (as determined by the problem’s answer key). 0.5
A wrong answer is selected (as determined by the problem’s answer key). 0

Part 2: Defending Answers with Evidence (Type/strength/quality of evidence used) Points Earned
The “best” answer selected is explicitly justified by $1 strong reason that uses evidence-based 2
literature (eg, medical literature, referenced journal article, explicitly named medical guidelines, or
stated patient data).
The “best” answer selected is explicitly justified by $1 moderate reason using other data (eg, class notes 1
or data not directly referenced in medical literature or guidelines).
The “best” answer selected uses $1 weak reason (eg, opinion or vague statements such as per guidelines 0.5
or literature) to defend why it is believed to be the best answer.
The “best” answer selected has no reasons or wrong reasons defending why it is believed to be the best 0
answer (eg, wrong patient data, wrong guidelines).

The other MC options are explicitly justified ($ 1 strong reason for each option) using the 2
medical literature, referenced journal article, explicitly named medical guidelines, or stated
patient data (if majority of reasons are strong, give full credit, if not, subtract 0.5 for each).
The other MC options are explicitly justified (1 moderate reason for each option) using other data 1
(eg, class notes or data not directly referenced in medical literature or guidelines) (if majority
of reasons are moderate, give full credit, if not, subtract 0.5 for each).
The other MC options are explicitly justified (1 weak reason for each option or majority of reasons) 0.5
using opinion (or vague statements eg, per guidelines or literature).
The other MC options (each option) have no reasons or wrong reasons as support. 0

Each MC answer (each option) has at least 1 exclusive reason that does not overlap with the 2
other reasons (eg, writing “not first line choice for all other options”).
Some of the MC answers (each option) have at least 1 exclusive reason that don’t overlap with 1
other reasons (eg, writing “not first line choice for some options”).
None of the MC answers (each option) has at least 1 exclusive reason that does not overlap with 0
other reasons, eg, not first line choice).

Part 3: Organization of the Answers Points Earned

The written answers (reasons justifying each choice) are well-organized and concise (it is clear 1
which answer matches each choice).
Answers are generally well-organized; occasionally skipped around, slightly unclear or wordy. 0.5
Answers are difficult to follow, more logical flow needed, mostly unclear and wordy. 0

TOTAL SCORE Points Earned

Part 1. Answer Prioritization Subtotal _____/3 points
Part 2. Evidence Subtotal _____/6 points
Part 3. Organization Subtotal _____/1 point
Total Overall Score _____/10 points
Strengths:
Areas for improvement:
Goal(s) for Next Session:

170