A LI VELY ELECTRONI C COMPENDI UM OF RESEARCH, NEWS, RESOURCES, AND OPI NI ON

Astronomy Education Review

2012, AER, 11, 010102-1, 10.3847/AER2009067
Improving Student Attitudes about Learning Science and
Student Scientific Reasoning Skills
Douglas K. Duncan
University of Colorado, Boulder, Colorado 80309
Leilani Arthurs
University of Colorado, Boulder, Colorado 80309 and University of Nebraska, Lincoln, Nebraska 68588-0340
Received: 10/26/09, Accepted: 08/12/11, Published: 01/24/12
VC
2012 The American Astronomical Society. All rights reserved.
Abstract
Student attitudes about learning science and student ideas about the nature of science were compared at the end
of two astronomy courses taught in Fall 2007, a course with a traditional astronomy curriculum and a
transformed course, whose traditional astronomy curriculum was supplemented by an embedded curriculum that
explicitly addressed the nature of science and student metacognition (i.e., thinking about ones own thinking.)
The embedded curriculum in the transformed course gave students practice at evaluating examples of valid
science and pseudoscience found on the internet; it also provided students opportunities to discuss what they
think about learning science. Student attitudes and ideas were assessed using the epistemological beliefs
assessment for physical science (EBAPS) survey, interviews, and written responses to an open-ended exam ques-
tion. Our results indicate that the embedded curriculum led the majority of students in the transformed course to
think that anyone can learn science, whereas a majority of students in the traditional course thought that only
individuals with innate abilities can learn science and think scientically. Students in the transformed course also
reported much more condence in their ability to evaluate the scientic validity of information found on the
internet. Furthermore, students from the transformed course valued making sense of science more than students
from the traditional course. The embedded curriculum could readily be used in any course for nonscience majors,
not just introductory astronomy.
1. INTRODUCTION AND PREVIOUS WORK
Many universities have requirements that nonscience majors take at least one science course. The goal is not to
turn them into scientists; rather it is to give them a better understanding of how science works, and an
understanding of scientic ways of reasoning. A desirable outcome is that students reasoning abilities improve.
At the precollegiate (K-12) level, the National Science Education Standards (1996) indicate that students should
understand the nature and practice of science. Understanding how science works and the nature of scientic
knowledge has been seen for decades as important to the scientic literacy of the general populace (Miller 1989;
Miller 1998). Scientic literacy can no longer be reserved for scientists alone because nonscientist members of
our society are and will continue to be confronted with personal and public policy decisions that involve science
(e.g., medical treatments for cancer or voting decisions on issues concerning Earths environment.) This demands
a working knowledge of what science is, how scientic knowledge is developed, how to distinguish pseudo-
science from valid science, and the limitations of science.
Although crucially important, the nature of science (NOS) is rarely taught or assessed. A potential problem is
that there are many denitions of the NOS. We follow Lederman (1992) who states that the NOS refers to the
epistemology of science, science as a way of knowing, or the values and beliefs inherent to the development of
scientic knowledge. We adopt a pragmatic approach in the current work by concentrating on aspects of the
NOS agreed upon by a majority of scientists and useful to students (e.g., Lederman 1992; McComas 2004; and
Abd-El-Khalick, Waters, Le 2008), particularly the ideas concerning what science is.
Assessing students views about the NOS is challenging. Forced-choice instruments and open-ended interviews
have limitations associated with suitability for a given audience, pertinent depth, and breadth of individual and
collective items, and relevant context of and content of the items (Elby et al. 2001, EBAPS home page). Forced-
choice instruments may not enable students to adequately express their own ideas. Open-ended instruments or
interviews may require follow-up questions that can potentially lead students toward particular responses
[Sandoval, 2003]. However, the goal of many of those who teach science courses to nonscience majors is to
improve their scientic reasoning and help them to make better informed decisions about scientic issues. This is
a more limited goal than teaching about all potential aspects of the NOS.
An increasing number of teachers, particularly in K-12 levels and in preservice teaching programs, are speci-
cally targeting the NOS through their course curriculum. Strategies that have been implemented for teaching
NOS include puzzle-solving activities (Clough 1997), pictorial gestalt switches (Michaels and Bell 2003), activ-
ities that require students to make inferences from limited data sets, and an emphasis on the importance of scien-
tic language (e.g., terminology such as law, theory, prove, true) (Clough and Olson 2004). Published examples
of similar efforts at the college level (excluding preservice teaching programs) are scarce.
Various efforts have been made to assess students views about the NOS before and=or after taking traditional
science courses. Traditional is used here to indicate that the NOS is not explicitly taught as a part of the
science course curriculum. A recent study focused on atmospheric science majors at a large research university
(Parker et al. 2008). Seventeen juniors and seniors participated in a study where they completed the views of
nature of science questionnaire version C (VNOS-C) during the rst week of classes; three of these 17 students
also consented to be interviewed. Based on their ndings, Parker et al. recommended explicit integration of
teaching the NOS in atmospheric science undergraduate courses. Another study, by Adams et al. (2006), used the
Colorado learning attitudes about science survey (C-LASS) to measure individual changes in student attitudes af-
ter taking a traditional physics course as compared with their attitudes at the start of the course. Over 7000 stu-
dents from 60 different college-level physics courses participated in the study. The C-LASS instrument was used
to measure how expertlike and novicelike students views about learning science were. Adams et al. found
that in every application of the survey to introductory physics classes, the overall class populations became less
expertlike in their views about learning science. Their ndings support Sandovals (2003) idea that doing science
doesnt necessarily change ideas about science. It has also been shown that students who become more procient
in scientic knowledge (specically, mechanics or electricity, and magnetism) do not necessarily become more
procient in their scientic reasoning abilities (Bao et al., 2009).
With respect to the eld of astronomy, surveys of those who teach introductory college-level astronomy
courses indicate that an important learning goal is to increase students understanding of the nature and process
of science (Slater et al., 2001). Most astronomy courses, however, do not explicitly address the NOS through
assigned home-work or other activities designed to give students practice thinking about the nature or process
of science. They also do not explicitly discuss scientic reasoning. Even astronomy lab activities that are
supposed to provide scaled-down examples of doing science rarely ask students to think about and discuss the
meaning of what they are doing. Such curricula apparently assume that just by learning astronomy content or
doing science activities students will automatically obtain more expertlike ideas about the NOS. The ndings
of Adams et al. (2006), however, suggest that students do not develop more expertlike views about NOS when
NOS is either only implicitly taught or not taught at all; in fact they are likely to develop more novicelike
views.
Herein, we discuss the efforts of one college-level astronomy instructor to explicitly teach selected aspects of the NOS
that deal with dening what science is, scientic reasoning, and metacognition to nonscience majors enrolled in an in-
troductory level astronomy course. The objective of the present study was to examine the following questions:
(1) What is the status of introductory-level astronomy students attitudes about learning science and ideas
about the nature of science after a semester of traditional astronomy instruction?
(2) What is the status of the same student attitudes and ideas when specic aspects of the NOS, scientic
reasoning and judgments, and metacognition are explicitly taught?
Our project studied two introductory courses for nonscience majors taught in fall 2007 at the University of
Colorado by two different instructors. Both instructors are highly rated by students and had taught the same
course more than once. The traditional course covered basic principles of light, motion, and gravity, and applied
them to the solar system. The transformed course covered the same basic principles but applied them to stellar
and extragalactic astronomy. In the traditional course, the NOS was only implicitly covered during the course of
instruction. The transformed course taught by Duncan explicitly discussed specic aspects of the NOS, scientic
reasoning, and metacognition using an embedded curriculum (see below), which was added to the traditional
astronomy curriculum. Students were explicitly taught what science is and the difference between scientic and
other ways of knowing. They were repeatedly asked to make and discuss their own scientic judgments
throughout the semester. Students were also regularly engaged in discussions that required them to think about
how they think about their own learning in the course and in general.
The traditional course was the rst of a 2-course sequence. The transformed course was the second in such a
sequence. The courses were not a perfect match since most students in the transformed course had one previous
semester of astronomy and were on average one semester older. Otherwise the class demographics were very
similar (mostly freshmen and sophomores; comparable numbers of male and female students; class sizes between
150 and 200). Eliciting student participation from these two courses allowed us to collect a sizeable amount of
data, including a large number of interviews and classroom observations by Arthurs. We hope that our interesting
results will prompt others to try similar experiments in their courses, some of which are more precisely matched.
Hereafter, students in the course that received traditional instruction will be referred to as the Control Class
and the students in the transformed course will be referred to as the Test Class.
2. EMBEDDED CURRICULUM
In order to explicitly teach elements of the NOS and scientic reasoning to his students, Duncan developed a
curriculum that was embedded within the traditional astronomy curriculum of his course. The embedded
curriculum was driven by four learning goals and executed through activities that stimulated student thought
about science (valid science and pseudoscience) and learning science. The activities were implemented under the
premise that giving students practice at thinking, reading, writing, and talking about science and their
experiences learning science would allow them to develop more expertlike understandings of the NOS and what
is involved in learning science. A central question that framed the course was What is science? Although the
answer to this question can be and is hotly debated by historians of science, philosophers of science, and
scientists, for the purposes of this introductory-level astronomy course, students were taught that science is: a
process of trying to understand the world by making models (or theories) which have predictive power and that it
usually involves observation; nding patterns and trends; posing explanations=hypotheses; iterative processes of
data collection, interpretation, and hypothesis testing; and telling others what you have discovered. They were
also taught that the practices of testing hypotheses and making data-based predictions distinguish science from
other ways of understanding the world such as religion, philosophy, and intuition. One technique Duncan used
throughout the course to hook his students into valuing science and wanting to learn science was his regular reci-
tation of Richard Feynmans famous quote, Science is a way of trying not to fool yourself, to which Duncan
also added or being fooled by others.
2.1. Learning Goals
The learning goals of the transformed course included the following. Students will be able to:
(1) Discuss the NOS and specically discuss the question of What is science?
(2) Develop metacognitive skills for learning science.
(3) Distinguish between valid science and pseudoscience (i.e., bogus science).
(4) Recognize how the terms theory, model, and believe differ between scientic and lay contexts.
2.2. Activities
To develop students views about the nature of scientic knowledge and attitudes about learning science, the
primary interventions used were weekly explicit in-lecture discussions and periodic take-home assignments.
These discussions amounted to 10% of the total in-lecture time during the semester. The discussions fell into a
few general categories, including what is science, cases of contemporary pseudo=science issues, and personal
attitudes about learning science. Given that the class attendance averaged 150 students on any given lecture day,
these in-lecture discussions were technology-supported. First, powerpoint slides were projected on a large screen
that was used to display visual images, graphical data, and pose multiple-choice questions. Then, I<CLICKER soft-
ware and hardware were used to poll students and collect their responses. Sometimes, students were also asked to
discuss their responses with nearby peers. After the polling session, the instructor was able to display the aggre-
gate results of the poll and engage students in a whole-class discussion.
2.2.1. Cases of Contemporary Pseudo=Science Issues
Approximately 25% of the cases discussed were drawn from astronomy and 75% from other sciences. All
involved scenarios that students might encounter in their own lives. Personal relevance was a more important
criterion for the selection of cases than was whether the case dealt with astronomy. Part of the teaching strategy
was to promote the value of using scientic reasoning to evaluate information in students own lives. Richard
Feynman was once asked to dene science and, in response, he said, Science is a way of trying not to fool
yourself. Duncan uses this statement with the addition of or be fooled by others in his transformed course as
an effective hook for engaging students in learning about NOSbecause no one likes to be fooled. Using cases
of personal relevance in these discussions were likely particularly useful to the nonscience majors taking the
course because nonscience majors commonly believe that science does not apply to their day-to-day lives
(Stempien 2007). Below are examples of actual cases and discussion questions involving contemporary
pseudo=science issues:
Example A: (1) Pose the question and have the students vote on: Which vehicle is safer to drive? Simultaneously
show photos of each one of the vehicle choices (Toyota Camry, Jeep Cherokee, and Ford F-series pick-up).
(2) View I<CLICKER histogram of polling results. (3) Display and discuss data concerning deaths associated
with each vehicle type.
Example B: (1) Pose the question and have the students vote on: If you are African-American in the US today,
which job are you more likely to have? (a) Sports (playing or coaching) or (b) Science and engineering
(including medicine). (2) Ask students to choose from among the following answer choices: (A.) (a) is more
likely, (B.) (b) is more likely, (C.) About the same, and (D.) (b) is ten times more likely than (a). (3) View
I<CLICKER histogram of polling results. (4) Display and discuss employment data of African Americans.
Example C: (1) Pose the question and have the students vote on: How many people do you think have psychic
power or extra-sensory perception? (A) Only a few people, (B) All people do but some have it more than
others, and (C) No one has been able to demonstrate such powers in a repeatable way when carefully tested.
(2) View I<CLICKER histogram of polling results.(3) Tell a Sherlock Holmes story about being observant. (4)
Pose the same question and have the students vote again. (5) View I<CLICKER histogram of new polling results.
2.2.2. Cases of Personal Attitudes About Learning Science
Nonscience majors often hold views about learning science that can hinder their ability to learn science
(Stempien 2007). Developing a working knowledge of what science is goes hand in hand with the ability to learn
science. A part of the embedded curriculum in the transformed astronomy course therefore directly addressed
students views about learning science. An attempt to simultaneously reveal and shape students attitudes about
learning science was also done through explicit technology-supported in-lecture discussions based on specic
scenarios. These discussions often led to a return to two metacognitive questions, (a) How do you learn science?
and (b) How do you know when you know? Examples of the actual scenarios and discussion questions involving
attitudes about learning science include:
Example A: (1) Present Blooms Taxonomy. (2) Discuss the fact that this class requires students to do more than
simply memorize facts.
Example B: (1) Present midterm grade distribution from the previous year. (2) Discuss what is needed to earn a
particular grade in the upcoming midterm. Contrast the higher grades of students who participate in peer dis-
cussion during class to the lower grades those who just listen.
2.2.3. Take-Home Assignments
In addition to the explicit in-lecture discussions about NOS, seven take-home assignments were also designed to
be a part of the embedded NOS curriculum. Generally, these assignments required students to read about a case
and provide a written response to associated questions. The rst student assignment is shown in Figure 1. The
readings associated with each assignment often presented surprising data, such as on the safety of different
vehicles (e.g., what looks big and safe often isnt) and stereotypes (e.g., what you see in the media often doesnt
reect reality). Discussion of the readings and students responses to the associated questions emphasized the
need for actual data rather than a reliance on appearances, anecdotes, or intuition. One reading assignment
described a test that showed how astrology works (i.e., it depends on psychology, not astronomy). Two reading
assignments involved alien abductions and repressed memory syndrome as examples of how personal experience
is not always reliable. Other reading assignments involved so-called psychics, who look impressive on YouTube
but are impressively exposed in other videos. Several reading assignments involved medical or health issues and
were particularly thought-provoking because students, their family members, and=or their friends had related per-
sonal experiences.
Some assignments were deliberately provocative because the goal was to have students recognize that scientic
topics can affect their lives in serious ways. For example, when discussing vaccination-refusers (i.e., those who
believe vaccines cause autism and refuse to have their children vaccinated) students were shown a photo of a
polio ward from the days before polio vaccines. When asked, less than 5% of the students had ever seen such a
photo. When discussing natural medicines students are told about the story of a professional baseball player
who died from using Ephedra. The baseball player was 23 years old, similar in age to most of the students in the
class. Discussion of these examples produced a strong reaction from many students, which provided an optimum
opportunity for teaching about NOS in the context of day-to-day realities. All the assignments are available at
http://casa.colorado.edu/~dduncan/pseudoscience.
3. ASSESSMENT
Three different techniques were used to assess the Control and the Test classes end-of-semester status of their
views about NOS. They were administered a forced-choice instrument called epistemological beliefs assessment
for physical science (EBAPS), participated in structured interviews to answer open-ended questions about the
nature of science and were asked on their nal exams whether their class changed the way that they think about
science. Acknowledging the aforementioned limitations of these kinds of assessment techniques as well as the
difculty in capturing comprehensive student thoughts about NOS, our objective was to use their combined
results to provide a snapshot of students views after having received a semesters worth of traditional and
transformed instruction about NOS within the context of an introductory college-level astronomy course.
3.1. Epistemological Beliefs Assessment for Physical Science
The EBAPS is a 30-item forced-choice instrument, which was initially developed and validated by Elby et al.
(2001) at the University of California at Berkeley. The intended audiences for EBAPS administration are high
school and college level students enrolled in physics, chemistry, or physical science courses. The two introduc-
tory level astronomy courses involved in this study integrated elements of physics and chemistry throughout the
curriculum. Although the EBAPS was not expressly designed for astronomy courses, we thought it could still be
a valuable instrument in our study. We were particularly interested in seeing whether any differences emerged
between the Control Class and the Test Class in terms of their responses to the subclasses that comprise the
EBAPS. These subclasses include: (1) structure of scientic knowledge, (2) nature of knowing and learning, (3)
Figure 1. The rst science vs pseudoscience assignment was given out in the rst week of class.
real-life applicability, (4) evolving knowledge, (5) source of ability to learn, and (6) overall NOS. The EBAPS is
available at http://www2.physics.umd.edu/~elby/EBAPS/home.htm.
The EBAPS was administered at the end of the semester to students in the Control and Test classes. The Control
Class had complete responses from 45% of students and the Test Class from 56%. Enrollment in the Control
Class was 380 students, in two sections of similar size that were taught separately by the same instructor. The
Test Class enrolled 123 students.
Out of the 30 EBAPS questions, six questions produced responses that were statistically different between the
Test and Control Classes (Figure 2). For these six questions, the difference between the classes was large (>3.5
r). In terms of the subclasses in the EBAPS, four questions came from subclass 5: source of ability to learn (Q 9,
16, 22, and, 25), 1 question came from subclass 6: overall NOS (Q 4), and one question came from the subclass
2: nature of knowing and learning (Q 30).
We interpret the collective responses to the rst ve aforementioned questions as an indication of students ideas
about who can do science and the responses to the last question (Q 30) as an indication of the importance that
students place on making sense of science. Figure 3 shows a typical example of one of the 6 questions where the
responses signicantly differed between the classes.
The responses differ substantially, as may be quantied through a chi-square test of the null hypothesis that the two
samples could have been drawn from the same population. Table 1 shows the data for both classes as well as the
expected data if both samples are summed together and the responses of the ve bins taken as the average of the two
samples. Chi-squared is equal to the sum of the squared differences between observed and expected, divided by the
expected (since the variance in a counting experiment such as this follows a Poisson distribution; r
2
n
expected
).
The reduced Chi-sq for 9 degrees of freedom is 22.2, showing negligible chance that the differences are not real.
A Kolmogorov-Smirnov test of the two cumulative distributions produces a difference statistic D0.30. The
Figure 2. Six EBAPS questions solicited differences in student responses from the Control Class and the Test Class that were
statistically signicant. Test class: green circles; control class: red squares.
corresponding probability is p <0.001 that the difference between the two classes is due to chance. We also com-
pared the classes by forming a single average index for each question in each class:
average response 2

N
strongly disagree
N
disagree
N
agree
2

N
strongly agree
:
Figure 2 plots this index for the six questions where the classes differ signicantly. For the question discussed in
Figure 3, the difference exceeds 4r. For all six questions it exceeds 3r.
3.2. Student Interviews
For the purposes of the overall project interviews were comprised of 9 questions. Five of the nine questions were
developed by Duncan, and four were drawn from the views of nature of science (VNOS) Questionnaire version
C (VNOS-C; Lederman et al. 2002). The VNOS authors suggest that the VNOS-C be administered as a written
questionnaire and that the administration be followed up with individual interviews to insure the validity of the
instrument. For the purposes of our project, however, we did not ask students for written responses to the selected
VNOS-C questions and instead asked the questions during student interviews. The nine interview questions asked
are included below. Questions #1, 6, 7, and 8 were derived from the VNOS-C.
1. What, in your view, is science? What makes science (or a scientic discipline such as physics, biology,
etc.) different from other disciplines of inquiry (e.g., religion, philosophy)?
2. What is a scientic theory?
3. What is the scientic method?
4. Does following the scientic method guarantee you will achieve correct results?
5. Does following the scientic method improve your chances of getting correct results?
6. After scientists have developed a theory (e.g., atomic theory), does the theory ever change? If you believe
that theories do change, explain why we bother to teach scientic theories. Defend your answer with
examples.
7. It is believed that about 65 million years ago the dinosaurs became extinct. Of the hypotheses formulated
by scientists to explain the extinction, two enjoy wide support. The rst, formulated by one group of
scientists, suggests that a huge meteorite hit the earth 65 million years ago and led to a series of events
that caused the extinction. The second hypothesis, formulated by another group of scientists, suggests that
Figure 3. (a) Test Class and (b) Control Class responses to the EBAPS question, When it comes to science, students either
learn quickly, or not at all. On the x-axis, 2 represents strongly disagree, 1 represents disagree, 0 is neutral, 1 agree,
and 2 strongly agree.
Table 1. Percentage responses of the actual data shown in Figure 3, along with expected values if both
were drawn from the same population.
Observed values Expected values
Test class 39 41 16 4 0 Test class 25 38 12 22 3
Control class 19 37 10 30 3 Control class 25 38 12 22 3
massive and violent volcanic eruptions were responsible for the extinction. How are these different
conclusions possible if scientists in both groups have access to and use the same set of data to derive their
conclusions?
8. Some claim that science is infused with social and cultural values. That is, science reects the social and
political values, philosophical assumptions, and intellectual norms of the culture in which it is practiced.
Others claim that science is universal. That is, science transcends national and cultural boundaries and is
not affected by social, political, and philosophical values, and intellectual norms of the culture in which it
is practiced. If you believe that science reects social and cultural values, explain why. Defend your
answer with examples. If you believe that science is universal, explain why. Defend your answer with
examples.
9. Richard Feynman has been quoted as saying, Science is a way of trying not to fool yourself. What do
you think he meant?
The complete results of the interviews will be presented in detail in a future paper. Here, we report on one
question: What is a scientic theory? (The word scientic was emphasized.) Twenty students from the Control
Class and 23 students from the Test Class were interviewed. In addition, nine students who had not yet taken an
astronomy course were interviewed. Students were given a small amount of course credit for participating in the
interviews, and it was emphasized that students of all levels of ability were desired as subjects. To examine and
compare their responses, we developed a 2-category rubric prior to rating student responses for what we consid-
ered an expertlike understanding and a novicelike understanding of what a scientic theory is. Responses were
categorized as novicelike if they included ideas such as (just) an idea, a guess, an opinion or hypothesis
without mentioning any testing. Responses were categorized as expertlike if they included phrases such as based
on [any of the following] data, evidence, experiment, and=or observations or a tested idea. The rubric was
intended to reect the common confusion between a scientic theory and the everyday use of the word theory.
Using the rubric to rate student responses, the authors had an initial inter-rater reliability of 89%. All discrepan-
cies in ratings were resolved through discussion. Figure 4 summarizes the ndings. They show that no statisti-
cally signicant difference is observed in the percentages of students in the Control Class and Test Class that
hold an expertlike understanding and a novicelike understanding on the single question of what a scientic theory
is. Students in both classes have more expertlike understandings of what a scientic theory is compared with stu-
dents that had not yet studied astronomy. Given that understanding what a scientic theory is is a demonstrated
area of deciency with respect to students ideas about NOS (Abd-El-Khalick, Waters, Le 2008) and that this is
demonstrated in the responses of students in this study that had not yet taken an astronomy course, the instruction
students received in both the traditional and transformed courses seems to have successfully facilitated their
attainment of a more expertlike understanding of what a scientic theory is.
Figure 4. Students in the rst week of astronomy instruction (No astronomy studied) misunderstand the use of the word
theory when speaking of scientic theories.
3.3. Final Exam Question
A nal assessment of students views about science in both the Control Class and the Test Class comes from a
question asked on the nal. The nal question of the nal exam was, Has this class changed the way you think
about science? If so, how? A small amount of credit was given for any answer. From each class, we randomly
selected 116 student responses to review, bin (i.e., sort), and code. During the initial binning process, 19 different
categories emerged. A 19-category coding rubric was then used by two independent coders to code the student
responses. It is worth noting that the students open-ended responses were such that one student could give a
response with a variety of comments that fell under different categories within the rubric. Intercoder reliability
was greater than 95% and all initial disagreements were resolved through discussion. We summarize key ndings
in Table 2.
It is worth noting that, as with the interview and EBAPS data, the student responses to the open-ended nal exam
question are self-reported perceptions and we did not have a means of verifying their accuracy in the students
actual day-to-day lives.
The most notable differences between student responses from the Test Class versus the Control Class deal with
(i) the ability to distinguish pseudoscience from science and (ii) the ability to be more critical or skeptical about
information and to make better informed decisions. These ndings are undoubtedly a consequence of the
embedded NOS curriculum used in the transformed course.
Another notable difference is visible in the higher number of students from the Control Class who explicitly
mentioned learning about one or more specic astronomy topics and being more aware of them in their
environment. Recalling that the student responses were free writes to an open-ended question, it is possible that
although students from the Test Class also learned many astronomy topics (as evidenced by their exam scores;
both classes averaged B-grades, their new knowledge of pseudoscience and the abilities they acquired through
the embedded curriculum were much more at the forefront of their minds and therefore possibly overshadowed
specic astronomy content that they learned.
A third difference is in the almost double the number of students from the Control Class who claimed that they
would=might continue to learn more about astronomy either independently or by enrolling in another course. In
this regard, one must remember that the Control Class was the rst course of a 2-course sequence that satises
the core science requirement and the Test Class was the second course in such a sequence. In fact, many students
from the Control Class said that they intended to take the second course in the sequence to satisfy the core sci-
ence requirement.
Table 2. Number of students in the control and test classes associated with each type of response. The type
of response represents a category of student responses, not the actual students wording. Table represents
data derived from a subset of 116 students from both the Control and Test classes.
Type of response Control Test
I learned about pseudoscience or can distinguish
pseudoscience from science. 1 49
I question the credibility of information more, can
nd credible sources of information, and am able to
make more educated decisions (and be less fooled by
scams). 7 75
I learned a lot about astronomy topics and now I
notice these things more in my surroundings and
environment (e.g., phases of the moon, shadows,
etc.). 72 38
I might/will learn more about astronomy either on
my own (e.g., google, reading books, etc.) or by
taking a class. 42 18
I changed my views about the nature of science (e.g.,
what we know changes over time, astronomy is more
than just memorizing facts, etc.). 25 23
A nal comparison in this particular data set shows that similar numbers of students shared comments that
expressed how their views on the nature of science had changed. These comments related to, for example, how
they now know that scientic knowledge changes with time, that physics is intimately connected to astronomy,
that astronomy is more complicated than originally perceived, etc. This nding is perhaps surprising in light of
the goals for the embedded curriculum. However, the embedded NOS curriculum did not address these aspects of
science as consistently and with as much focus as it did with goal #3 (stated under Section 2.1) and it is likely
that these ndings are a reection of this difference within the curriculum itself.
4. CONCLUSION
The addition of this embedded curriculum to the transformed astronomy course provided students regular
opportunities to practice thinking, reading, writing, and talking about genuine science, pseudoscience, and how
they learn science. Students from the transformed astronomy course showed signicant differences in their ideas
about NOS when compared with students from the traditional astronomy course. We were able to distill three
major differences using the EBAPS, student interviews, and a nal exam question.
The 6 EBAPS items that showed differences between the Test and Control classes were not a random subset of
the 30 EBAPS items. In particular, ve of the items related to the question of Who can do science? The
differences in responses between the two classes were large, and all support our rst important nding:
Overall, students in the Test Class believed that anyone can do science if they work at it, whereas many
more students in the Control Class believed that one must have natural ability in order to do science.
The sixth EBAPS item that showed a signicant difference dealt with the nature of knowing and learning
science, particularly the importance of making sense of science. Student responses to this item support our
second nding:
Overall, students in the Test Class valued making sense of science as part of their learning experience more
than students in the Control Class did.
The combination of survey results and student responses to the unprompted essay question at the end of the nal
exam lead to our third nding:
The strategy of giving students practice distinguishing valid science and pseudoscience in applications
outside of the introductory astronomy class leads them to self-report more confidence in their ability to
distinguish valid science from pseudoscience, and that they are more likely to think critically in their own
lives.
As the National Science Standards suggest, in order to participate effectively in a democracy, citizens must
understand the nature of scientic claims that increasingly inuence or even become matters of public debate
(1996). They must also be willing to participate. The present investigation suggests that explicit instruction of
NOS and regular practice enhances this important outcome.
We plan to use the results of this study to develop a survey instrument designed specically for use in introductory
level astronomy courses to evaluate the effectiveness of instruction on students as well as their attitudes and
beliefs about science through a Collaboration of Astronomy Teaching Scholars (CATS) research project.
Acknowledgments
We would like to thank the NSF for funding under Grant No. 0715517, a CCLI Phase III Grant for the Collab-
oration of Astronomy Teaching Scholars (CATS) Program. Also, we thank the CU Science Education Initiative
for its support of this study. We thank the reviewers for their valuable comments and suggestions. Last, but not
least, we thank Seth Hornstein and all the students involved in this study.
References
Abd-El-Khalick, F., Waters, M., and Le, A.-P. 2008, Representations of Nature of Science in High School
Chemistry Textbooks Over the Past Four Decades, Journal of Research in Science Teaching, 45(7), 835.
Adams, W. K., Perkins, K. K., Podolefsky, N. S., Dubson, M., Finkelstein, N. D., and Wieman, C. E. 2006,
New Instrument for Measuring Student Beliefs About Physics and Learning Physics: The Colorado Learning
Attitudes About Science Survey, Phys. Rev. ST Phys. Educ. Res., 2, 010101.
Bao, L., Cai, T., Koenig, K., Fang, K., Han, J., Wang, J., Quing, L., Ding, L., Cui, L., Luo, Y., Wang, Y., Li, L.,
and Wu, N. 2009, Learning and Scientic Reasoning, Science, 323(5914), 586.
Clough, M. P. 1997, Strategies and Activities for Initiating and Maintaining Pressure on Students Na ve
Views Concerning the Nature of Science, Interchange, 28(23), 191.
Clough, M. P., and Olson, J. K. 2004, The Nature of Science Always Part of the Science Story, The Science
Teacher, 71(9), 28.
Epistemological Beliefs Assessment for Physics Science (EBAPS) home page. http://www2.physics.umd.edu/
~elby/EBAPS/home.htm, 09=27=2007.
Elby, A., Frederiksen, J., Schwarz, C., and White, B. 2001, The Epistemological Beliefs Assessment for
Physical Science, www.agguide.org.
Lederman, N. G. 1992, Students and Teachers Conceptions of the Nature of Science: A Review of the
Research, Journal of Research in Science Teaching, 29(4), 331.
Lederman, N. G., Abd-El-Khalick, F., Bell, R. L., and Schwartz, R. 2002. Views of Nature of Science
Questionnaire: Toward Valid and Meaningful Assessment of Learners Conceptions of Nature of Science,
Journal of Research in Science Teaching, 39(6), 497.
McComas, W. F. 2004, Keys to Teaching the Nature of Science, The Science Teacher, 71(9), 24.
Michaels, E., and Bell, R. 2003, The Nature of Science and Perceptual Frameworks, The Science Teacher,
70(8), 36.
Miller, J. D., 1989, Scientic literacy, in Paper presented at the Annual Meeting of the American Association
for the Advancement of Science (San Francisco, California), 23 p.
Miller, J. D. 1998, The Measurement of Civic Scientic Literacy, Public Understanding of Science, 7, 203.
National Committee on Science Education Standards and Assessment, National Research Council, and Center
for Science, Mathematics, and Engineering Education, 1996, National Science Education Standards, National
Academy Press, Washington, D.C.
Parker, L. C., Krockover, G. H., Lasher-Trap, S., and Eichinger, D. C. 2008, Ideas About the Nature of Science
Held by Undergraduate Atmospheric Science Students, Bulletin of the American Meteorological Society, 89(11),
1681.
Sandoval, W. A. 2003, The Inquiry Paradox: Why Doing Science Doesnt Necessarily Change Ideas About
Science, in Proceedings of the Sixth International Computer-Based Learning in Science Conference, eds. C. P.
Constantinou and Z. C. Zacharia, 825.
Slater, T., Adams, J. P., Brissenden, G., and Duncan, D. K. 2001, The Physics Teacher, 31, 8.
Stempien, J. 2007, private communication.
R
010102-1010102-11