PSY 321 Reading Guide for Chapter 1

1. What is the difference between research producers and research consumers?

Some psychology students are fascinated by the research process and intend to become
producers of research. Perhaps they hope to get a job studying brain anatomy,
documenting the behavior of dolphins or monkeys, administering personality
questionnaires, observing children in a school setting, or analyzing data. They
expect to write up their results and present them at research meetings. These
students dream about working as research scientists or professors.

Other psychology students may not want to work in a lab, but they do enjoy reading
about the structure of the brain, the behavior of dolphins or monkeys, the personalities
of their fellow students, or the behavior of children in a school setting. They are
interested in being consumers of research information—reading about research
so they can later apply it to their work, hobbies, relationships, or personal growth.

Why is the consumer role important (be sure your answer includes evidence-based
treatments)?
Even though they may produce some research during their undergraduate years, most
psychology majors do not eventually become full-time researchers. Regardless of the
career you choose, however, becoming a savvy consumer of information is
essential. You will need to develop the ability to read about research with
curiosity and a critical eye.

Think about how often you encounter news stories or look up information on the
Internet. Much of the time, the stories you read and the websites you visit will present
information based on data. For example, during an election year, Americans may come
across polling information in the media almost every day. Many online sources have
science sections that include stories on the latest research. Entire websites are dedicated
to psychology-related topics, such as treatments for autism, subliminal learning tapes, or
advice for married couples. While some of the research is accurate and useful, some of it
is dubious, and some is actually fake. How can you tell high-quality research information
when you see it? A course in research methods enables you to ask the
appropriate questions so you can evaluate information correctly.

Finally, being a smart consumer of research could be crucial to your future career.
Whether your goal is to be a social worker, a teacher, a sales representative, a human
resources professional, an entrepreneur, or a parent, you will need to know how to
interpret research data with a critical eye. Clinical psychologists, social workers,
and family therapists read research to know which therapies are the most effective. In
fact, obtaining a license in these helping professions requires knowing the
research behind evidence-based treatments—that is, therapies that are
supported by research. Teachers also use data to find out which methods work best.
And the business world depends on quantitative information: Research is used to predict
what sales will be like in the future or what consumers will buy.

2. How do the findings regarding the Scared Straight program illustrate the benefits of
being a good consumer of research?
The Scared Straight program is another commonsense story that turned out to be wrong.
As you read in Chapter 1, such programs propose that when teenagers susceptible to
criminal activity hear about the difficulties of prison from actual inmates, they will be
 scared away from committing crimes in the future. It certainly makes sense that
impressionable young people would be frightened and deterred by such stories.
However, research has consistently found that Scared Straight programs are
ineffective; in fact, they sometimes even cause more crime. The intuitive appeal
of such programs is strong (which explains why some communities still invest in them),
but the research warns against them. One psychologist estimated that the widespread
use of the program in New Jersey might have “caused 6,500 kids to commit crimes they
otherwise would not have committed” (Wilson, 2011, p. 138). Faulty intuition can
even be harmful.

3. What does it mean to be an empiricist?

Being an empiricist means basing one’s conclusions on systematic observations.

Define empiricism.
Answering psychological questions with direct, formal observations.

4. Define the following terms: theory, hypothesis, data, preregistered

A theory is a set of statements—as simple as possible—that describes general
principles about how variables relate to one another. For example, Harlow’s
theory was that contact comfort, not food, was the primary basis for a baby’s attachment
to its mother. This theory led Harlow to investigate particular kinds of questions—he
chose to pit contact comfort against food in his research. The theory meant that Harlow
also chose not to study unrelated questions, such as the babies’ food preferences or
sleeping habits.

A hypothesis, or prediction, is stated in terms of the study design. It’s the specific
outcome the researcher will observe in a study if the theory is accurate.
Harlow’s hypothesis stated that the babies would spend more time on the cozy mother
than on the wire mother. Notably, one theory can lead to a large number of hypotheses
because a single study is not sufficient to test the entire theory—it is intended to test only
part of it. Most researchers test theories with a series of empirical studies, each designed
to test an individual hypothesis.

Data are a set of observations. (Harlow’s data were the amount of time the baby
monkeys stayed on each mother.) Depending on whether the data are consistent with
hypotheses based on a theory, the data may either support or challenge the theory. Data
that match the theory’s hypotheses strengthen the researcher’s confidence in the theory.
When the data do not match the theory’s hypotheses, however, those results indicate that
the theory needs to be revised or the research design needs to be improved.

Ideally, hypotheses are preregistered. That is, after the study is designed but
before collecting any data, the researcher states publicly what the study’s
outcome is expected to be (perhaps using a time-stamped Internet tool). It’s
unconvincing if a researcher collects the data first and then claims, “I knew that would
happen all along.”

5. What is the purpose of the theory-data cycle?

In the theory-data cycle, scientists collect data to test, change, or update their
theories. Even if you haven’t yet conducted formal research, you have probably tested
ideas and hunches of your own by asking specific questions that are grounded in theory,
making predictions, and reflecting on data. The theory-data cycle is like a gamble:
Researchers place a public bet in advance that the study will come out in favor of the
theory. They are willing to risk being wrong every time they collect data.

What are the components of the theory-data cycle (hint: see figure 1.5)

How does Harlow’s theory illuminate the theory-data cycle?

In the natural world, a mother provides both food and contact comfort at once, so when
the baby clings to her, it is impossible to tell why. To test the alternative theories, Harlow
had to separate the two influences—food and contact comfort. The only way he could do
so was to create “mothers” of his own. He built two monkey foster “mothers”—the only
mothers his labreared baby monkeys ever had. One of the mothers was made of bare
wire mesh with a bottle of milk built in. This wire mother offered food, but not comfort.
The other was covered with fuzzy terrycloth and was warmed by a lightbulb suspended
inside, but she had no milk. This cloth mother offered comfort, but not food.

This experiment sets up three possible outcomes. The contact comfort theory would be
supported if the babies spent most of their time clinging to the cloth mother. The
cupboard theory would be supported if the babies spent most of their time clinging to the
wire mother. Neither theory would be supported if monkeys divided their time equally
between the two mothers. In setting up this experiment, Harlow purposely created a
 situation that might prove his theory wrong.

When Harlow put the baby monkeys in cages with the two mothers, the evidence in favor
of the contact comfort theory was overwhelming.

Harlow’s data showed that the little monkeys would cling to the cloth mother for 12–18
hours a day (Figure 1.4). When they were hungry, they would climb down, nurse from
the wire mother, and then go right back to the warm, cozy cloth mother. In short, Harlow
used the two theories to make specific predictions about how the monkeys would interact
with each mother. The data he recorded (how much time the monkeys spent on each
mother) supported only one of the theories. The theory-data cycle in action!

6. Why do researchers never say that they “proved” their theories?

As empiricists, scientists avoid inferences they cannot support with direct
observation. Consider this set of observations: This raven is black. That raven is black.
Every raven I have ever seen is black. Are we therefore justified in concluding that All
ravens are black? Not if we are empiricists. We have not observed all possible ravens, so
it is possible that a nonblack raven exists. If scientists take their role as empiricists
seriously, they are not justified in making generalizations about phenomena they have
not observed (Hume, 1888). The possible existence of one unobserved nonblack raven
prevents us from saying we’ve proved that all ravens are black.

7. Define falsifiability.
A theory should lead to hypotheses that, when tested, could fail to support the
theory—in other words, falsifiability is a characteristic of good theories (Popper, 1963).

What data would have falsified Harlow’s theory?

Harlow gambled with his study: If the monkeys had spent more time on the wire
mother than the cloth mother, the study would not have supported the contact
comfort theory.

Read Figure 1.7 on page 15 and discuss how the tinfoil hat study does not allow for
falsifiability?
The tinfoil hat study is an example of a theory that is not falsifiable. Certain people might
wear a tinfoil hat, operating under the idea that the hat wards off government mental
surveillance. But like most conspiracy theories, this notion of remote government
mindreading is not falsifiable. If the government has been shown to read
people’s minds, the theory is supported. But if there is no physical evidence,
that also supports the theory because if the government does engage in such
surveillance, it wouldn’t leave a detectable trace of its secret operations.

8. How is science self-correcting?

By being open to falsification and skeptically testing every assumption,
science can become self-correcting; that is, it discovers its own mistaken theories and
corrects them.

9. What is a scientific journal?

Scientific journals usually come out every month and contain articles
written by various researchers.
 Define peer-reviewed.
The journal editor sends the paper to three or four experts on the subject.
The experts tell the editor about the work’s virtues and flaws and the editor,
considering these reviews, decides whether the paper deserves to be
published in the journal.

How are scientific journals different from popular magazines?

Unlike popular newsstand magazines, the articles in a scientific journal are
peer-reviewed.

10. What are peer reviewers supposed to ensure?

Peer reviewers are supposed to ensure that the articles published in scientific
journals contain important, well-done studies. When the peer-review process
works, research with major flaws does not get published. (Some journals reject more
than 90% of submissions after peer review.)

How does the process of peer-review continue even after a study is published?
Other scientists can cite an article and do further work on the same subject. Moreover,
scientists who find flaws in the research (perhaps overlooked by the peer reviewers) can
publish letters, commentaries, or competing studies.

11. How does the “Mozart effect” example demonstrate how media coverage can
misrepresent science when they write for a popular audience.
In 1993, researcher Frances Rauscher found that when students heard Mozart music
played for 10 minutes, they performed better on a subsequent spatial intelligence test
when compared with students who had listened to silence or to a monotone speaking
voice (Rauscher et al., 1993). Rauscher said in a radio interview, “What we found was
that the students who had listened to the Mozart sonata scored significantly higher on
the spatial temporal task.” However, Rauscher added, “It’s very important to note that
we did not find effects for general intelligence . . . just for this one aspect of intelligence.
It’s a small gain and it doesn’t last very long” (Spiegel, 2010).

But despite the careful way the scientists described their results, the media that reported
on the story exaggerated its importance:

The headlines in the papers were less subtle than her findings: “Mozart makes
you smart” was the general idea. . . . But worse, says Rauscher, was that her very
modest finding started to be wildly distorted. “Generalizing these results to
children is one of the first things that went wrong. Somehow or another the myth
started exploding that children that listen to classical music from a young age will
do better on the SAT, they’ll score better on intelligence tests in general, and so
forth.” (Spiegel, 2010).

PSY 321 Reading Guide for Chapter 2

1. What is the Abstract section of an empirical journal article and what does it describe?
The abstract is a concise summary of the article. It briefly describes the study’s
hypotheses, method, and major results. When you are collecting articles for a
project, the abstracts can help you quickly decide whether each article describes the kind
of research you are looking for, or whether you should move on to the next article.
2. What does the Introduction section include and where can each of these be found in this
section?
The introduction is the first section of regular text, and the first paragraphs typically
explain the topic of the study. The middle paragraphs lay out the background
for the research. What theory is being tested? What have past studies found? Why is
the present study important? Pay special attention to the final paragraph, which states
the specific research questions, goals, or hypotheses for the current study.

3. What does the Method section explain in detail? An ideal Method section provides
enough detail to do what?
The Method section explains in detail how the researchers conducted their study.
It usually contains subsections such as Participants, Materials, Procedure, and
Apparatus. An ideal Method section gives enough detail that if you wanted to repeat
the study, you could do so without having to ask the authors any questions.

4. What does the Results section describe and provide?

The Results section describes the quantitative and, as relevant, qualitative
results of the study, including the statistical tests the authors used to
analyze the data. It usually provides tables and figures that summarize key
results. Although you may not understand all the statistics used in the article, you might
still be able to understand the basic findings by looking at the tables and figures.

5. What does the Discussion section include and where can each of these components be
found?
The opening paragraph of the Discussion section generally summarizes the
study’s research question and methods and indicates how well the results of
the study supported the hypotheses. Next, the authors usually discuss the
study’s importance: Perhaps their hypothesis was new, or the method they used was a
creative and unusual way to test a familiar hypothesis, or the participants were unlike
others who had been studied before. In addition, the authors may discuss alternative
explanations for their data and pose interesting questions raised by the
research.

6. What does the Reference section contain?

The References section contains a full bibliographic listing of all the sources the
authors cited in writing their article, enabling interested readers to locate these
studies. When you are conducting a literature search, reference lists are excellent places
to look for additional articles on a given topic. Once you find one relevant article, the
reference list for that article will contain a treasure trove of related work.

PSY 321 Reading Guide for Chapter 3

1. Study the following table.

Construct (conceptual Conceptual definition: Operational definition

variable): The name of A careful, theoretical (operationalization):
the concept being definition of the How the construct is
studied. construct. measured or
manipulated in an actual
study.
 “Satisfaction with life” “A person’s cognitive Five questionnaire items on
evaluation of his or her life” the Satisfaction with Life
(Diener et al., 1985) scale, answered on a scale of 1
(strongly disagree) to 5
(strongly agree). An example
item is “All in all, I am
satisfied with my life”.

“Perseverance” (in young “The ability to push through How long a child will choose
children) when confronted with . . to engage in a slow-paced,
.obstacles” (White et al., boring activity that involves
2017) pressing a button when they
see a picture of cheese on a
screen and not pressing it
when they see a cat on the
screen.

2. What is the difference between a variable and its levels?

A variable, as the word implies, is something that varies, so it must have at least
two levels, or values. Take this headline: “Most students don’t know when news is
fake.” Here, “knowing when news is fake” is the variable, and its levels are knowing when
news is fake, and not knowing when news is fake. Similarly, the study that inspired the
statement “Countries with more butter have happier citizens” contains two variables: per
capita butter consumption (whose levels might go from 0 kilograms per year up to 2
kilograms per year) and the level of happiness (with levels ranging from 1, least happy, to
10, most happy). In contrast, if a study concluded that “15% of Americans smoke,”
nationality is not a variable because everyone in the study is American. In this example,
nationality would be a constant, not a variable. A constant is something that could
potentially vary but that has only one level in the study in question. (In this example,
“smoking” would be a variable, and its levels would be smoker and nonsmoker.)

What might be the levels of the variable “favorite color”?

Red, yellow, blue, green, purple.

3. Explain why some variables can only be measured, not manipulated.

The researchers in any study either measure or manipulate each variable. The distinction
is important because some claims are tested with measured variables, while other claims
must be tested with both measured and manipulated variables. A measured variable
is one whose levels are simply observed and recorded. Some variables, such as
height and IQ, are measured using familiar tools (a ruler, a test). Other variables, such as
gender and hair color, are also said to be “measured.” To measure variables such as
depression and stress, researchers devise a special set of questions to represent the
various levels. In each case, measuring a variable is a matter of recording an observation,
a statement, or a value as it occurs naturally.

In contrast, a manipulated variable is a variable a researcher controls, usually

by assigning study participants to the different levels of that variable. For example, a
researcher might give some participants 10 milligrams of a medication, others 20 mg,
and still others 30 mg. Or a researcher might assign some people to take a test in a room
 with many other people and assign others to take the test alone. In both examples, the
participants could end up at any of the levels because the researchers do
the manipulating, assigning participants to be at one level of the variable or another.

Can “history of trauma” be a manipulated variable? Can “level of eye contact” be a

manipulated variable?
“History of trauma” is probably not a manipulated variable, but “level of eye contact”
might be manipulated if researchers assigned some people to make regular eye contact
with a conversation partner and other people to look away from a conversation partner.

4. What is the difference between a conceptual variable and the operational definition of a
variable?
Each variable in a study can be referred to in two ways. When researchers are discussing
their theories and when journalists write about their research, they use more abstract
names, called constructs or conceptual variables. When testing hypotheses with
empirical research, they create operational definitions of variables, also known as
operational variables, or operationalizations. To operationalize a concept of
interest means to turn it into a measured or manipulated variable.

How might the conceptual variables “level of eye contact,” “intelligence,” and “stress” be
operationalized by a researcher?
How long (in seconds) do participants maintain eye contact?
What is the score on a standard intelligence test?
What are participants’ ratings on a stress scale?

5. How many variables are there in a frequency claim? An association claim? A causal
claim?
A frequency claim describes a particular rate or degree of a single variable. An
association claim is about two variables, in which the value (level) of one variable is
said to vary systematically with the value of another variable. A causal claim argues that
a specific change in one variable is responsible for influencing the value of
another variable.

6. Which part of speech in a claim can help you differentiate between association and
causal claims?
Causal claims, however, go beyond a simple association between the two variables. They
use language suggesting that one variable causes the other — verbs such as cause,
enhance, affect, decrease, and change. In contrast, association claims use verbs such as
link, associate, correlate, predict, tie to, and be at risk for.

7. How are causal claims special, compared with the other two claim types?
Whereas an association claim merely notes a relationship between two variables, a
causal claim goes even further, arguing that one of the variables is
responsible for changing the other.

8. What three criteria must causal claims satisfy?

First, it must establish that the two variables (the causal variable and the
outcome variable) are correlated; the relationship cannot be zero.
Second, it must show that the causal variable came first and the outcome variable
came later.
Third, it must establish that no other explanations exist for the relationship.
9. Construct validity: How well the variables in a study are measured or manipulated.
External validity: The extent to which the results of a study generalize to some larger
population (e.g., whether the results from this sample of teenagers apply to all U.S.
teens), as well as to other times or situations (e.g., whether the results based on coffee
apply to other types of caffeine).
Statistical validity: How well the numbers support the claim—that is, how strong the
effect is and the precision of the estimate (the confidence interval). Also takes into
account whether the study has been replicated.
Internal validity: In a relationship between one variable (A) and another (B), the extent
to which A, rather than some other variable (C), is responsible for changes in B.

10. Study the following table.

Frequency claims Association claims Causal Claims

Usually based on a Usually supported Must be supported by
Type of Validity survey or poll, but by a correlational an experimental
can come from other study. study.
types of studies.

Construct How well has the How well has the How well has the
researcher measured researcher measured researcher measured
the variable in each of the two or manipulated the
question? variables in the variables in the
association? study?

External To what populations, To what populations, To what populations,

settings, and times settings, and times settings, and times
can we generalize this can we generalize this can we generalize this
estimate? How association claim? causal claim? How
representative is the How representative is the
sample? Was it a representative is the sample? How
random sample? To what representative are the
sample? other manipulations and
situations might the measures?
association be
generalized?

Internal Frequency claims do People who make Was the study an

not association claims are experiment? Does the
usually assert not asserting study achieve
causality, so internal causality, so internal temporal
validity is not validity is not precedence? Does the
relevant. relevant. A writer study control for
should alternative
avoid making a explanations by
causal claim from a randomly assigning
simple participants to
 association, however. groups? Does the
study avoid internal
validity threats?

Statistical What is the What is the estimated What is the estimated

confidence interval effect size: How effect size: How large
(margin of error) of strong is the is the difference
the estimate? Are association? How between groups?
there other estimates precise is the How precise is the
of estimate: What is the estimate: What is the
the same percentage? confidence interval? confidence interval?
What do estimates What do estimates
from other studies from other studies
say? say?

11. Study the following table.

Covariance The study’s results show that as A changes, B changes; e.g., high levels of
A go with high levels of B, and low levels of A go with low levels of B.

Temporal precedence The study’s method ensures that A comes first in time, before B.

Internal validity The study’s method ensures that there are no plausible alternative
explanations for the change in B; A is the only thing that changed.

Practice Activity: For each headline (a, b, and c) in Learning Actively #2 on p. 84, please do
the following:
1. Identify the type of claim being made
2. Based on the type of claim being made, which validities apply? (Hint: see table 3.6 for
quick reference on which validities will apply)
3. For construct validity applies, what questions would you ask? For example: How did they
operationalize “mood” and “focus”?
4. For external validity applies, what questions would you ask? For example: From what
populations did they sample “workaholics”?

PSY 321 Reading Guide for Chapter 7

1. Define population.
A population is the entire set of people or products in which you are
interested.
2. Define sample.
The sample is a smaller set, taken from that population.
3. What does the external validity of a study concern?
The external validity of a study concerns whether the sample used in the study is
adequate to represent the unstudied population.
4. Before researchers can decide whether a sample is biased, what do they have to specify?
Before researchers can decide whether a sample is biased or unbiased, they have to
specify a population to which they want to generalize: the population of
interest.
5. Samples are either biased or representative.
6. What is meant by a biased sample?
In a biased sample, also called an unrepresentative sample, some members of the
population of interest have a much higher probability than other members
of being included in the sample.
7. What is meant by an unbiased sample?
In an unbiased sample, also called a representative sample, all members of the
population have an equal chance of being included in the sample.
8. What are different ways to get a biased sample?
A sample could be biased in at least two ways: Researchers might study only those
they can contact conveniently or only those who volunteer to respond.
9. What is meant by a convenience sample?
Using a sample of people who are easy to contact and readily available to
participate.
10. What is meant by self-selection (NOT selection effects)?
A term used when a sample is known to contain only people who volunteer to
participate. Self-selection can cause serious problems for external validity.
11. What is meant by probability sampling? Does probability sampling help get an unbiased
or representative sample?
In probability sampling, also called random sampling, every member of the
population of interest has an equal and known chance of being selected for
the sample, regardless of whether they are convenient or motivated to volunteer.
Therefore, probability samples have excellent external validity and can generalize
to the population of interest.
12. What is the most basic form of probability sampling?
The most basic form of probability sampling is simple random sampling. To visualize
this process, imagine that the name of each member of the population of interest is
written on a plastic ball. The balls are rolled around in a bowl, and then a mechanism
spits out a number of balls equal to the size of the desired sample. The people whose
names are on the selected balls will make up the sample.
13. What is cluster sampling?
Cluster sampling is an option when people are already divided into arbitrary groups.
Clusters of participants within a population of interest are randomly selected, and then
all individuals in each selected cluster are used. If a researcher wanted to randomly
sample high school students in the state of Pennsylvania, for example, he could start with
a list of the 952 public high schools (clusters) in that state, randomly select 100 of those
high schools (clusters), and then include every student from each of those 100 schools in
the sample.
14. What is multi-stage sampling?
In multi-stage sampling, two random samples are selected: a random sample of clusters
and then a random sample of people within those clusters. In the high school example,
the researcher would start with a list of high schools (clusters) in the state and select a
random 100 of those schools. Then, instead of including all students at each school, the
researcher would select a random sample of students from each of the selected schools.
15. Distinguish between random sampling and random assignment.
In the context of research methods, it’s important not to confuse random sampling and
random assignment. With random sampling (probability sampling), researchers create a
sample using some random method, such as drawing names from a hat or using a
random-digit phone dialer, so that each member of the population has an equal chance
of being in the sample.
Random sampling enhances which validity?
Random sampling enhances external validity.
 What is random assignment?
Random assignment is used only in experimental designs. When researchers want to
place participants into two different groups (such as a treatment group and a comparison
group), they usually assign them at random.
Which validity does it help enhance?
Random assignment enhances internal validity by helping ensure that the comparison
group and the treatment group have the same kinds of people in them, thereby
controlling for alternative explanations.
16. Samples obtained through random selection achieve excellent external validity, but
such samples can be difficult to obtain.
17. Are larger samples more representative?
For external validity, is a bigger sample always a better sample? The answer may surprise
you: not really. The idea that larger samples are more externally valid than smaller
samples is perhaps one of the most persistent misconceptions in a research methods
course.

￭ Experiments (a study in which at least one variable is manipulated and another is measured)
are the only way to investigate causal relationships between variables. ￭ In the “Taking Notes”
study, the manipulated variable was the notetaking method (laptop/longhand). In the
“Motivating Babies” study, the manipulated variable was the model that the babies
watched. In the “Taking Notes” study, the measured variable was the student performance
on the essay questions. In the “Motivating Babies” study, the measured variable was how
many times each baby pressed the inert button. ￭ Independent variable (manipulated
variable) is called “independent” because the researcher has some independence in assigning
people to different levels (conditions) of this variable. An experiment must have at least one
manipulated (independent) variable. ￭ Dependent variable (measured variable) is called the
“dependent” variable because how a participant acts on the measured variable depends on the
level of the independent variable. ￭ Any variable that an experimenter holds constant on
purpose is called a control variable. Some of the control variables used in the Mueller and
Oppenheimer (2014) study were: People in both groups watched lectures in the same room and
had the same experimenter. They watched the same videos and answered the same questions
about them. Some of the control variables in the Leonard, et al. study (2017) were: The toys the
model was using. The model used the same cheerful, enthusiastic voice. The researchers also
kept constant how long the model demonstrated each toy (30 seconds), the gender of the model
(always female), the chair the infant sat in, and the cubicle where the experiment took place.
Control variables allow researchers to separate one potential cause from another and are
therefore important for establishing internal validity. ￭ The three rules for causal claims are:
(1) Covariance. Do the results show that the causal variable is related to the outcome variable?
Are distinct levels of the independent variable associated with different levels of the dependent
variable. (2) Temporal precedence. Does the study design ensure that the causal variable
comes before the outcome variable in time? (3) Internal validity. Does the study design rule
out alternative explanations for the results? ￭ The study on hugging dogs was flawed because
there was no comparison group. A control group is a level of an independent variable that
is intended to represent “no treatment” or a neutral condition. When a study has a control
group, the other level or levels of the independent variable are usually called the treatment
group(s). When the control group is exposed to an inert treatment such as a sugar pill, it is
called a placebo group. Not all experiments need a control group. However, all experiments
do need a comparison group. ￭ In experiments, the experimenter can make sure that the
causal (independent) variable comes first. Internal validity is one of the most important
validities to interrogate when you encounter a causal claim. For a study to be internally valid,
the study must ensure that the causal variable, and not other factors, is responsible
for the change in the outcome variable. Confounds are possible alternative explanations.
The word confound means confuse. ￭ When the kinds of participants in one level of the
independent variable are systematically different from those in the other, selection effects can
result. They can also happen when the experimenters let participants select which group they
want to be in. Well-designed experiments often use random assignment to avoid selection
effects. It is very unlikely that all the focused or fussy babies would have been clustered into the
same group in the baby study because assigning participants at random to different levels
of the independent variable (by flipping a coin, rolling a die, or using a random number
generator) controls for all sorts of potential selection effects. Random assignment creates
a situation in which the experimental groups will become virtually equal before the
independent variable is applied. ￭ To create matched groups from a sample of 30, the
researchers would first measure the participants on a particular variable that might matter to
the dependent variable. Student achievement, operationalized by GPA, for instance, might
matter in the study of notetaking. The researchers would next match up participants in pairs,
starting with the two having the highest GPAs, and within that matched set, randomly assign
one of them to each of the two notetaking conditions. They would then take the pair with the
next-highest GPAs and within that set again assign randomly to the two groups. They would
continue this process until they reach the participants with the lowest GPAs and assign them at
random too. ￭ In an independent-groups design (also called a between-subjects design
or between-groups design) separate groups of participants are placed into different levels of
the independent variable. Two basic forms of independent-groups designs are the
posttest-only design (participants are tested on the dependent variable once) and the
pretest/posttest design (participants are tested on the dependent variable both before and
after exposure to the independent variable). ￭ In a within-groups design, or
within-subjects design, each person is presented with all levels of the independent variable.
A repeated-measures design is one in which participants are measured on a dependent variable
more than once, after exposure to each level of the independent variable. In a
concurrent-measures design, participants are exposed to all the levels of an independent
variable at roughly the same time, and a single attitudinal or behavioral preference is the
dependent variable. The main advantage of a within-groups design is that it ensures the
participants in the two groups will be equivalent. Another advantage is that it requires fewer
participants overall.
____________________________________________________

Three types of measures researchers must choose between when operationalizing measured
variables: self-report, observational, and physiological. ◼ The conceptual definition,
or construct, of a variable is the researcher’s definition of the variable in question at a
theoretical level. The operational definition of a measured variable represents a researcher’s
specific decision about how to measure or manipulate the conceptual variable. ◼ Ed Diener
conceptually defined happiness as “subjective well-being” (well-being from a person’s own
perspective) and he operationally defined it by asking people to respond to a five item
questionnaire using a 7-point scale. ◼ The general process researchers go through to study
conceptual variables is that they first state a definition of their construct and then they create an
operational definition. ◼ A self-report measure operationalizes a variable by recording
people’s answers to questions about themselves in a questionnaire or interview. ◼ An
observational measure, sometimes called a behavioral measure, operationalizes a variable
by recording observable behaviors or physical traces of behaviors. ◼ A physiological
measure operationalizes a variable by recording biological data, such as brain activity,
hormone levels, or heart rate. ◼ No one method of operationalization is the best. Researchers
normally expect all three measures to show similar patterns of results. ◼ Categorical (also
called nominal) variables are categories and quantitative (also called continuous) variables are
coded with meaningful numbers. ◼ Reliability refers to how consistent the results of a
measure are. Validity refers to whether the operationalization is measuring what it is supposed
to measure. ◼ Reliability can be assessed in three ways: test-retest reliability (a study
participant will get pretty much the same score each time they are measured with it),
interrater reliability (consistent scores are are obtained no matter who measures the
variable), and internal reliability, also called internal consistency (a study participant gives a
consistent pattern of answers no matter how the researchers phrase the question). ◼
Researchers know if their indirect operational definition of a construct is really measuring that
construct (they evaluate a measurement tool’s validity) by collecting a variety of data and
evaluating it in light of their theory about the construct. ◼ To demonstrate the difference
between reliability and validity, consider the example of head circumference as an
operationalization of intelligence. Head size measurements are usually reliable because
circumference is easy to measure. However, head circumference is not related to intelligence.
Therefore, the head circumference test may be reliable, but it is not valid as an intelligence test:
It does not adequately capture the construct of intelligence.
____________________________________________________

◼ Replicable, or reproducible (will they get the same results if they conduct the same study
again). A finding from a study must be replicable to be considered credible because otherwise it
could just be a fluke. ◼ Three major types of replications: direct replication (researchers repeat
an original study as closely as they can to see whether the effect is the same in the newly
collected data), conceptual replication (researchers explore the same research question but use
different procedures), and replication-plus-extension (researchers replicate their original
experiment and add variables to test additional questions). ◼ Replication studies can help
interrogate external validity (the degree to which a study’s results are generalizable to other
participants and other settings). ◼ When the author states, “It’s A population, not THE
population” she means that the population to which researchers want to generalize usually is not
the population of every living person. Instead, when researchers are generalizing from a sample
to a population, they will specify what the population of interest is. It might be all the college
students in Manitoba. It might be all U.S. soldiers. It might be all the third graders in a Beijing
elementary school. It might be a group of lab-reared rhesus monkeys. Researchers are at liberty
to specify what their population of interest is based on the theories they are testing. ◼ When the
author states, “External validity comes from HOW, not HOW MANY” she means that when you
are assessing the generalizability of a sample to a population, a randomly selected sample of 200
participants has external validity; a haphazardly selected sample of 2,000 participants does not.

￭ Experiments (a study in which at least one V is manipulated and another is measured) are
the only way to investigate causal relationships between Vs. ￭ Independent V (manipulated V)
the R has some independence in assigning people to different levels (conditions) of this V. An
experiment must have at least one IV. ￭ Dependent V (measured V) how a P acts on the
measured V depends on the level of the IV. ￭ Control Vs (any V that an experimenter holds
constant on purpose) allow Rs to separate one potential cause from another and are important
for establishing internal validity. ￭ Three rules for causal claims are: (1) Covariance. Do the
results show that the causal V is related to the outcome V? Are distinct levels of the IV associated
with different levels of the DV? (2) Temporal precedence. Does the study design ensure that
the causal V comes before the outcome V in time? (3) Internal validity. Does the study design
rule out alternative explanations for the results?

￭ Study on hugging dogs was flawed because there was no comparison group. ￭ Control
group (a level of an IV that is intended to represent “no treatment” or a neutral condition).
When a study has a control group, the other level or levels of the IV are usually called the
treatment group(s). Placebo group (when the control group is exposed to an inert
treatment such as a sugar pill). Not all experiments need a control group. However, all
experiments do need a comparison group. ￭ The experimenter can make sure that the causal
(independent) V comes first. Internal validity is one of the most important validities to
interrogate when you encounter a causal claim. For a study to be internally valid, the study must
ensure that the causal V, and not other factors, is responsible for the change in the
outcome V. Confounds (meaning confuse) are possible alternative explanations.

￭ Selection effects (when the kinds of Ps in one level of the IV are systematically different
from those in the other). They can also happen when the experimenters let Ps select which group
they want to be in. Well-designed experiments often use random assignment to avoid selection
effects. It is very unlikely that all the focused or fussy babies would have been clustered into the
same group in the baby study because assigning Ps at random to different levels of the IV (by
flipping a coin, rolling a die, or using a random number generator) controls for all sorts of
potential selection effects. Random assignment creates a situation in which the
experimental groups will become virtually equal before the IV is applied. ￭ To create
matched groups from a sample of 30, the Rs would first measure the Ps on a particular V that
might matter to the DV. The Rs would next match up Ps in pairs and within that matched set,
randomly assign one of them to each of the two conditions.

￭ Independent-groups AKA between-subjects or between-groups design (separate groups of

Ps are placed into different levels of the IV) > (1) posttest-only design (Ps are tested on the
DV once) and (2) pretest/posttest design (Ps are tested on the DV both before and after
exposure to the IV). Three reasons to include a pretest: allows R to evaluate whether random
assignment made groups equal on variables highly related to variable of interest, allows R to
measure change in DV, allows Rs to measure mortality rates (P dropout rates). Disadvantages: It
is more time consuming to include a pretest and pretest may impact Pts’ behavior in a way that
threatens internal validity of study. ￭ The Solomon Four Groups design is used to determine if
pretest changed Ps behavior. In this type of design, Rs randomly assign ½ of the Ps to take
pretest and posttest and ½ to only take posttest. Keeping everything else the same for
Experiment 1 and Experiment 2. Then compare posttest performance between Experiment 1 and
Experiment 2. If difference between groups on posttest same for both experiment, then pretest
likely not affecting performance. If difference between groups on posttests differ between
Experiments 1 and 2, then pretest is likely affecting performance.

￭ Within-groups AKA within-subjects design (each person is presented with all levels of the
IV) > (1) repeated-measures design (Ps are measured on a DV more than once, after
exposure to each level of the IV) and (2) concurrent-measures design (Ps are exposed to all
the levels of an IV at roughly the same time, and a single attitudinal or behavioral preference is
the DV). Advantages of a within-groups design: (1) it ensures the Ps in the two groups will be
equivalent and (2) it requires fewer Ps overall. Disadvantages of a within-groups design: (1)
order effect (when exposure to one level of IV changes P’s behavior in subsequent levels of IV: a)
Fatigue effect (deterioration in performance; P becomes tired, bored, distracted), b) Practice
effect (improvement in performance as result of repeated practice with task), c) Contrast effect
(response to second condition in experiment altered by going through first condition)) - To
address order effects, Rs can counterbalance the order of the conditions, (2) demand
characteristics (when Ps guess what the study is about and behave differently than they would
have without the guess), and (3) within-groups design might not be possible.

￭ Three types of measures Rs must choose between when operationalizing measured V:

self-report (recording people’s answers to questions about themselves in a questionnaire or
interview), observational, sometimes called a behavioral measure (recording observable
behaviors or physical traces of behaviors), and physiological (recording biological data, such
as brain activity, hormone levels, or heart rate). ￭ The conceptual definition, or construct,
of a V is the R’s definition of the variable in question at a theoretical level. The operational
definition of a measured V represents a R’s specific decision about how to measure or
manipulate the conceptual V. ￭ The general process Rs go through to study conceptual Vs is that
they first state a definition of their construct and then they create an operational definition. ￭ No
one method of operationalization is the best. Rs normally expect all three measures to show
similar patterns of results. ￭ Categorical (AKA nominal Vs) are categories and quantitative
(AKA continuous) Vs are coded with meaningful numbers. ￭ Reliability (how consistent the
results of a measure are): (1) test-retest reliability (a study P will get pretty much the same
score each time they are measured with it), (2) interrater reliability (consistent scores are are
obtained no matter who measures the V), and (3) internal reliability, also called internal
consistency (a study P gives a consistent pattern of answers no matter how the Rs phrase the
question). Validity (whether the operationalization is measuring what it is supposed to
measure). First step is reliability: you cannot have a valid measure that is not reliable; you can
have a reliable measure that is not valid. ￭ Rs know if their indirect operational definition of a
construct is really measuring that construct (they evaluate a measurement tool’s validity) by
collecting a variety of data and evaluating it in light of their theory about the construct. ￭ The
difference between reliability and validity: consider the example of head circumference as an
operationalization of intelligence. Head size measurements are usually reliable because
circumference is easy to measure. However, head circumference is not related to intelligence.
The head circumference test may be reliable, but it is not valid as an intelligence test.

￭ Replicable, or reproducible (will they get the same results if they conduct the same study
again). A finding from a study must be replicable to be considered credible because otherwise it
could just be a fluke. ￭ Three major types of replications: direct replication (Rs repeat an original
study as closely as they can to see whether the effect is the same in the newly collected data),
conceptual replication (Rs explore the same research question but use different procedures),
and replication-plus-extension (Rs replicate their original experiment and add Vs to test
additional questions). ￭ Replication studies can help interrogate external validity (the degree
to which a study’s results are generalizable to other Ps and other settings). ￭ “It’s A population,
not THE population” (the population to which Rs want to generalize usually is not the
population of every living person. Instead, when Rs are generalizing from a sample to a
population, they will specify what the population of interest is). ￭ “External validity comes from
HOW, not HOW MANY” (when you are assessing the generalizability of a sample to a
population, a randomly selected sample of 200 Ps has external validity; a haphazardly selected
sample of 2,000 Ps does not).

￭ Face validity (the extent to which a measure is subjectively considered a plausible

operationalization of the conceptual variable in question). Criterion validity (an empirical
form of measurement validity that establishes the extent to which a measure is associated with a
behavioral outcome with which it should be associated). Predictive validity (the ability of a
measurement to predict a future outcome). Concurrent validity (measures how a new test
compares against a validated test).

￭ Survey types (phone/text, personal interviews, paper and pencil questionnaire, internet). ￭
Survey question types (open-ended, forced choice AKA multiple choice, rating scale). ￭
Potential threats to construct validity in survey research: question wording (leading questions:
questions that suggest an answer, double barreled questions: ask two questions in one but only
allow one response, double negatives), question order, response format issues, sensitive topics,
self-report measure).
GOOD LUCK!

Article on Racism in Psychological Research:

In his research published on 6/24/20, Steve O. Roberts found that prominent psychological
publications that highlight race are rare, and when race is discussed, it is authored mostly and
edited almost entirely by white scholars.
Roberts and his research team looked at more than 26,000 empirical articles published between
1974 and 2018 in top-tier academic journals for three major areas of psychology: cognitive,
developmental and social. The journals the researchers focused on were Cognition, Cognitive
Psychology, Child Development, Developmental Psychology, The Journal of Personality and
Social Psychology and Personality and Social Psychological Bulletin.
Role of editor-in-chief: In addition to managing the journal’s editorial board and recruiting
other scholars to assess submissions for a process known as peer review, the editor-in-chief
makes the final decision on what research is fit to publish.
To determine the race of the editors and the authors they publish, the researchers searched and
categorized photos they found of them online, which were often from their faculty webpage.
They then emailed the scholars to ask for their self-identified racial identity. This allowed the
researchers to ensure that their own assessments were accurate, which they were.
They found that of the 60 editors-in-chief between 1974 and 2018, 83 percent were white, 5
percent were people of color (POC) and 12 percent were unidentifiable because the researchers
were unable to code their race (for example, were deceased, retired or had no images online). Of
the publications with research that highlighted race, 87 percent were edited by white editors.
The data also revealed that racial makeup among editors-in-chief also varied among different
areas of psychology. For example, there has never been a POC editor in either of the journals
about cognitive psychology, a subfield that studies mental processes such as perception,
memory, thinking, attention and language. The data also revealed that an editor-in-chief’s race
predicted the publication of research that highlighted race. When editors were white, 4 percent
of all publications highlighted race. When they were POC, the publication rate almost tripled to
11 percent.Roberts’ data showed that from the 1970s to the 2010s, only 5 percent of publications
in the top-tier psychological journals he examined highlighted race. Here too, differences
emerged in different areas of the discipline – in cognitive psychology fewer than 0.01 percent of
publications in that subfield looked at race, compared with 8 percent in developmental
psychology and 5 percent in social psychology.
Roberts hypothesizes that cognitive psychologists may avoid race because of their pursuit to
study race-neutral, universal phenomena.“A handful of studies published in top-tier cognitive
journals make clear that cognitive processes, such as auditory processing, categorization and
memorization, do indeed vary as a function of racialized experiences,” Roberts explained. “To
not acknowledge this, or to only study human thinking with white participants, is a disservice to
science.”
Another explanation for the lack of racial diversity could be that white editors are subject to the
same reluctance that white people in general have when it comes to discussing race. Previous
research has shown that white people are more likely than persons of color to avoid
conversations about race. Either they don’t feel qualified enough on the issue to discuss it or
they want to project an image of color-blindness, said Roberts.
Among the publications that the researchers studied, they found that 63 percent of the paper’s
first authors – the persons typically responsible for conducting the research and writing the
manuscript – were white, 23 percent were POC and 14 percent were unidentifiable when it came
to race. The researchers next examined alternative explanations for why white authors might be
overrepresented in top-tier psychology journals. For example, they explored whether the
disparity might be explained by the quality of research. A measure academics use to assess
quality is a scholar’s citation score; that is, how often they are referenced by other academics.
Roberts found there was no difference between the citation counts of authors of color and white
scholars.
The researchers also wondered if the disparity could be explained by the quantity of the
researchers. If there are simply more white authors, then white authors should be
overrepresented across all journals. However, Roberts found that in lower-tier specialty
journals, white authors were actually underrepresented.
“The overrepresentation of white authors in top-tier psychology journals is not explained by the
quality of the research or by the quantity of the researchers. But it is explained by structural
racism,” said Roberts. “These journals are dominated by white psychologists, which has
implications for what and who is excluded from the scientific record.”
The first thing journals can do is communicate a top-down commitment to diversity, said
Roberts. This means explicitly stating whether the journal publishes research that is sensitive to
diversity and whether it values the editing, writing and participation of diverse scientists. Other
recommendations are to include diverse individuals across all levels of the publication process,
evaluate the diversity of research participants in the review process, release public diversity
reports annually and establish a diversity task force. Authors can make a change as well. Roberts
recommends that, for instance, authors detail and justify the racial demographics of their
research participants. Doing so is uncommon in mainstream psychology journals, says Roberts.

Chapter 4: Ethical Guidelines for Psychology Research

1. No matter what type of claim researchers are investigating, they are obligated by law, by
morality, and by today’s social norms to treat the participants in their research
with kindness, respect, and fairness.
2. What was the time frame of the Tuskegee Syphilis study? In the late 1920s and early
1930s
3. What were the demographics of the participants (race and health status)? 600 black
men: about 400 were already infected with syphilis and about 200 were not.
4. Researchers wanted to study the effects of untreated syphilis on the men’s
health over the long term.
5. As part of the study, what did the participants experience, what were they kept from
doing? They received no treatment. They were subjected to spinal taps. They
were kept from joining the armed forces.
6. Why did the study finally end? 1972
7. Over the course of the study, what happened to the participants and their families? Over
the course of the study, many of the men got sicker, and dozens died. Several
men inadvertently infected their partners, in some cases causing congenital
syphilis in their children.
8. The ethical violations of the Tuskegee Study fall into what three categories? They were
not treated respectfully, the men in the study were harmed, and the
researchers targeted a disadvantaged social group.
9. In what year was the Belmont Report created? Started in 1976 (completed in 1979)
By the request of whom? the U.S. Congress
10. What are the three main principles for guiding ethical decision making included in the
Belmont Report? Respect for persons, beneficence, and justice.
11. What are the two provisions of the principle of respect for persons? Individuals
potentially involved in research should be treated as autonomous agents:
They should be free to make up their own minds about whether they wish to
participate in a research study. Some people have less autonomy, so they are
entitled to special protection when it comes to informed consent. What must
researchers do to apply this principle? Every participant is entitled to the
precaution of informed consent: each person learns about the research
project, considers its risks and benefits, and decides whether to participate.
Children, people with intellectual or developmental disabilities, and
prisoners should be treated with special consideration.
12. According to the principle of beneficence, researchers must take precautions to protect
against what and ensure what? To protect participants from harm and to ensure
their well-being.
13. What must researchers assess to apply this principle? The risks and benefits of the
study they plan to conduct.
14. How did the Tuskegee Syphilis Study fail to treat the participants in accordance with the
principle of beneficence? They harmed participants through risky and invasive
medical tests, they harmed the participants’ families by exposing them to
untreated syphilis, and they withheld benefits from the men in the study.
15. Today, are researchers able to withhold treatments that are known to be helpful? No
16. What’s the difference between an anonymous study and a confidential study? In an
anonymous study, researchers do not collect any potentially identifying
information. In a confidential study, researchers collect some identifying
information but prevent it from being disclosed.
17. What does the principle of justice call for? A fair balance between the kinds of
people who participate in research and the kinds of people who benefit from
it.
18. How did the Tuskegee Syphilis study violate this principle? Anybody can contract
syphilis and benefit from research on it, but the participants in the study
were all poor African American men.
19. When the principle of justice is applied, it means that researchers must consider what?
The extent to which the participants involved in a study are representative
of the kinds of people who would also benefit from its results.
20. The APA outlines five general principles for guiding individual aspects of ethical
behavior. These principles are intended to protect not only research participants but
who else? Students in psychology classes and clients of professional
therapists
21. What are the five general ethical principles of the APA ethical guidelines (2002)? Be able
to define all 5. (Hint: Three of the five are from the Belmont Report; only two principles
are new) Fidelity and responsibility: a clinical psychologist teaching in a
university may not serve as a therapist to one of his or her classroom
students, and psychologists must avoid sexual relationships with their
students or clients. Integrity: professors are obligated to teach accurately,
and therapists are required to stay current on the empirical evidence for
therapeutic techniques. Beneficence and nonmaleficence: Treat people in ways
 that benefit them. Do not cause suffering. Conduct research that will benefit society.
Fidelity and responsibility: Establish relationships of trust; accept responsibility for
professional behavior (in research, teaching, and clinical practice). Integrity: Strive to
be accurate, truthful, and honest in one’s role as researcher, teacher, or practitioner.
Justice: Strive to treat all groups of people fairly. Sample research participants from the
same populations that will benefit from the research. Be aware of biases. Respect for
people’s rights and dignity: Recognize that people are autonomous agents.
Protect people’s rights, including the right to privacy, the right to give
consent for treatment or research, and the right to have participation
treated confidentially. Understand that some populations may be less able
to give autonomous consent, and take precautions against coercing such
people.
22. What does IRB stand for? Institutional Review Board
23. What is the IRB responsible for? Interpreting ethical principles and insuring that
research using human participants is conducted ethically.
24. Do all organizations that conduct research have to have an IRB? No. If an institution
uses federal money, it needs an IRB. Private research companies do not
need an IRB.
25. An IRB panel is composed of whom? (Number and backgrounds) Five or more
people. One must be a scientist, one must have academic interests outside
the sciences, one (or more) must be a community member that has no ties to
the institution. If prisoners are to be used in a study, one must be a prison
advocate.
26. Before conducting a study, researchers must do what? Fill out a detailed application
describing their study, its risks and benefits (to both participants and
society), its procedures for informed consent, and its provisions for
protecting people’s privacy. The IRB then reviews each application.
27. What must researchers include in their applications/proposals?
28. What type of research should IRBs not permit? Research that violates people’s
rights, research that poses unreasonable risk, or research that lacks a sound
rationale.
29. Ideally, IRBs should balance what? The welfare of research participants and the
researchers’ goal of contributing important knowledge to the field.
30. As part of informed consent, what is a researcher obligated to explain to potential
participants? To explain the study to potential participants in everyday
language and give them a chance to decide whether to participate.
31. When might informed consent not be necessary? If the study is not likely to cause
harm, involves a completely anonymous questionnaire, or takes place in an
educational setting, written informed consent might not be required.
Similarly, when the study involves naturalistic observation of participants in
low-risk public settings, such as a museum, classroom, or mall, people can
reasonably expect to be observed by others anyway.
32. Define deception. (Be sure your answer includes omission and commission) In some
cases, researchers withheld some details of the study from participants —
 deception through omission. In other cases, researchers actively lied to
participants — deception through commission.
33. Is deception always considered unethical? No. If not, what must be considered in
determining whether or not it is ethical? What are the ethical costs and benefits of
doing the study with deception, compared to without. If deception is used, must
participants be debriefed? Yes.
34. What is included in a debriefing? The researchers describe the nature of the
deception and explain why it was necessary.
35. Do non-deceptive studies also provide a debriefing session? Sometimes. If so, what is
the purpose? At many universities, student participants in research receive a
written description of the study's goals and hypotheses, along with
references for further reading. The intention is to make participation in
research a worthwhile educational experience, so students can learn more
about the research process in general, understand how their participation
fits into the larger context of theory testing, and learn how their
participation might benefit others. In debriefing sessions, researchers might
also offer to share results with the participants.
36. What are far reaching consequences of falsified data? When people fabricate data,
they mislead others about the actual support for a theory. Fabricated data
might inspire other researchers to spend time (and often grant money)
following a false lead or to be more confident in theories than they should
be.
37. Why might researchers falsify data? In many universities, the reputations,
income, and promotions of professors are based on their publications and
their influence in the field. In high-pressure circumstances, the temptation
might be great to delete contradictory data or create supporting data. In
addition, countering Merton’s norm of disinterestedness, some researchers
become personally invested in their own hypotheses and believe that any
data that do not support their predictions must be inaccurate.
38. Research misconduct violates what two current goals of psychological science?
Openness and transparency.
39. How does open data support scientific progress? Open data upholds Merton’s norm
of communality, which states that science belongs to everyone. Open data
supports scientific progress because other scientists can replicate published
work and test their own novel hypotheses.
40. What does it mean for scientists to report their process transparently? Describing all
measured variables and statistical analyses in the Method section.
41. Define plagiarism. Representing the ideas or words of others as one’s own.
42. How does a writer avoid plagiarism? You must cite the sources of all ideas that
are not your own, to give appropriate credit to the original authors.
43. Should researchers self-plagiarize? No. Why/why not? When researchers publish
multiple articles in a line of research, they may end up recycling portions of
the Method section from previous work. But they should not repeat
sentences verbatim in the introduction or Discussion section. In a Results
 section, recycling sentences is unacceptable because it means the article is
presenting previously published data as if it were new.

ALTHOUGH YOU WON’T BE TESTED ON IT (THIS CLASS FOCUSES PRIMARILY

ON HUMAN RESEARCH), PLEASE READ THE SECTION ON ANIMAL RESEARCH!
IT IS IMPORTANT!

Reading Guide for Chapter 11 (Exam 3)

Threats to internal validity: Design confound: Confusion about what is causing the
change in the dependent variable. Tx: Turn potential third variables into control variables.
Selection effects: Occurs in an independent groups design when the kinds of participants at
one level of the IV are systematically different from those at the other level. Tx: Use random
assignment or matched groups to equalize groups. Order effect: In a within-groups design,
exposure to one condition changes participant responses to a later condition. Tx:
Counterbalance the orders of presentation of the levels of the IV. One-group,
pretest/posttest design: Participants > Pretest > Treatment > Posttest (no comparison
condition). Maturation effect: When an observed change in an experimental group could
have emerged more or less spontaneously over time. Tx: Use a comparison group. History
threat: When it is unclear whether a change in the treatment group is caused by the treatment
itself or by an external or historical factor that affects most members of the group.Tx: Use a
comparison group. Regression to the mean/Regression threat: An extreme finding is
likely to be closer to its own mean the next time it is measured , because the same combination
of chance factors that made the finding extreme are not present the second time. Works at both
extremes (good and bad). Occurs only when a group is measured twice and only when the group
has an extreme score at pretest. Tx: Use a comparison group. Testing threat: A kind of order
effect in which scores change over time just because participants have taken the test more than
once. Tx: Have comparison group take the same two tests, use posttest only, use alternative
forms of the measure for pretest and posttest. Observer bias: When observer expectations
influence their interpretation of the results. Can be a threat to a study’s internal validity in
almost any study in which there is a behavioral DV. Control groups cannot control for observer
bias. Tx: Double-blind study. Demand characteristics: When participants guess what a study
is supposed to be about and change their behavior in the expected direction. Tx: Double-blind
study (Neither the participants nor the researchers who evaluate them know who is in the
treatment group and who is in the comparison group). Masked-design: Observers are
unaware of the experimental conditions to which participants have been assigned. Placebo
effect: People receiving an experimental treatment experience a change only because they
believe they are receiving a valid treatment. Responsible researchers consciously avoid
internal validity threats when they design and interpret their work.

Chapter 12: Experiments with More Than One Independent Variable

Interaction = a difference in differences = the effect of one independent variable depends on

the level of the other independent variable

1. Carefully read the description of the study by Duke et al. (2011) on alcohol and
aggression. What is the study’s IV? The Duke et al. (2011) alcohol and aggression
study’s IV was level of alcohol (placebo group vs. drunk group). DV? The DV
was aggression as measured by intensity and duration of a shock delivered.
Is it a between-subjects or within-subjects design? This was a between-subjects
design. What are the study’s results? The results demonstrated that alcohol
 causes people to behave aggressively.
2. Carefully read the description of the study by Strayer and Drews (2011) on the dangers of
distracted driving. What is the study’s IV? Cell phone use DV? Driving quality Is it a
between-subjects or within-subjects design? Within-subjects design What are the
study’s results? The results showed that when drivers were simply talking on
cell phones (not even texting or using apps), their reactions to road hazards
were 18% slower.
3. Adding an additional independent variable allows researchers to look for an interaction
effect (or interaction). What is an interaction? An interaction effect (or
interaction) is whether the effect of the original independent variable
depends on the level of another independent variable.
4. An interaction of two variables allows researchers to establish whether or
not “it depends.”
5. What is the mathematical way to describe an interaction between two independent
variables? Apply this to the ice cream, pancakes, hot, cold example. There is a
difference in differences.
6. Define factorial design. One in which there are two or more independent
variables.
7. What are independent variables referred to when conducting a factorial design? factors
8. Carefully review Strayer and Drews (2004) factorial design. What are the two
independent variables? cell phone use and driver age How many levels does each IV
have? Two Why is this design referred to as a 2 x 2 factorial design? How many cells are
in this study? There are two independent variables (two factors)—cell phone
use and age—and each one has two levels (driving while using a cell phone
or not; younger or older driver). This particular design is called a 2 × 2
(two-by-two) factorial design, meaning that two levels of one independent
variable are crossed with two levels of another independent variable. Since
2 × 2 = 4, there are four cells in this design.
9. What is meant by a participant variable? Age is an example of a participant
variable—a variable whose levels are selected (i.e., measured), not
manipulated.
10. For simplicity purposes, participant variables are often referred to as independent
variables/factors when conducting factorial designs. However, why aren’t participant
variables true independent variables? Because the levels are not manipulated,
variables such as age, gender, and ethnicity are not truly “independent”
variables. However, when they are studied in a factorial design, researchers
often call them independent variables for the sake of simplicity.
11. What is one reason mentioned in this section as to why researchers conduct studies with
factorial designs? To test whether an independent variable affects different
kinds of people, or people in different situations, in the same way.
12. Did the study by Strayer and Drews (2004) find the same relationship between cell
phone use and driving for young and old drivers? Yes Therefore, did the effects of cell
phone use on driving interact with, or depend on, age? No
13. Did Strayer and Drews (2004) find that the relationship between cell phone use and
driving generalized to young and old drivers? Yes
14. DeWall et al. (2010) tested whether the effect of alcohol intake on aggressive behavior
depends on body weight. Did they find an interaction between alcohol intake and body
weight? Yes In other words, did the effect of alcohol intake on aggressive behavior
depend on body weight? Yes
15. The process of using a factorial design to test limits is sometimes called testing for what?
testing for moderators
16. What is a moderator? A variable that changes the relationship between two
other variables. What does a moderator result in? An interaction The effect of one
independent variable depends on (is moderated by) the level of another independent
variable.
17. In the cell phone and age study by Strayer and Drew, did driver age moderate the impact
of cell phone use on braking onset time (driving performance)? No Did DeWall et al.
show that body weight moderates the effect of alcohol on aggression? Yes
18. The goal of most experiments in psychological science is to test hypotheses derived
from theories
19. Many theories make statements about how variables interact with one another.
20. The best way to study how variables interact is to combine them in a factorial
design and measure whether the results are consistent with the theory.
21. In a design with two independent variables, there will always be three results: 2 main
effects (one for each IV) and 1 interaction.
22. What is meant by a main effect? The overall effect of one independent variable
on the dependent variable, averaging over the levels of the other
independent variable. In other words, a main effect is a simple difference.
23. Is the main effect the most important effect in a study? No How might you think of a
main effect instead? The term main effect may be misleading because it seems
to suggest that it is the most important effect in a study. It is not. In fact,
when a study’s results show an interaction, the interaction itself is the most
important effect. Think of a main effect instead as an overall effect—the
overall effect of one independent variable at a time.
24. When results from a factorial design are plotted on a line graph and the lines are NOT
parallel (nonparallel), there may be what? An interaction This would have to be
confirmed by what? a statistical test
25. If the lines are parallel, there is probably NO what? interaction
26. Can you also estimate interactions from a bar graph? Yes
27. Interactions can be described using what phrases? “especially for,” “only for,” and
“depends on.”
28. When a study shows both a main effect and an interaction, what is almost always more
important? The interaction is almost always more important. How does the
study on alcohol, aggression, and body weight provide an example of this? Drunk men
that are heavy are the most aggressive of all.

Reading Guide for Chapter 14

1. What does it mean when a study is “replicable”? Why must a finding from a study be
replicable in order to be considered important? Replication (reproducible), a part of
interrogating statistical validity, tells us whether the results were a fluke or not.
2. What are the three major types of replications? Be familiar with the definition of each. In
direct replication, researchers repeat an original study as closely as they can. In
conceptual replication, researchers explore the same research question but use
different procedures. In a replication-plus-extension study, researchers replicate
their original experiment and add variables to test additional questions.
3. Why do researchers do “direct” replications? To see whether the effect is the same
in the newly collected data.
4. Be familiar with the two replication-plus-extension studies described on pp. 428 – 429
that further explored the relationship between type of notetaking during a lecture
(long-hand vs. laptop) on exam performance. Longhand note takers scored a bit
higher on factual questions than laptop notetakers. Laptop note takers
scored higher than longhand note takers on conceptual questions.
5. What is the replication crisis? Replications were not showing consistent results
with the original study.
6. What are some of the reasons proposed for why replication studies might fail? Some
psychological effects are replicable, others are not. It could be an issue with
the replication study (differences) or with problems of the original study.
7. The Open Science Collaboration attempt to replicate studies showed that about 39% of
the studies replicated. The Many Labs Project showed that a replication rate of 85%.
What was different about these two projects that might explain this vast difference in
replication rates? Open Science Collaboration (100 studies x 100 different labs)
and Many Labs Project (16 studies x 36 labs and results were combined =
14/16 studies successfully replicated)
8. What improvements/changes have been introduced to address the problems identified in
the replication crisis? Review articles that summarize all of the literature in a
literature review. Meta-analysis that averages the results of multiple studies
(using both published and unpublished data to avoid the file drawer
problem).
9. What do responsible journalists do when reporting on scientific findings? They not
only report on the latest studies, they also give readers a sense of what the
entire literature says on a particular topic.
10. Read the section under “Generalizing to Other Participants” to review how a probability
sample allows researchers to generalize results of a study from sample to population of
interest. If a study is intended to generalize to some population, the
researchers must draw a probability sample from that population.
Probability sampling techniques can result in a representative sample; they
include simple random sampling, cluster sampling, multistage sampling,
stratified random sampling, oversampling, systematic sampling, and
combinations of these. All of them select people or clusters at random, so all
members of the population of interest are equally likely to be included in the
sample.
11. How do conceptual replications allow researchers to explore the external validity of a
scientific finding? When researchers test their questions using slightly
different methods, different kinds of participants, or different situations, or
when they extend their research to study new variables, they are
demonstrating how their results generalize to other populations and
settings. The more settings and populations in which a study is conducted,
the better you can assess the generalizability of the findings.
12. What is the difference between working in “theory-testing mode” and “generalization
mode”? In theory-testing mode, researchers design studies that test a theory,
leaving the generalization step for later studies, which will test whether the
theory holds in a sample that is representative of another population. In
generalization mode, researchers focus on whether their samples are
representative, whether the data from their sample apply to the population
of interest, and even whether the data might apply to a new population of
interest.
13. What do cultural psychologists focus on? Which mode do they work in? Cultural
psychologists work in generalization mode.
14. Who are WEIRD participants? WEIRD: Western, Educated, Industrialized, Rich,
Democratic