SECTION: RESEARCH QUESTION

What are the IV and DV in the following research question: “Does Cannabis impair
cognitive function?”

a) IV: Cognitive function

DV: Cannabis use

b) IV: Cannabis use; DV

Cognitive function

What are the IV and DV in the following research question: “Do people buy more on
landing page A compared to landing page B?”

a) IV: Landing page A

DV: Landing page B

b) IV: Landing page (A, B)

DV: Sales

c) IV: Sales
DV: Landing page (A, B)

Imagine a world in which every person has blue eyes. In this world, would it make
sense to think of eye color as a variable?

a) Yes, everything can be thought of as a variable.

b) No, because eye color does not vary in this world.

Look at the research question below. Is the IV in this research question numerical or
categorical?

“Do people buy more on landing page A compared to landing page B?”

a) Numerical.

b) Categorical.

c) It can be both.

Look at the research question below. Is the DV in this research question numerical
or categorical?

1
“Do older people get promoted more often than younger people?”

a) The DV is age, which is numerical.

b) The DV is age, which is categorical.

c) The DV is promotion (yes, no), which is categorical.

d) The DV is promotion (yes, no), which is numerical.

2
 SECTION: RESEARCH DESIGN

In a real study conducted in New York city, researchers found that murder rates were
higher during times when people buy more ice-cream. How do you explain this?

a) There is only one possible explanation: Ice-cream causes people to commit murder.

b) There is only one possible explanation: Committing murder causes people to crave
more ice-cream.

c) Both explanations above are consistent with the results. However, another
possible (and potentially more likely) explanation is that a third variable
caused both higher murder rates and more ice-cream consumption. For
example, it could be that hot weather increased the likelihood of both murder
and ice-cream consumption.

Imagine that a survey showed that people with a higher education level have a lower
chance of being overweight. What can you conclude from this?

a) There are three possible explanations.

Explanation 1: Education causes people to be less overweight.
Explanation 2: Being less overweight causes people to be more educated.
Explanation 3: A third variable (e.g., strictness of parenting) causes people to
be more educated and less overweight.

b) Education caused people to be less overweight. Explanations 2 and 3 can be ruled

out in this case.

c) Being overweight caused people to be less educated. Explanations 1 and 3 can be

ruled out in this case.

What is the difference between an experiment and a non-experiment?

a) In an experiment, the dependent variable is manipulated by the experimenter

(through randomization).
In a non-experiment, the dependent variable is not manipulated by the
experimenter (through randomization).

b) In an experiment, the independent variable is manipulated by the experimenter

(through randomization).
In a non-experiment, none of the variables is manipulated by the experimenter.

c) In an experiment, all variables are manipulated by the experimenter (through

randomization).
In a non-experiment, none of the variables is manipulated by the experimenter.

3
A researcher is interested in the relationship between amount of sport and academic
performance. Therefore, the researcher recruits 100 participants and randomly
divides them into two groups: 50 participants are asked to do a mild workout every
day and 50 participants are asked to do an intense workout every day. The
researcher wants to compare the academic performance between the two groups
after a few weeks of having followed these workout schedules. What type of design
is this researcher using?

a) Experimental between-subject design.

b) Experimental within-subject design.

c) Non-experimental between-subject design.

d) Non-experimental within-subject design.

A researcher shows pictures of men and women to participants and asks them to
judge how trustworthy the people on the pictures look. The results show that
participants judge women as more trustworthy. The researcher concludes that
people are prejudiced against men when judging trustworthiness from faces.
However, later it turns out that the women on the pictures smiled a bit more
compared to the men. So, it could be that difference in perceived trustworthiness
was not due to the perceived sex but due to the difference in smiling. What kind of
problem is this?

a) A third variable problem (Explanation 3).

b) A confounder.

c) A within-subject problem.

4
 SECTION: MEASUREMENTS

What is an example of an operational definition of aggression? Select the BEST

answer!

a) How angry a person looks according to observers.

b) The number of curse words a person uses.

c) How often a person interrupts another person.

d) ALL answers above could be seen as operational definitions of aggressions.

What are the two aspects of a good measurement?

a) Reliability and validity.

b) Objectivity and quantification.

c) Reliability and quantification.

d) Objectivity and validity.

A researcher wants to measure arithmetic skills and designs a couple of arithmetic

exercises for this purpose. However, it turns out that each person's performance
fluctuates a lot from exercise to exercise. What kind of problem is this and how was it
measured?

a) Problem: Low validity.

Measured through: Convergent validity.

b) Problem: Low validity

Measured through: Discriminant validity.

c) Problem: Low reliability.

Measured through: Test-retest reliability.

d) Problem: Low reliability.

Measured through: Internal consistency.

5
A researcher wants to measure arithmetic skills and designs a couple of different
arithmetic exercises for this purpose. However, it turns out that each person's
performance fluctuates a lot from exercise to exercise. What can the researcher do
to address this problem?

a) Use more arithmetic exercises and use the average performance as a

measurement of arithmetic skills.

b) Switch to self-report: ask people how good their arithmetic skills are.

c) There is nothing that can be done. It's better to change the research question.

What is one of the main problems of self-report measurements?

a) Subjectivity.

b) Reliability.

c) Social desirability.

6
 SECTION: DATA COLLECTION

Good data collection is...

a) Random, objective, and precise.

b) Random, independent, and objective.

c) Random, precise, and independent.

d) Random, independent, and large.

What is an example of data collection NOT being random?

a) Students have a higher probability of participating than non-students, because

the study was conducted on the university campus.

b) By coincidence, most of the participants turned out to be men with women being
under-represented.

A researcher conducted a study on voting preferences. What is an example of data

collection not being independent?

a) One participant convinced the other participants to vote for his candidate.

b) One participants convinced his friends who share his voting preference to also take
part in your study.

c) ALL answers above.

A researcher decides to randomly select 100 men and 100 women for a study. What
type of data collection is this researcher using?

a) Random sampling.

b) Stratified sampling.

c) Purposive sampling.

d) Convenient/ voluntary sampling.

A researcher selects 100 participants that fit a list of criteria (e.g., participants that
have a specific set of symptoms). What type of data collection is this?

a) Random sampling.

7
b) Stratified sampling.

c) Purposive sampling.

d) Convenient/ voluntary sampling.

8
 SECTION: DATA ANALYSIS 1

What are the three GENERAL steps in data analysis?

a) Compute averages, compute standard deviations, and compute cronbach's alpha.

b) Prepare the data, describe patterns in the data, and generalize to the
population.

c) Filter the data, aggregate the data, generalize the data.

A researcher wants to analyze the relationship between amount of protein eaten and
body weight. What approach would you recommend?

a) Scatterplots and correlations.

b) Boxplots to compare the groups/ conditions.

c) A contingency table with column percentages.

d) A residual plot.

A researcher wants to analyze the relationship between eye color (blue, brown) and
attractiveness (1-10) as judged by a group of independent observers. What
approach would you recommend?

a) Scatterplots and correlations.

b) Boxplots to compare the groups/ conditions.

c) A contingency table with column percentages.

d) A residual plot.

A researcher wants to analyze which of two landing pages makes a purchase most
likely. Each customer was randomly sent to one of the landing pages (page A, page
B) and then could either purchase or not (yes, no). What analysis approach would
you recommend?

a) Scatterplots and correlations.

b) Boxplots to compare the groups/ conditions.

c) A contingency table with column percentages.

d) A residual plot.

9
A researcher wants to analyze the relationship between amount of income and
purchase behavior (yes, no). What approach would you recommend?

a) Scatterplots and correlations.

b) Boxplots with the IV and DV switched.

c) A contingency table with column percentages.

d) A residual plot.

10
 SECTION: DATA ANALYSIS 2

Let's say that 1000 perfectly identical clones escaped from a (highly dubious)
research facility. Finding all 1000 clones would be a huge undertaking, so the
researchers first want to check whether the clones are dangerous at all. How many
clones do the researchers need to capture to get a representative sample?

a) They need all 1000 clones.

b) They need at least 100 clones.

c) They need at least 50 clones.

d) Actually, one clone would already be representative of the remaining clones.

What are the three main forces in statistical tests?

a) Effect size, variation, and sample size.

b) Correlation, mean difference, and percentages.

c) Random, independent, and large.

A researcher finds that participants in the treatment group are less ill compared to
participants in the control group with p = 0.02. What can the researcher conclude?

a) There is a true difference between the groups in the population (H1).

b) The difference between the groups does not exist in the population (H0).

c) The researcher cannot conclude anything because it could be that the sample size is
too small.

A researcher finds that participants in the treatment group are less ill compared to
participants in the control group with p = 0.67. What can the researcher conclude?
(Hint: think about the two common mistakes)

a) There is a true difference between the groups in the population (H1).

b) The difference between the groups does not exist in the population (H0).

c) It could be that the difference between the groups does not exist in the
population (H0) but it could also be that the sample size is simply too small.

11
A researcher finds in a representative sample that the older people are the grumpier
they are. The correlation for this relationship is r = 0.1 with p = 0.006. What can the
researcher conclude?

a) The correlation is small, so it probably does not generalize to the population (H0).

b) The correlation is small but reliable enough to generalize to the population

(H1).

c) The researcher cannot conclude anything because the sample size may be too
small.

12
 SECTION: SAMPLE SIZE & POWER

A researcher concludes that the treatment works but in reality the treatment does not
work. This is an example of a...

a) Correct positive

b) False positive

c) False negative

d) Correct positive

Let's say that you answered one of my quiz questions incorrectly even though you
are highly skilled at research. This is an example of a...

a) Correct positive.

b) False positive.

c) Correct negative.

d) False negative.

Let's say that a person believes that ghosts exist even though they do not exist. This
is an example of a...

a) Correct positive

b) False positive

c) False negative

d) Correct negative

Which two aspects in statistical tests can be influenced DIRECTLY by the user in
order to minimize false positives (type 1 errors) and false negatives (type 2 errors)?

a) Effect size and variation.

b) P value threshold and sample size.

c) Variation and sample size.

13
Let's say that the statistical power of your study is 95% and that there is a correlation
between the IV and the DV in the population (H1). What is the probability that you
will see a correlation with p < 0.05 in your sample?

a) 100%

b) 95%

c) 80%

d) 50%

What is the MINIMUM statistical power that is acceptable in a study?

a) 100%

b) 95%

c) 80%

d) 50%

14
 SECTION: INTERPRETING FINDINGS

What are the four steps in interpreting results?

a)
1. Is the effect size large enough?
2. Are measurements free of social desirability?
3. Is the design experimental?
4. Is the sample truly random?

b)
1. Reliability: p < 0.05?
2. Measurements: valid (and reliable)?
3. Explanations: explanations 1-3 and confounders.
4. External validity: sample and study setting representative?

A researcher finds that a treatment is effective in reducing depression with p = 0.001.

However, in follow-up analyses it turns out that the reliability of the depression
measurement was low with the correlation between first and second depression
measurement being r = 0.21. What does this mean for the effectiveness of the
treatment?

a) The results are not reliable, so the treatment did not truly work.

b) The results are not reliable, so nothing can be concluded from these findings.

c) The treatment probably worked given that p was smaller than 0.05. Low
measurement validity does not change this conclusion.

In a recent study on the consequences of cannabis use, researchers found that:

“Compared to non-users, regular cannabis users were [...] at higher risk of
depression and less likely to have a paid job"
(https://onlinelibrary.wiley.com/doi/abs/10.1111/dar.13239). What can you conclude
from these findings?

a) Cannabis use causes depression and unemployment.

b) One possible explanation is that cannabis use causes depression and

unemployment. However, there are other possible explanations. For example,
poor socio-economic conditions (or other third variables) may cause people to
use cannabis and also cause them to be more depressed and less likely to find
work (without any causal link between cannabis use and depression/
unemployment).

Suppose that you get a null-result (p = 0.93) and you are wondering whether the
nullhypothesis is true or whether the null-result is due to methodological problems.
What three steps should you follow?

15
a)
1. Check the sample size (did you have enough power?).
2. Check measurements and manipulations (are they reliable, valid, and
generally correct?).
3. Check your external validity (are sample and setting representative?).

b)
1. Check whether the results become significant if you remove outliers.
2. Check whether the results become significant if you add more participants.
3. Check whether the results become significant if you test men and women
separately.

A company wants to investigate which of two ads (old ad, new ad) is more effective
at increasing sales for their product. They apply the ads on multiple websites and
each time an algorithm randomly selects between the old and the new ad and then
displays the chosen ad. The results show that sales are higher for the new ad but the
difference is not significant (p = 0.63). A power analysis shows that the study with its
300 participants had 95% power for detecting a medium (but not a small) effect.
What can you conclude from this?

a) The new ad is more effective than the old ad.

b) The old and the new ad are equally effective given that the p value is larger than
0.05.

c) There are several explanations. One explanation is that the ads are equally
effective. Another possible explanation is that the new ad is slightly more
effective than the old ad but the study lacked enough power to detect this
small difference.

16