• Research Methods

in Psychology
 Josh and friends were discussing recent changes in the
DUI laws. Jasmin stated that anyone can drink a few beers
with no impact on their driving. Ben said a few beers
would impact Jasmin but not a big guy like him.

The discussion raised questions in Josh’s mind on just

how much beer it takes to impact driving. He then
wondered how to go about answering this question.

In this chapter we will explore how Josh might

explore the impact of drinking beer on driving.
 What is “Good” Research?
• Good research
– Is done in a systematic and deliberate manner
– Uses agreed upon procedures
– Can be repeated (replicated) by other
researchers

• The more you learn about research

methods, the more you will recognize how
much poorly designed research exists
 Theories
• A theory attempts to provide a framework for
studying some aspect of behavior

• Theories are abstract statements of relationships

that are designed to describe behavior in a very
broad manner

• For our study of alcohol consumption and driving,

our theory suggests a relationship between alcohol
and performance
 Hypotheses
• Hypotheses are specific tests of theories

• Hypotheses for our study include:

– Hypothesis 1 suggests that the more beer subjects drink the more
likely they will run into an orange cones in a driving experiment

- Hypothesis 2 suggests that those served “near beer,” will run into
fewer orange cones than those who drank real beer

- Hypothesis 3 suggests that there is no relationship between body

weight, alcohol consumed, and the number of cones hit
 Theories & Hypotheses
• Let’s try a metaphor:

• The apex of the umbrella represents a theory

• Each spoke of the umbrella represents a different

hypothesis

• Each spoke touches the apex (theory) and can also touch
the ground (real world) when you set the umbrella upright

• The apex (theory) of the umbrella never touches the

ground. It’s links to the ground are the hypotheses (spokes
of the umbrella)
 Hypotheses and Variables
• Hypotheses measure some form of behavior

• The things being measured are called variables

– Variables vary, they are not static

• Continuous variables involve amounts

– Temperature
– Blood alcohol level
– Body weight

• Categorical variables involve either/or circumstances

– Male/female
– Real beer/near beer
 Independent Variables
• Independent Variables (plural)

• Things the researcher manipulates (controls) as

part of a study

• For our experiment the researcher would control

– Amount of alcohol each subject consumed
– Spacing of the orange cones
– Type of vehicle used
– Weight of subjects
 Dependent Variables
• Dependent Variable (singular)

• The behavior being investigated by the researcher

• Not under the direct control of the researcher

• Only one dependent variable for each study

• In our study the dependent variable is the number of

orange cones hit by each subject

• The dependent variable must be able to fail

 Standardized Procedures
• For research to be effective subjects need to
be treated in the same way as much as
possible

• For example:
– Same model of car in the driving experiment
– Same kind of alcohol (beer/wine)
– Same distance between orange cones
 Samples
• Since it is impossible to include everyone in a
given experiment, researchers use a “sample” of
people who were selected to represent the
“population” their drawn from

• For example, a small group of students serve as a

“sample” of all the students from a university

• Professors could not be in a sample of students as

they are not from the same population (students on
campus)
 Internal Validity

• Internal Validity addresses the way a study

is constructed

– The design of the study must be sound

– If the researcher didn’t record how much

alcohol students consumed then the study’s
internal validity would be poor
 External Validity
• External validity refers to he ability to generalize
beyond the sample

• If we used 18-20 year old college students in our

study is it reasonable to generalize our findings to:

• 18-20 year old college students from across the USA

• 18-20 year old college students from anywhere in the world

• But not to a group of 40 + old alcoholics in a treatment center

 Objective Measures
• Objective measures are designed to control for the
researcher’s inherent bias

• Let’s look at two examples of poor objectivity. Which of

the following questions was submitted for a survey by the
NRA and which one by Handgun Control:

– Do you support our Constitution’s Bill of Rights , including the 2nd

Amendment’s Right to Bear Arms?

– Do you support anyone having access to guns, including convicted

criminals, drug addicts, and child molesters?
 Reliability of Measures
• Reliability refers to how consistently a measure
comes up with similar results

• Professors normally use several measurements to

ensure they have a reliable estimation of each
student’s performance
• Those who do well on exams usually write excellent papers

• A measure is unreliable if we cannot depend on it

to produce similar results over repeated trials
 Test-Retest Reliability

• Administer a test this week

• Administer the exact same test a week later

• The scores should be similar, not exact, but

similar
 Internal Consistency

• Internal consistency involves consistency of

answers across several items

• Do you like mushrooms?

• Do you like mushrooms with ketchup?
• Do you like mushrooms on pizza?

• (If you don’t like mushrooms, then you should

answer no to the other questions)
 Interrater Reliability
• Several individuals rate a variable on a given scale

• Three raters stand at an intersection and record subjects estimated age

– 0-30 years old
– 30-60 years old
– 60 + years old

• If two raters estimate a subject as being 30-60, and two rate the subject
as over 60, we have poor interrater reliability

• Often used in naturalistic observation studies

 Validity
• Validity addresses whether a tool is measuring
what it is suppose to measure

• Once you establish than a measure is reliable then

you determine if it is valid

• A researcher never addresses whether a measure is

valid until is first demonstrated that it is reliable

• You might say we have lots of RVs in the world

of research but not a single VR
 Validity

• Suppose when you applied for a driver’s license you

were measured on the following:

• Drive around the block and not hit anything

• Park the car between two orange cones

• Take a paper and pencil test on street signs

• Take a shorthand test

 Shorthand test to drive a car?

• Reliability
– Very good reliability as everyone consistently
fails the shorthand test

• Validity
– Poor validity as shorthand has nothing to do
with driving a vehicle
 Forms of Validity
Face Validity:

– Does a question look like it belongs on a

measure

– Shorthand test to drive a car would have very

poor face validity
 Forms of Validity
Construct Validity:
– Convergent
• A measure correlates with similar tools designed to
measure the same phenomena under investigation

– Discriminate Validity
• The measure correlates poorly with tools of an
unrelated nature
 Forms of Validity
Criterion
– A measure’s ability to accurately differentiate
among those in a sample

• Difficult, but fair, final examination has good

criterion validity

• An easy exam that everyone passes easily has poor

criterion validity
 Error
• Studies involving humans are never 100% accurate

• The difference between the actual outcome and 100% is

referred to as “error”

• Any measurement of humans- Measurement & Error

• The lower the error rate the better the measurement tool

• Good research often involves the use of multiple measures

in order to reduce the degree of error
 Descriptive Research
Descriptive research describes phenomena as it exists in the
“real world”

Case Study

• In-depth observation of one person (or small group)

• Difficult to generalize results beyond the person under investigation

• Interpretive approach involves trying to understand why this one

person did what they did (hit their partner)

• Very susceptible to researcher bias

 Naturalistic Observation
• In-depth observation of a phenomena in its natural setting

– Observe how many people wear a team jersey on game day

• No contact with subjects

– Researchers count how many people wear jerseys and how many
do not

• Can only record “what is” and cannot ask the subjects
“why”

• Researcher try to not be noticed by those being observed

 Survey Research

• Ask subjects questions

– In person, paper and pencil, on line

• A key issue is the honesty of responses

– Asking your peers how many beers they drank as they
are getting in their cars to drive home

• Did the intended person actually competed the

survey
– A survey for a CEO is delegated to a subordinate of the
CEO
 Survey Research
• Surveys often use samples from a given
population

• Stratified Random Sample

– Breaks up sample by some criteria
– 40% of sample to be male
– 20% males over 50 and 20% males under 20
 Mean, Median, Mode
• Mean • Scores:
– Add up all scores, divide total – 10,10,10,9,9,11,11=70
by number of scores and you
have the arithmetic mean for a • Mean
group of scores – 70/7=10

• Median • Median
– Have the scores in a sample are – 10
above the median score and – 2 scores over 10
half are below it – 2 scores under 10

• Mode • Mode
– Most frequent score in a – 10 (Three 10s)
sample of scores
 Range and Standard Deviation
• Range
– The difference between the lowest and highest
scores in a sample

• Standard Deviation
– The amount a given score deviates from the
mean of a sample
 Experimental Research
• Involves manipulation of independent variables in
order to determine the impact on a dependent
variable

• Designed to confirm a causative connection

– Doing this causes that

• In our hypothetical experiment:

– # beers given to subjects & spacing of cones-
Independent variables
– # of orange cones hit- Dependent variable
 Steps in an Experiment
• Form a hypothesis
– Beer and hitting cones

• Operationalize Variables
– Brand of beer, drive through cones, type of car, etc.

• Develop a standard procedure

– Control Group (near beer)
– Experimental Group (real beer)
– Demand Characteristics (same person talks to subjects)
– Blind Studies (Subjects won’t know if real or near beer)

• Select and Assign Participants

 Experimental Research
• Apply Statistical Analyses of Data
– Descriptive
– Inferential
• Probability Theory

• Draw Conclusions
– Be very conservative
– In this study beer consumption appears to impact
driving ability
– Suggest further research efforts
 Correlational Research
• Explores the degree to which two variables are
related

• Correlational Coefficient=
– The statistic used to compare to variables
– Range: -1.0---1.0

• Correlation does not equal Causation

– Even if two variables have a causative relationship,
correlational research doesn’t check for it
 Correlational Research
• Positive Correlation
– Two variables move in same direction
• Caloric intake and body weight

• Negative Correlation
– Two variables move in opposite directions
• Exercise and body fat

• No correlation
– Two variables demonstrate no pattern of predictable
movement
• Height and SAT scores
 Evaluation a Research Study
• Does the theoretical framework make
sense?
• Is the sample adequate and appropriate?
• Are the measure and procedures adequate?
• Are the data conclusive?
• Are the broader conclusions warranted?
• Does the study say anything meaningful?
• Is the study ethical?
 The Conference
• Josh eventually finished his research study
that involved:
– 200 subjects
– 100 male & 100 female subjects
– Budweiser beer in cans and near bear
– Beer was transferred to unmarked glasses
– The person handing out the beers didn’t know
if it was real beer or near beer
– Controlled for subjects weights
– Used a common access Ford Mustang
 The Conference
• Josh wrote his research paper, ran it past his
professor and then got accepted at an academic
conference on substance abuse

• Academic conferences serve as an opportunity for

colleagues to review one’s research before you
attempt to get in published in a journal

• Here is a sample of the kind of questions Josh

might expect
 Conference Questions
• Bob Wright:

– As Budweiser beer is brewed in numerous locations across the

country, with each location being a different age, different (more
or less modern) brewing equipment, and having different water
sources, did you check the beer cans to ensure they came from the
same brewery?

• Jan Smith:

– Did you use a medical scale or some inexpensive discount house

scale to measure the students weights? If the later, how much
variance between the recorded weights and the subjects actual
weights might exist?
 Conference Questions
• Jacque LeSeur:
– You stated that you used a Ford Mustang? Was
this the new model Mustang or one of the
original Mustangs? This is important as the two
models have very different wheel bases.

What else did Josh miss???????