Data collection

Introduction
• Data collection means gathering
information to address those critical
research questions
• To plan data collection, you must
think about the questions to be
answered and the information
sources available.
 ,Cont

• you must begin to think ahead about

how the
• information could be organized,
analyzed, interpreted and then
reported to various audiences
• There are many methods available to
gather information, and a wide
variety of information sources.
 Pre-Data Collection Steps

1. Clearly define the goals and objectives of the

data collection
2. Reach understanding and agreement on
operational definitions and methodology for
the data collection plan
3. Ensure data collection (and measurement)
repeatability, reproducibility, accuracy,
and stability
 What kind of data should
?be collected
• The information you collect is the evidence
you will have available to answer the
evaluation questions.
• Poor evidence is information which cannot
be trusted, is limited, or simply is not
relevant to the questions asked.
• Good evidence is information that comes
from reliable sources
and through trustworthy methods that
address important questions
 There are two general types
:of information
• descriptive and judgmental.
Descriptive information
• can include the following examples:
• Characteristics of the project
• Reports of project accomplishments
• Current skill or knowledge levels of project personnel
and the target audience
• Amount of participation by the target audience
• Policies concerning cost share
• Rules regarding research
• Types of participants
• Demographic data
 Judgmental information
• Includes the following examples:
• Opinions from experts or consultants
• Consumer preferences
• Target audience’s beliefs and values
• Technical agency personnel’s
interpretation of laws
• Stakeholders perceived priorities
• Stakeholders interpretation of guidelines
 What methods should be
?used to collect data

The selection of a method for collecting

information must balance several
concerns including
• Resources available
• Credibility
• Analysis and reporting
• Resources
• And the skill of the evaluator
 Developing a data
collection plan
• Identify types of data needed for the study
• Select the types of measures to measure
each variable
• Select and/or develop instruments
• Secure written permission to use each
instrument
• Pilot test researcher-developed instrument
& revise plan
• Develop data collection forms and
procedures
• Implement data collection plan
 Identify types of data-1
needed for the study
1-Testing hypothesis or answering research
questions
2-Describe characteristics of sample
Demographics - age, gender, ethnic origin,
education background, marital status
Health-related variables - health habits, diet,
exercise, illness, length of illness,
3- Control for extraneous variables
Measure as many as possible
Intrinsic and extrinsic factors
(variables)
May want to see if main effects also
apply to
 .Cont
4- Analyze data
5-Interpretation of results
Consider results that do not fit desired
outcomes
Identify what data might explain
undesirable
Obtain administrative data
Records of subject ID=s
Data on number of times subjects
recruited or approached, dates of data
collection, length of time for data
collection, location, time of day, and who
collected data
Who, what, when, where, and how of data
collection
measures to measure each
variable

• Each variable must be measured - quantified

• Highly structured measurement takes time
and testing to develop, yet gives data easier
to analyze both descriptively and
inferentially
• Objective measurement of phenomenon
• Better to use multiple methods to measure
variables - self-reports, observations, and
physiologic measures
• Need to consider ethics, cost, time, staff,
and stress on pt./family &/or agency
personnel
 Select and/or develop-3
instruments
Identify existing instruments
Fit with conceptual definition of variable
Quality of instrument - validity & reliability
Validity:

Will the information collection methods you have designed produce

information that measures what you say you are measuring? Be
sure that the information you collect is relevant to the
evaluation questions you are intending to answer.
Reliability:

Will the evaluation process you have designed consistently

measure what you want it to measure? If you use multiple
interviews, settings, or observers, will they consistently measure
the same thing each time? If you design an instrument, will people
interpret your questions the same way each time?
 Areas problematic for
subjects

• Difficult to understand & reading

level too high
• Offensive terminology
• Sequencing of questions
•
 Self-reports
• Approaches to collect self report data
• Unstructured
• Semi- Unstructured
• Self report techniques
• Interviews
• verbal communication between research
and subject; commonly used in
exploratory and descriptive studies
Unstructured interview
• 1- subject's world view; open-ended
questions with probes and prompting;
qualitative; audiotape
• Semi-structured interview -
• 1 subject; list of topics or questions for
discussion with additional probes, aka
topic guide; uses both open
• and close-ended questions; taped
• Structured interview –
• 1 subject; specific questions asked in
• Consistent order using the same
words each time; no variation from
questions and no explanation of unclear
questions
Focus group interview
• 5 to 15 subjects in a group;
• Interviewer/moderator asks
open-ended questions; efficient
yet some individuals inhibited by
others in the group
•
• Gathering unstructured self report
data
• good listener good questioner; make
subject relaxed and open; maintain
focus on the topic; summarize and
close on a positive note
• Evaluate unstructured approach
data
• Explore new topics and experiences;
time consuming and demanding
Structured self report instrument
Types of closed-ended questions
1- Dichotomous items –
2 opposing choices; yes/no or true/false
2- Multiple choice -- >2 choices
from which subject selects 1 option
3- Rank-order - subject places
options in rank or order of importance
based on tool directions
4.Rating - Likert scale; semantic differential
scale
• Administering self-report instruments
• Interviews
• Put subject at ease
• Punctual, friendly, courteous,
respectful
• Privacy – obtain written consent
• Check tape recorder function before each
interview; bring extra batteries and tapes
• Remain neutral
• Use interview guide for consistency
 Advantages of interview

• Better response rates

• Wider range of subjects
• Can clarify questions
• Greater depth of questions
• Decreases missing information
• Order of questions controlled
• Adds supplementary observational
data
 Questionnaires
• Group or individuals – easy with group
• Personal contact with subjects or mailed
• Want response rate of at least 60% to
avoid response bias
• Cover letter stating completion and
return of questionnaire indicates
• consent of subject
• Can drop off and pick up in person or mail
• Mailing with follow-up plan –
questionnaire, postcard, questionnaire,
• postcard
• Advantages - Cheaper, anonymity, no
interviewer bias
• Response biases of subjects
• Social desirability – chooses
answer most socially acceptable
• Response set – items influence
the subjects response to other
items
• Extreme responses – selects
response on either extreme end of
poles
 Tips for wording the
question
• Clearly stated
• Bias decreased
• Sensitive information – respect subject
• Worded so that subjects can respond to questions
• Short sentences – ask only 1 question at a time; positive
statements
• Avoid double barreled questions
• Avoid leading questions; focus on research objectives and
questions
• Use closed-ended questions with socially unacceptable
topics
• Do not assume subjects are well informed about topic
• Impersonal wording of items without "I"
• Mutually exclusive item choices
 Observational methods
Gathering data through visual, auditory, tactile and other
senses
• Phenomenon
• Characteristics of individuals
• Verbal & nonverbal communication behavior
• Activities
• Skill attainment and performance
• Environmental characteristics
• Steps in observation
• Decide what to observe
• Determine how to conduct observation to ensure
every variable observed in same manner Orient and
provide written instructions to observers
• Validate accuracy and consistency of observation
techniques
 Observational Methods
• Unstructured observations
• No planning – spontaneous
• Observer freedom
• Risk loss of objectivity
• Requires excellent memory &
interpersonal skills
• Structured observations
• Define what is to be observed
• Define how observations will be
made, recorded, and coded
• Recording structured observations
•
• Participant observations – observer & subject
interact to some degree duration observation
• Overt – subject aware of observation
• Covert – subject not aware of
observation
• Non-participant observations –
observer and subject do not interact; can be
overt or covert
• Timing of observations
• Time sampling – making an observation
for a specified length of time at pre-determined
intervals (eg observe for 5 minutes every 15
minutes or observe for 5 minutes 3 randomly
selected times each hour)
•
• Event sampling – making an observation
whenever an event occurs
 Conclusion
The most important issue related to data
collection is selecting the most
appropriate information or
.evidence to answer your questions
To plan data collection, you must think
about the questions to be answered and
the information sources available. Also,
you must begin to think ahead about
how the information could be organized,
analyzed, interpreted and then reported
to various audiences
 References
• Glesne, C. and Peshkin, A. (1992). Becoming
qualitative researchers: An
• introduction. White Plains, NY: Longman
Publishing.
• Patton, M.Q. (1990). Qualitative evaluation and
research methods (2nd
• ed.). Thousand Oaks, CA: Sage Publications.
• Morse, J.M. (ed.) (1993). Critical issues in
qualitative research methods.Thousand Oaks, CA:
Sage Publications.
• Rubin, H.J. and Rubin, I.S. (1995). Qualitative
interviewing. Thousand,Oaks, CA: Sage
Publications.