Module 8

How to Construct Questionnaires

Questionnaire - A tool for collecting information to describe, compare, or explain an event or

situation, as well as, knowledge, attitudes, behaviors, and/or sociodemographic characteristics on a
particular target group.
Questionnaire General Format
● Self-administered (by mail or personal contact)
● In person (face-to-face)
● Telephone interviews
The Questions or Items
● Are the focus on any survey or questionnaire
● It is crucial to know how to ask the questions in written and spoken form
● The way you ask the questions determines the answers
Questions Context
● Identify the questionnaire’s specific ● Standardize the response format
purpose(s) ● Ask questions in a social, cultural, and
● Clarify the terms used to state the economic context
questionnaire’s purpose(s) ● Keep confidentiality and voluntary
● Be sure to have the specific objectives of the participation
questionnaire ● Include a letter of introduction or
● Know the respondents presentation
● Standardize the interviewer
Questions Format
● OPEN QUESTIONS - more information but difficult to codify, enter, and analyze.
● CLOSED QUESTIONS - less information but easy to codify, enter, and analyze.
Questions and Measurement of Responses
● Nominal or categorical choices
● Ordinal
● Numerical
How to Obtain Valid Information
● Ask purposeful questions ● Use time periods based on importance of the questions
● Ask concrete questions ● Use conventional language
How to Obtain Valid Information
● Use complete sentences ● Avoid two-edged questions
● Avoid abbreviations ● Avoid negative questions
● Review questions with experts and potential ● Adopt/adapt questions used successfully in
respondents other questionnaires
● Use shorter questions
Questionnaire Construction
While thinking what item or question to include in your questionnaire, you should ask yourself
the following:
1. Which objective is the question related to?
2. Is the question relevant to the aim of the study?
3. Is the question relevant to the research hypothesis?
4. Can the answer be obtained from other sources?

A diagram showing that during the research process, research objectives

are translated into specific questions

HOW TO CONSTRUCT A QUESTIONNAIRE

There are three fundamental stages that you should take in constructing your questionnaire:
1. Identifying the first thought questions.
2. Formulating the final questionnaire.
3. Wording of questions.
STAGE 1: IDENTIFYING THE FIRST THOUGHT QUESTIONS
1. you should go back to your proposal and the literature file.
2. start formulating the first thought list of questions (draft), write down all possible questions (say,
20-30 questions)
3. Whatever questions you intend to ask, they should not be arbitrary and need to be based on your
literature review
STAGE 2: FORMULATING THE FINAL QUESTIONNAIRE
You need to introduce a number of sections or categories for the questionnaire. Give these
sections a title or a theme which should correspond closely with the objectives. The first section of
the questionnaire was named General Information'. The second section was named 'Overall Business
Philosophy’
In Ex. (App. 3): he listed six issues that he thought important and relevant to ask. These six issues
were:
1. Type of business. 4. Marketing plan.
2. Business philosophy. 5. Management philosophy on the market.
3. Management philosophy on products. 6. Management success criteria.
STAGE 3: WORDING OF QUESTIONS
Some criteria for constructing the questionnaire:
1. Your questions/questionnaire should be 5. Avoid hypothetical questions
short but comprehensive. 6. The questions must not be ambiguous
2. Avoid leading questions 7. The questions should be logical in their
3. Avoid double questions sequence
4. Avoid presuming questions 8. The questionnaire must be attractive in
appearance
TYPES OF QUESTIONS
● The 'open' form or unrestricted type.
● The 'closed form or restricted type.
The main advantages of the open type of questions
1. They give the respondent the opportunity to express their views.
2. Easy to ask
3. More appropriate to construct with interview questionnaires
4. When sensitive information is required from the respondent.
The main advantage: more difficult for analysis and interpretation
Some considerations to determine appropriateness of which type of question to ask
1. The objective of the questionnaire.
2. The respondent's level of information about the topic in question.
3. The extent to which the topic has been thought through by the respondent.
4. The ease with which respondents can communicate the content of the answer or the extent to
which respondents are motivated to communicate on the topic.
Factual questions: Factual questions are mostly designed to elicit information related to the
following:
1. Facts related to the background of an individual or organization.
2. Facts related to events or projects.
Opinion questions (subjective measurement)
There are several formats for opinion questions can be asked:
1. Checklist. It is essentially a list of items that respondents are offered to mark or tick. It is a
straightforward means of collecting information and the data can be analyzed easily. Designed
for respondents who have accurate information, answers of questions can be with a high degree
of certainty. The checklist format is a quick format but can be rather rigid. The response to each
item does not have a degree of intensity, but a dichotomy.
2. Grid. A grid is an elaboration of the checklist format, except it provides answers to two or more
questions at the same time
3. Rating scale (Also Matrix Rating). It is one of the most common formats. Like the grid, except
the respondent has the choice to express his/her degree of agreement or disagreement on a
particular scale. Quantifiers: they reflect the intensity of the particular judgment involved.
4. Likert scale. This type of scaling is similar to the rating scale, except the questions consist of
attitudinal statements on the survey object
5. Numerical rating scale. The rating may run out of 100, out of 10, out of 5 and so on.
6. Ranking. To place a set of attitudes or objects in ranking order indicating their importance,
priorities or preferences.
7. Semantic differential scales. In this method, the respondent is asked to indicate his/her position
on a seven-point bipolar scale defined with contrasting adjectives at each end.
CRITERIA FOR CONSTRUCTING A QUESTIONNAIRE
The characteristics of a good questionnaire can be summarized as follows:
1. It must deal with a topic of some significance that is important enough to the respondent to
merit a response.
2. It must seek information not obtainable from other sources. You should not ask people to do your
data gathering for you especially when the data is readily available elsewhere.
3. It should be as short as possible but comprehensive enough to allow you to derive what you need
4. It should be attractive in appearance
5. Where it contains directions they must be clear and complete.
6. questions should be arranged in categories which allow easy and accurate responses.
7. Questions must be as objective as possible without offering leading questions.
8. In their sequencing, questions should run from the general to the specific, from simple to
complex, and from those that will create a favorable impression upon the respondent to those
that may be sensitive.
9. You should avoid questions that may annoy or embarrass the respondent.
10. The questionnaire must provide for ease of tabulation and/or interpretation, and should be
designed accordingly.
The design of the questionnaire also requires careful attention, and consideration should be
given to:
1. Examining other questionnaires
2. Obtaining as much help as possible in order to make sure it is easy to be understood by different
audiences.
3. Allowing for a pilot stage at the draft phase in order to get valuable responses and to detect
areas of possible shortcomings.
4. Connecting with the supervisor and make sure that he/she has seen and approved the
questionnaire.
5. Trying different question orders at the pilot stage. This may assist in finding the best approach.
6. Experimenting with different types of questions, both open and closed.
7. In order to obtain the maximum number of respondents and collect the most relevant information
make sure:
a. You identify the sender and the person who receives the returned questionnaires
b. You know the purpose of the research and the questionnaire
c. There is an incentive for the respondent to complete the questionnaire.
d. You reassure the respondent of the confidentiality of the questionnaire.
e. You allow the respondent the opportunity to provide some personal insight not contained
elsewhere.
When trying to design the questionnaire, you need to ask yourself a number of questions such
as:
1. Are you using quantitative or qualitative measures?
2. Is your study a comparative or a singular investigation?
3. If it is a comparative study, how many survey groups are you investigating and who are they?
4. How do the groups differ in their characteristics?
5. What is the depth of the investigation? Is it a specific problem or a generalized one?
6. How many responses or cases do you need in order to answer your questions and achieve your
objectives?
7. If you are testing a theory, are you using control variables and why?

A Pilot Study - A pilot study provides a trial run for the questionnaire, which involves testing the
wording of the question, Identifying ambiguous questions, testing the technique that you use to
collect the data, and measuring the effectiveness of your standard invitation to respondents. You will
probably be amazed by how much you will learn from a pilot study. The little extra time it takes to
complete a pilot study test will be well spent.
Some Questions can be asked for reviewers:
1. How long did it take you to complete? 5. In your opinion, has any major topic been
2. Were the instructions clear? omitted?
3. Were any of the questions unclear or 6. Was the layout of the questionnaire clear
ambiguous? If so, will you say which and attractive?
why? 7. Any comments?
4. Did you object to answering any of the
questions?
Covering Letter - After constructing your questionnaire, your next step is to write a covering letter to
accompany your questionnaire. The letter should explain the purpose of the survey or questionnaire
in order to encourage a high response. It is important in postal questionnaires because it is the only
way you can persuade the subjects to respond to your questionnaire.

Module 9
How to Analyze Your Data

Qualitative Research - “Development of concepts which help us to understand social phenomena in

natural (rather than experimental) settings, giving due emphasis to the meanings, experiences and
views of the participants.”
Dimensions of Qualitative Methods
● Understanding context
- How economic, political, social, cultural, environmental and organizational factors influence
health
● Understanding people
- How people make sense of their experience of health and disease
● Understanding interaction
- How the various actors involved in different public health activities interact each other
Qualitative vs. Quantitative: Basic Differences
Qualitative Quantitative

Purpose To describe a situation, gain To measure magnitude- how

insight to particular practice. widespread is a practice.

Format No pre-determined response Pre-determined response

categories categories, standard measures

Data In-depth explanatory data Wide breadth of data from

from a small sample large statistically
representative sample

Analysis Draws out patterns from Tests hypotheses, uses data to

concepts and insights support conclusion

Result Illustrative explanation and Numerical aggregation in

individual responses summaries, responses are
clustered

Sampling Theoretical Statistical

Qualitative vs. Quantitative: Analytic approaches

Qualitative Quantitative

Research Question Broader, contextual, flexible Fixed/Focused

Expected Outcome Usually not redefined, Identified in advance

emergent research question

Hierarchy of phases Circular Linearity

Confounding factors Searched in the field Controlled during design and

analysis

Time Dimension Rapid to Slower Slower

Qualitative vs. Quantitative: Data Collection Method

Qualitative Quantitative

Sampling Open ended and less structured Random Sampling

protocols (flexible)

Tools Depend on interactive Structured data collection

interviews instruments
Results Produce results that give Produce results that generalize,
meaning, experience and views compare and summarize

Important concepts in Designing Qualitative Research

Concept Description

Natural Setting Participants are free from any control and data are collected in their
natural environment

Holism The whole is more than the sum, take magnitude of contextual factors in
to account

Human as a research Researcher is involved in every step being responsive, flexible, adaptive
instrument and good listener

Emergent design Study design emerges as further insights are gained through data
collection and analysis

Saturation or A stage where additional interview or observation is not believed to add

redundancy new information-enough is enough

Common Qualitative Study Designs

Study Design Description

Ethnography Portrait of people-study of the story and culture of a group usually to

develop cultural awareness and sensitivity

Phenomenology Study of individual’s lived experiences of events – e.g. the experience of

AIDS care

Grounded Theory Going beyond adding to the existing body of knowledge-developing a

new theory about a phenomenon-theory grounded on data

Participatory action Individuals & groups researching their own personal beings, sociocultural
research settings and experiences

Case study In-depth investigation of a single or small number of units at a point (over
a period) in time. E.g. Evaluation of a service.

Sampling in Qualitative Research

● Aim - Purposive sampling-selection of the most productive sample to answer the research
question. Ongoing interpretation of data will indicate who should be approached, including
identification of missing voices
● Technique - To generate a sample which allows understanding the social process of interest
● Size - The one that adequately answers the research question-until new categories, themes or
explanations stop emerging from the data. Depend on available time and resources
Qualitative Data - Data are not easily reduced to numbers. Data that are related to concepts,
opinions, values and behaviors of people in social context. Transcript of individual interviews and
focus groups, field notes from observation of certain activities, copies of documents, audio/video
recordings.
Types of Qualitative Data
● Structured text, (writings, stories, survey comments, news articles, books etc.)
● Unstructured text (transcription, interviews, focus groups, conversation)
● Audio recordings, music
● Video recordings (graphics, art, pictures, visuals)

Qualitative Data Collection Methods

Methods Brief Collection

Observation The researcher gets close enough to study subjects to observe (with or
without participation) usually to understand whether people do what
they say they do, and to access knowledge of subjects

Interview This involves asking questions, listening to and recording answers from
an individual or group on a structured, semistructured or unstructured
format in an in-depth manner

Focus Group Discussion Focused (guided by a set of questions) and interactive session with a
group small enough for everyone to have chance to talk and large
enough to provide diversity of opinions

Other methods Rapid assessment procedure (RAP), free listing, pile sort, ranking, life
history (biography)

Questions for Qualitative Interviews

Types of Questions Examples

Hypothetical If you get the chance to be an HIV scientist, do you think you can
discover a vaccine for HIV?

Provocative I have heard people saying most evaluations are subjective - what do
you think?

Ideal In your opinion, what would be the best solution for eliminating
gender-based violence?

Interpretative What do you mean by good?

Leading Do you think prevention is better than cure?

Loading Do you watch what culturally degrading TV show on condom use?

Multiple Tell me your three favorite authors, the book you like best by each
author, and why you like those books?

Focus of Qualitative Questions

● Experience: When you told your manager that the project has failed, what happened?
● Opinion: What do you think about the role of evaluation for program improvement?
● Feelings: When you got to know that the project was a success, how did you feel?
● Knowledge: Tell me about the different ways of promoting PME?
● Input: When you have lectures on evaluability assessment, what does the instructor tell you?
Qualitative Data Analysis (QDA) - is the range of processes and procedures whereby we move from
the qualitative data that have been collected into some form of explanation, understanding or
interpretation of the people and situations we are investigating. QDA is usually based on an
interpretative philosophy. The idea is to examine the meaningful and symbolic content of qualitative
data.
Approaches In Analysis
● Deductive Approach - Using your research questions to group the data and then look for
similarities and differences. Used when time and resources are limited. Used when qualitative
research is a smaller component of a larger quantitative study
● Inductive Approach - Used when qualitative research is a major design of the inquiry. Using
emergent framework to group the data and then look for relationships
Points of focus in analyzing text data
● The primary message content
● The evaluative attitude of the speaker toward the message
● Whether the content of the message is meant to represent individual or group-shared ideas
● The degree to which the speaker is representing actual vs. hypothetical experience
Qualitative vs. Quantitative Data Analysis
Qualitative Quantitative
● Begins with more general openended ● Key explanatory and outcome variables
questions, moving toward greater precision identified in advance.
as more information emerges. ● Contextual/confounding variables identified
● Pre-defined variables are not identified in and controlled.
advance. ● Data collection and analysis distinctly
● Preliminary analysis is an inherent part of separate phases.
data collection. ● Analysis use formal statistical procedures.
Tools for helping the Analytical Process
● Summaries: should contain the key points that emerge from undertaking the specific activity.
● Self-Memos: allow you to make a record of the ideas which occur to you about any aspect of
your research, as you think of them.
● Research Diary
Terms used in Qualitative Data Analysis
● Theory: A set of interrelated concepts, definitions and propositions that presents a systematic
view of events or situations by specifying relations among variables.
● Theme: idea categories that emerge from grouping of lower-level data points.
● Characteristics: a single item or event in a text, similar to an individual response to a variable or
indicator in a quantitative research. It is the smallest unit of analysis.
● Coding: the process of attaching labels to lines of text so that the researcher can group and
compare similar or related pieces of information.
● Coding sorts: compilation of similarly coded blocks of labels of text from different sources in to a
single file or report.
● Indexing: process that generates a world list comprising all the substantive words and their
location within the texts entered in to a program.
Principles of Qualitative Data Analysis
1. People differ in their experience and understanding of reality (constructivist-many meanings).
2. A social phenomenon can’t be understood outside its own context (context-bound i.e. book is the
pen).
3. Qualitative research can be used to describe phenomenon or generate theory grounded on data.
4. Understanding human behavior emerges slowly and non-linearly.
5. Exceptional cases may yield insights in to a problem or new idea for further inquiry.
Features of Qualitative Data Analysis
● Analysis is circular and non-linear ● Level of analysis varies
● Iterative and progressive ● Uses inflection i.e. “this was good”
● Close interaction with the data ● Can be sorted in many ways
● Data collection and analysis is simultaneous ● Qualitative data by itself has meaning, i.e.
“apple”
The Process of Qualitative Data Analysis
● Step 1: Organize the data
● Step 2: Identify framework
● Step 3: Sort data in to framework
● Step 4: Use the framework for descriptive analysis
● Step 5: Second order analysis
Types of Qualitative Data Analysis
1. Content Analysis - Content analysis is the procedure for the categorization of verbal or
behavioral data for the purpose of classification, summarization and tabulation. The content can
be analyzed on two levels.
- Descriptive: what is the data?
- Interpretative: what was meant by the data?
2. Narrative Analysis - Narrative are transcribed experiences. Every interview/observation has
narrative aspect – the researcher has to sort-out and reflect up on them, enhance them, and
present them in a revised shape to the reader. The core activity in narrative analysis is to
reformulate stories presented by people in different contexts and based on their different
experiences.
3. Discourse Analysis • A method of analyzing a naturally occurring talk (spoken interaction) and all
types of written texts. Focus on ordinary people method of producing and making sense of
everyday social life: How language is used in every situations?
- Sometimes people express themselves in a simple and straightforward way
- Sometimes people express themselves vaguely and indirectly
 - Analyst must refer to the context when interpreting the message as the same phenomenon can
be described in a number of different ways depending on context
4. Framework Analysis
● Familiarization: Transcribing and reading the data
● Identifying a thematic framework: initial coding framework which is developed both from a
priori issues and from emergent issues
● Coding: Using numerical or textual codes to identify specific piece of data which correspond
to different themes
● Charting: Charts created using headings from thematic framework (can be thematic or by
case)
● Mapping and interpretation: Searching for patterns, associations, concepts and explanations
in the data.
5. Grounded Theory
● Analytic Induction
- Starts with an examination of a single case from a ‘pre-defined’ population in order to
formulate a general statement about a population, a concept or a hypothesis.
- Then the analyst examines another case to see whether it fits the statement
- If it does, a further case is selected
- If it doesn’t fit there are two options
● Either the statement is changed to fit both cases or the definition of the population is
changed is such a way that the case is no longer a member of the newly defined
population. - Then another case is selected and the process continues - In such a way
one should be able to arrive at a statement that fits all cases of a
population-as-defined. - This method is only for limited set of analytic problems: those
that can be solved with some general overall statement.
Strategies for analyzing observations
● Chronology: describe what was observed chronologically overtime, to tell the story from the
beginning to the end.
● Key events: describing critical incidents or major events, not necessarily in order of occurrence
but in order of importance
● Various Setting: describe various places, sites, settings or locations in which events/behaviors of
interest happen
● People: describing individuals or groups involved in the events
● Process: describing important processes (e.g. control, recruitment, decision-making, socialization,
communication.
● Issues: Illuminating key issues – how did participants change.