Qualitative Research Design

1
 Qualitative Research Methods
• Its aim is to give a complete, detailed descriptions of the
phenomena to be studied
• Objective facts + values
• Key philosophical assumption - understanding how
people make sense of their worlds and the experiences
people have

• Key concern - knowing or understanding from the

participants’ perspectives

• Key focus - understanding (rather than predicting

or controlling) social settings or social phenomena

2
When to choose?
1. Describe the phenomena

2. Build a theory
a) To gain new insights about a particular phenomena
b) Develop new concepts or theoretical perspectives about
the phenomena
c) Discover the problem that exists in the phenomena

3. Verification – to test the validity of certain assumptions,

claims, theories or generalization with the real world

4. Evaluation – to evaluate the effectiveness of a

particular policies, design artifacts, programs, etc

5
Qualitative Research Methods
 There are many qualitative research methods.
 Case study

 Action research

 Ethnography

 Grounded Theory

 Etc.

6
What is a case study?
 case study is an empirical research method.
 it is not a subset or variant of other methods, such as
experiments, surveys or historical study.

 Best suited to applied problems that need to be studied in

context.
 Phenomena under study cannot be separated from context.
Effects can be wide-ranging.

 Settings where researcher has little control over variables, e.g.

field sites.

 Effects take time to appear.

 weeks, months, or years rather than minutes or hours.

7
Why conduct a case study?
 To gain a deep understanding of a phenomenon
 Example:
 To understand the capability of a new tool
 To identify factors affecting communication in code inspections
 To characterize the process of coming up to speed on a project
 In building a framework, model or any design

 Objective of Investigation
 Exploration-To find what’s out there
 Characterization-To more fully describe
 Validation-To find out whether a theory/hypothesis is true

 Subject of Investigation
 An intervention, e.g. tool, technique, method, approach to
design, implement, or organizational structure
 An existing thing or process, e.g. software implementation,
project success, defects, network performance …
8
Parts of a Case Study Research Design

 Main five parts of a case study research design

 Part 1: Research/study questions

 Part 2: Propositions (if any)

 Part 3: Unit(s) of analysis

 Part 4: Logic/reasoning to link data to the propositions

9
 Part 1: Study Questions

 Case studies are most appropriate for research questions that

are of the “how” and “why” variety

 The initial task is to clarify precisely the nature of the study

questions (i.e. make sure they are actually “how” or “why”
questions)

 Examples:
 “Why do 2 organizations have a collaborative relationship?”
 "Why do developers prefer this tool/model/notation?"
 "How are inspections carried out in practice?“
 "How does agile development work in practice?"
 "Why do programmers fail to document their code?“
 "How does software evolve over time?“
 "Why have formal methods not been adopted widely for safety critical
applications?“
 "How does a company identify which software development projects to start?"
10
Types of Case Studies
 Explanatory
Arbitrates between competing explanations

Example: How important is implementation bias in requirements

engineering?
Rival theories: existing architectures are useful for anchoring, vs.
existing architectures are over-constraining during RE

 Descriptive
Describes sequence of events and underlying mechanisms

Example: How does pair programming actually work?

Example: How do software immigrants naturalize?

11
Types of case …
 Causal
Looks for causal relationship between concepts

Example: Requirements errors are more likely to

cause safety-related defects than programming
errors

 Exploratory
Criteria or parameters

Example: What do CMM level 3 organizations have in

common?

12
Part 2: Study Propositions (hypothesis)
 Propositions are statements that help direct attention to
something that should be examined in the case study, i.e.
point to what should be studied
 Example: “Organizations collaborate because they derive
mutual benefits”

 Propositions will tell you where to look for relevant

evidence
 Example: Define and ascertain the specific benefits to each
organization

 Some studies may not have propositions –this implies a

topic of “exploration”
 Note: Even exploratory studies should have both clearly-stated
purposes and clearly-stated criteria for success

13
Part 3: Unit of Analysis
 The unit of analysis defines what a “case” is in a case study
 Example: a unit of analysis (case) may be an individual, and the case study may be
the life history of that person

 Other units of analysis include decisions, social programs, processes,

changes
 Note: It is important to clarify the definition of these cases as they may be
subjective, e.g. the beginning and end points of a process

 What unit of analysis to use generally depends on the primary research

questions

 Once defined, the unit of analysis can still be changed if desired, e.g. as a
result of discoveries based on data

 To compare results with previous studies (or allow others to compare

results with yours), try to select a unit of analysis that is or can be used by
others
14
Examples of Units of Analysis
 For a study of software adoption
 Individuals
 Development team
 Organization

 For a study of programming

 Programming episode
 Pairs of programmers
 Development team
 Organization

 For a study of software evolution

 Modification report
 File
 System
 Release
 Stable release

15
Part 4: Linking Logic
 Logic or reasoning to link data to propositions

 One of the least well developed components in case

studies

 Many ways to perform this, but none as precisely defined

as the treatment/subject approach used in experiments

 One possibility is pattern matching

 Describe several potential patterns, then compare the

case study data to the patterns and see which one is
closer
16
Data Analysis

 Criteria for High Quality Analysis

 Present all the evidence

 Develop rival hypotheses

 Address all major rival interpretations

 Address most significant aspect of the case study

 Use prior or expert knowledge

17
Some Specific Analytic Techniques
 Pattern Matching

 Explanation Building

 Time-Series Analysis

 Cross-Case Synthesis
 Note: They are intended to deal with problems of developing
internal and external validity in doing case studies

18
Further Notes on Data analysis-coding
 Coding is the process of examining the raw qualitative
data in the transcripts and extracting sections of text
units (words, phrases, sentences or paragraphs) and
assigning different codes or labels so that they can easily
be retrieved at a later stage for further comparison and
analysis, and the identification of any patterns.

 Codes can be based on:

 Themes, Topics
 Ideas, Concepts
 Terms, Phrases
 Keywords
 Which are found in the data

19
 Quantitative Research Design

20
Quantitative Research Methods
 Uses deductive method of knowledge acquisition
 Intends to falsifies an existing theory
 Tries to generate generalizable knowledge
 Accepts objectivity of knowledge
 Uses a standard measurement instruments
 The researcher is independent in the process of
knowledge construction
 Quantifies the phenomena in terms of numbers
 Intends with prediction and controlling

21
 Cont…
The purposes of research can be categorized as:
 Descriptive (fact finding)
 Explorative (looking for patterns)
 Analytical (explaining why or how)
 Predictive (forecasting the likelihood of particular events)

 Methods are like

 Laboratory Experiment
 Survey Research – Questionnaire (possibly interview)

22
Descriptive

 It was started during the Roman Empire

 Took census to know the number of people under its
empire
 Used for taxation purposes
 Seeks to accurately describe current or past
phenomena - to answer such questions as:
a) What type of open source software frequently used in
Ethiopia?
b) What type of computers mostly imported in Ethiopia
c) What are the main software tools used by end users?

23
 Explorative

Seeks to understand psychological and sociological

phenomena in an effort to find behavioral patterns:
a) How do users use the new software system?

b) Why would users use the software in this way contrary

to original purpose. For example use of Internet for
news rather work related matters

c) What are the antecedent factors for IT project success?

24
Analytical

 Seeking to explain the reasons behind a particular

occurrence by discovering causal relationships.
 Once causal relationships have been discovered, the
search then shifts to factors that can be changed
(variables) in order to influence the chain of causality.
Typical questions are:
a) Do user involvement in software development and users IT skills increase
software success?
b) Does Hardware performance have an influence on software failure?
c) Are road related factors more important in determining accident severity?
d) Can network signal be improved by ….
e) Does hybrid approach provide better performance.?

25
Predictive
 Seeks to forecast the likelihood of particular
phenomena occurring in given circumstances.
a) To what extent the new SW dev’t technique increase
productivity?
b) Does internet bandwidth predict growth online business
services ?
c) Can the new software system increases the organization’s
efficiency?
d) To what extent does new software predict growth of the
organization profit?
 This compares independent (IV) and dependent variable
(DV).
26
 Instruments/tools of Data Collection

 Questionnaire
 Offline/Online questionnaire (there are free sites)

 Computer Simulation

 Databases (web sites, news …)

 Computer log files – Data like

 Attempts made by Hackers
 Attempts made to break passwords

27
Quantitative Data Analysis
 Laboratory experiment
 Using machine learning algorithms (for NLP, DM, IR….)
 Different parameter settings and various experiments
 Statistical - two types
1. Descriptive statistics
 Used to describe the phenomena with frequency, mean, median,
mode
2. Inferential statistics
• Is used to infer about the population from the sample data
• T-test to compare means of two groups
• Analysis of Variance (ANOVA)- (When groups more than two)
• Regression , ……