O—Observation or measurement

R—Random assignment

The three experimental designs discussed in this section are:

The One Shot Case Study

This is a single group studied only once. A group is introduced to a treatment or condition and then observed for changes
which are attributed to the treatment

XO

The Problems with this design are:

 A total lack of control. Also, it is of very little scientific value as securing scientific evidence to make a
comparison, and recording differences or contrasts.
 There is also a tendency to have the error of misplaced precision, where the researcher engages in tedious
collection of specific detail, careful observation, testing and etc., and misinterprets this as obtaining good
research. However you can not misinterpret that a detailed data collection procedure equals a good design.
 History, maturation, selection, mortality and interaction of selection and the experimental variable are all threats
to the internal validity of this design.

One Group Pre-Posttest Design

This is a presentation of a pretest, followed by a treatment, and then a posttest where the difference between O 1 and O2 is
explained by X:

O1 X O 2

 However, there exists threats to the validity of the above assertion:

 History—between O1 and O2 many events may have occurred apart from X to produce the differences in
outcomes. The longer the time lapse between O1 and O2 the more likely history becomes a threat.
 Maturation—between O1 and O2 students may have grown older or internal states may have changed and
therefore the differences obtained would be attributable to these changes as opposed to X.
 Testing—the effect of giving the pretest itself may effect the outcomes of the second test (i.o., IQ tests taken a
second time result in 3-5 point increase than those taking it the first time). In the social sciences, it has been
known the process of measuring may change that which is being measured—the reactive effect occurs when the
testing process itself leads to the change in behavior rather than it being a passive record of behavior (reactivity—
we want to use non-reactive measures when possible).
 Instrumentation—examples are in threats to validity above.
 Statistical regression—or regression toward the mean. Time-reversed control analysis and direct examination for
changes in population variabilities are useful precautions against such misinterpretations. What this means is that
if you select samples according to their extreme characteristics or scores, the tendency is to regress toward the
mean. Therefore those with extreme high scores appear to be decreasing their scores, and those with extreme low
scores appear to be increasing their scores. However this interpretation in not accurate, and to control for
misinterpretations, researchers may want to do a time-reversed (posttest-pretest) analysis to analyze the true
treatment effects. Researchers may exclude outliers form the analysis.
 Other—History, maturation, testing, instrumentation interaction of testing and maturation, interaction of testing
and the experimental variable and the interaction of selection and the experimental variable are also threats to
validity for this design.

The Static Group Comparison

This is a two group design, where one group is exposed to a treatment and the results are tested while a control group
is not exposed to the treatment and similarly tested in order to compare the effects of treatment.

X O1
 O2
Threats to validity include:
 Selection—groups selected may actually be disparate prior to any treatment.
 Mortality—the differences between O1 and O2 may be because of the drop-out rate of subjects from a
specific experimental group, which would cause the group to be unequal.
 Other—Interaction of selection and maturation and interaction of selection and the experimental variable.

Three True Experimental Designs

The next three designs discussed are the most strongly recommended designs:

The pretest-posttest Control Group Design

This designs takes on this from:

R O1 X O2

R O3 O4

This design controls for all of the seven threats to validity described in detail so far. An explanation of how this
design controls for these threats is below.

 History—this is controlled in that the general history events which may have contributed to the O1 and O2
effects would also produce the O3 and O4 effects. This is true only if the experiment is run in a specific
manner—meaning that you may not test the treatment and control groups at different times and in vastly
different settings as these differences may effect the desults. Rather, you must test simultaneously the
control and experimental groups truly are run simultaneously, then there must be different experimenters
involved, and the differences between the experimenters may contribute to effects.

A solution to history in this case is the randomization of experimental occasions—balanced in terms of

experimenter, time of day, week and etc.

 Maturation and testing—these are controlled in that they are manifested equally in both treatment and
control goups.
 Instrumentation—this is controlled where conditions control for intrasession history, especially where
fixed tests are used. However when observers or interviewers are being used, there exists a potential for
problems. If there are insufficient observers to be randomly assigned to experimental conditions, the care
must be taken to keep the observers ignorant of the purpose of the experiment.
 Regression—this is controlled by the mean differences regardless of the extremely of scores or
characteristics, if the treatment and control groups are randomly assigned from the same extreme pool. If
this occurs, both groups will regress similarly, regardless of treatment.
 Selection—this is controlled by randomization.
  Mortality—this was said to be controlled in this design, however upon reading the text, it seems it may
or may not be controlled for, Unless the mortality rate is equal in treatment and control groups, it is not
possible to indicate with certainty that mortality did not contribute to the experiment results. Even when
even mortality actually occurs, there remains a possibility of complex interactions which may make the
effects drop-out rates differ between the two groups. Conditions between the two groups must remain
similar—for example, if the treatment group must attend treatment session, then the control group must
also attend session where either not treatment occurs, or a “placebo” treatment occurs. However even in
this there remains possibilities of threats to validity. For example, even the presence of a “placebo” may
contribute to an effect similar to the treatment, the placebo treatment must be somewhat believable and
therefore may end up having similar results!

The factors described so far effect internal validity. These factors could produce changes which may be
interpreted as the result of the treatment. These are called main effects which have been controlled in this design
giving it internal validity.

However in this design, there are threats to external validity (also called interaction effects because they involve
the treatment and some other variable the interaction of which cause the threat to validity.) It is important to note
here that external validity or generalizability always turns out to involve extrapolation into a realm not
represented in one’s sample.
In contrast, internal validity are solvable within the limits of the logic of probability statistics. This means that we
can control for internal validity based on probability statistics within the experiment conducted, however external
validity or generalizability can not logically occur because we can’t logically extrapolate to different conditions.
(Hume’s truism that induction or generalization is never fully justified logically).
External threats include:
 Interaction of testing and X—because the interaction between taking a pretest and the treatment itself
may effect the results of the experimental grop, it is desirable to use a design which does not use a pretest.
 Interaction of selection and X—although selection is controlled for by randomly assigning subjects into
experimental and control groups, there remains a possibility that the effects demonstrated hold true only
for that population from which the experimental and control groups were selected . An example is a
researcher trying to select schools to observe, however has been turned turned down by 9, and accepted
by the 10th. The characteristics of the 10th school may be vastly different than the other 9, and therefore
not representative of an average school. Therefore in any report, the researcher shoul describe the
population studied as well as any populations which rejected the invitation.
 Reactive arrangements—this refers to the artificiality of the experimental setting and the subject’s
knowledge that he is participating in an experiment. This situation is unrepresentative ot the school
setting or any natural setting, and can seriously impact the experiment results. To remediate this problem,
experiments should be incorporated as variants of the regular curricula, tests should be integrated into the
normal testing routine, and treatment should be delivered by regular staff with individual students.
 Research should be conducted in schools in this manner—ideas for research should originate with teachers or
other school personnel. The designs for this research should be worked out with someone expert at research
methodology, and the research itself carried out by those who came up with the research idea. Results should
be analyzed by the expert, and then the final interpretation delivered by an intermediary.

Tests of significance for this design—although this design may be developed and conducted appropriately,
statistical tests of significance are not always used appropriately.
 Wrong statistic in common use—many use a t-test by computing two ts, one for the pre-post difference in
the experimental group and one for the pre-post difference of the control group. If the experimental t-test
is statistically significant as opposed to the control group, the treatment is said to have an effect. However
this does not take into consideration how “close” the t-test may really have been. A better procedure is to
run a 2X2 ANOVA repeated measures, testing the pre-post difference as the within-subject factor, the
group difference as the between-subject factor, and the interaction effect of both factors.
 Use of gain scores and covariance—the most used test is to compute pre-posttest gain scores for each
group, and men to compute a t-test between the experimental and control groups on the gain scores, Also
used are usually preferable to simple gain-score comparisons.
 Statistics for random assignment of intact classrooms to treatments—when intact classrooms have been
assigned at random to treatments (as opposed to individuals being assigned to treatments), class means
are used as the basic observations, and treatment effects are tested against variations in these means. A
covariance analysis would use pretest means as the covariate.

The soloman Four-Group Design

The design is as:

R O1 X O2
R O3 O4
R X O5
R O6

In this design, subjet are randomly assigned to four different groups: experimental with both pre-posttests,
experimental with no pretest, control with pre-postter=sts, and control without pretests.By using experimental
and control groups with and without pretests, both the main effects of testing and the interaction of testing and
the treatment are controlled. Therefore generalizability increases and the effect of X is replicated in four
different ways. Statistical tests for this design—a good way to test the results is to rule out the pretest as a
 “treatment” and treat the the posttestscores with a 2X2 analysis of variance design-prestested against
unprestested

The Posttest-Only Control Group Design

This design is as:
R x O1
R O2
This design can be though of as the last two groups in the Solomon 4-group design. And can be seen as
controlling for testing as main effet and interaction, but unlike this design, it doesn’t measure them. But the
measurement of these effects isn’t necessary to the central question of whether of not X did have an effect. This
design is appropriate for times when pretests are not acceptable.
Statistical tests for this design—the most simple form would be the t-test. However covariance analysis and
blocking on subject variables (prior grades, test scores, etc.) can be used which increase the power of the
significance test similarly to what is provided by a pretest.

Discussion or causal inference and generalization

As illustrated above. Cook and Campbell devoted much efforts to avoid/reduce the threats against internal validity
(cause and effect) and external validity (generalization ). However, some widespread concepts may also
contribute other types of threats against internal and external validity.
Some researchers downplay the importance of causal inference and assert the worth of understanding. This
understanding includes “what,” and “why.” However is “why” considered a “causes and effect” relationship? If a
question “why X happens” is asked and the answer is “Y happens.” Does it imply mat “Y causes X”? If X and Y
are correlated only, it does not address the question “why” Replacing “cause and effect” with “understanding”
makes the conclusion confusing and misdirect researchers away from the issue of internal validity.” Some
researchers apply a phenomenological approach to “explanation.” In this view, an explanation is applied to only a
particular case in a particular time and place, and thus generalization is considered inapptonmate. In tact, a
particulate explanation does not explain anything. For example, if one askes. “Why Alex Yu behaves in that
way,” the answer could be “because he is Alex Yu. He is a unique human being. He has a particular family
background and a specific social circle.” These “ particular” statements are always right, thereby misguide
researchers away from the issue of external validity.

Q.4: Define experimental research. What are the different experimental designs used in the
experimental research?
ANS: Experimental Research
Experimental research, often considered to be the “gold standard” in research designs, is one of the most rigorous
of all research designs. In this design, one or more independent variables are manipulated by the researcher (as
treatments), subjects are randomly assigned to different treatment levels (random assignment), and the results of
the treatments on outcomes (dependent variables) are observed. The unique strength of experimental research is
its internal validity (causality) due to its ability to link cause and effect through treatment manipulation, while
controlling for the spurious effect of extraneous variable.
Experimental research is bet suited for explanatory research (rather than for desvriptive or exploratory research),
where the goal of the study is to examine cause-effect relationships. It also works well for research that involves a
relatively limited and well-defined set of independent variables that can either be manipulated or controlled.
Experimental research can be conducted in laboratory or field setting. Laboratory experiments, conducted in
laboratory (artificial) settings, tend to be high in internal validity, but this comes at the cost of low external
validity (generalizability), because the artificial (laboratory) setting in which the study is conducted may not
reflect the real world. Field experiments , conducted in field settings such as in a real organization, and high in
both internal and external validity. But such experiments are relatively rare, because of the difficulties associated
with manipulating treatments and controlling for extraneous effects in a field setting.
Experimental research can be grouped into two broad categories: true experimental designs and quasi-
experimental design. Both designs require treatment manipulation, but while true experiments also require
random assignments, quasi-experiments do not. Sometimes, we also refer to non-experimental research, which is
not really a research design, but an all-inclusive term that includes all types of research that do not employ
treatment manipulation or random assignment, such as survvoy research, observational research and correlational
studies.
Basic Concepts
Treatment and control groups. In experimental research, some subjects are administered one or more experimental
stimulus called a treatment (the treatment group) while other subjects are not given such a stimulus (the control
group). The treatment may be considered successful if subjectgs in the treatment group rate more favorably on
outcome variables than control group subjects. Multiple levels of experimental stimulus may be administered, in
which case, ther may be more than one treatment group. For example, in order to test the effects of a new drug
intended to treat a certain medical condition like demenda if a sample of dementia patients is randomly divided
into three groups, with the first group receiving a high dosage of the drug, the second group receiving a low
dosage, and the third group receives a placebo such as a sugar pill (control group), then the first two groups are
experimental groups and the third group is a control group. After administering the drug for a period of time, if
the condition of the experimental group subjects improved significantly more than the control group subjects, we
can say that the drug is effective. We can also compare the conditions of the high and low dosage experimental
groups to determine if the high dose in more effective than the low dose. Treatment manipulation. Treatments are
the unique feature of experimental research that sets this design apart from all other research methods. Treatment
manipulation helps control for the “cause” in cause-effect relationships. Naturally, the validity of experimental
research depends on how well the treatment was manipulated. Treatment manipulation must be checked using
pretests and pilot tests prior to the experimental study. Any measurements conducted before the treatment is
administered are called pretest measures , while those conducted after the treatment are posttest measures.
Random selection and assignment. Random selection is the process of randomly drawing a sample from a
population or a samplingframe, This approach is typically employed in survey research, and assures that eatch
unit in the population has a positive chance of being selected into the sample. Random assignment is however a
process of randomly assigning subjects to experimental or control groups. This is a standard practice in true
experimental research to ensure that treatment groups are similar (equivalent) to each other and to the control
group, prior to treatment administration. Random selection is related to sampling, and is therefore, more closely
related to the external validity (generalizability) of findings. However, random assignment is related to design,
and is therefore most related to internal validity. It is possible to have both random selection selection and random
assignment in well-designed experimental research, but quasi-exprimental research involves neither random
selection nor random assignment.
Threats to internal validity. Although experimental designs are considered more rigorous than other research
methods in terms of the internal validity of their inferences (by virtue of their ability to control causes through
treatment manipulation ), they are not immune to internal validity threats. Some of these threats to internal
validity are described below, within the context of a study of the impact of a special remedial math tutoring
program for improving the math abilities of high school students.
 History threat is the possibility that the observed effects (dependent variables) are caused by extraneous
or historical events rather than by the experimental treatment. For instance, students’ post-remedial math
score improvement may have been caused by their preparation for a math exam at their school, rather
than the remedial math program.
 Maturation threat refers to the possibility that observed effects are caused by natural maturation of
subjects (e,g., a general improvement in their intellectual ability to understand complex concepts) rather
than the experimental treatment.
 Testing threat is a threat in pre-post designs where subject’ posttest responses are conditioned by their
pretest responses. For instance, if students remember their answers from the pretest evaluation, they may
tend to repeat them in the posttest scuril. Not conducting a pretest can help avoid this threat.
 Instrumentation threat, which also occurs in pre-post designs, refers to the possibility that the difference
between pretest and posttest scores is not due to the remedial math program, but due to changes in the
administered test, such as the posttest having a higher or lower degree of difficulty than the pretest.
 Mortality threat refers to the possibility that subjects may be dropping out of the study at differential rates
between the treatment and control groups due to a systematic reason, such that the dropouts were mostly
students who scored low on the pretest. If the low-performing students drop out, the results of the posttest
will be artificially inflated by the preponderance of high-performing students.
 Regression threat, also called a regression to the mean, refers to the statistical tendency of a group’s
overall performance on a measure during a posttest to regress toward the mean of that measure rather than
in the anticipated direction. For instance, if subjects scored high on a pretest, they will have a tendency to
score lower on the posttest (closer to the mean) because their high scores (away from the mean) during
the pretest was possibly a statistical aberration. This problem tends to be more prevalent in non-random
samples and when the two measures are imperfectly correlated.
 Two-Group Experimental Designs

The simplest true experimental designs are two group designs involving one treatment group and one control
group, and are ideally suited for testing the effects of a single independent variable that can be manipulated as a
treatment. The two basic two-group designs are the pretest –posttest control group design and the posttest –only
control group design, while variations may include covariance designs. These designs are often depicted using a
standardized design notation, where R represent random assignment of subjects to groups. X represents the
treatment administered to the treatment group, and O represents pretest or posttest observations of the dependent
variable (with different subscripts to distinguish between pretest and posttest observations of treatment and
control groups).
Pretest-posttest control groups, subjected to an initial (pretest) measurement of the dependent variables of interest,
the treatment group is administered a treatment (representing the independent variable of interest). And the
dependent variables measured again (posttest). The notation of this design is shown in figure10.1.

R O1 X O2 Treatment Group

R O3 O4 Control Group

Figure 10.1. Pretest-posttest control group design

The effect E of the experimental treatment in the pretest posttest design is measured as the difference in the

posttest and pretest scores between the treatment and control groups:
E=(O2 –O1) – (O4 – O3)
Statistical analysis of this design involves a simple analysis of variance (ANOVA) between the treatment and
control groups. The pretest posttest design handles several threats to internal validity, such as maturationl testing,
and regression , since these threats can be expected to influence both treatment and control groups in a similar
(random) manner. The selection threat is controlled via random assignment. However, additional threats to
internal validity may exist. For instance, mortality can be a problem if there are differential dropout rates between
the two groups, and the pretest measurement may bias the posttest measurement (especially if the pretest
introduces unusual topics or content).
Posttest-only control group design. This design is a simpler version of the pretest –posttest design where pretest
measurements are omitted. The design notation is shown in Figure 10.2.

R X O1 Treatment Group

R O2 Control Group
Figure 10.2. posttest only control group design.
The treatment effect is measured simply as the difference in the posttest scores between the two groups:
E = (O 1 – O 2)
The appropriate statistical analysis of this design is also a two-group analysis of variance (ANOVA). The
simplicity of this design makes it more attractive than the pretest –posttest design in terms of terms of internal
validity. This design controls for maturation, testing , regression, selection, and pretest-posttest interaction ,
though the morality threat may continue to exist. Covariance designs. Sometimes, measures of dependent
variables may be influenced by extraneous variables called covariates. Covariates are those variables that are not
of central interest to an experimental study, but should nevertheless be controlled in an experimental design in
order to eliminate their potential effect on the dependent variable and therefore allow for a more accurate
detection of the effects of the independent variables of interest. The experimental designs discussed earlier did not
control for such covariates. A covariance design (also called a concomitant variable design) is a special type of
pretest posttest control group design where the pretest measure is essentially a measurement of the covariates of
interest rather than that of the dependent variables. The design notation is shown in figure 10.3, where C
represents the covariates:

R C X O1 Treatment Group

R C O2 Control Group

Figure 10.3. Covariance design

Because the pretest measure is not a measurement of the dependent variable, but rather a covariate, the treatment
effect is measured as the difference in the posttest scores between the treatment and control groups as:
E = (O 1 – O2 )
Due to the presence of covariates, the right statistical analysis of this design is a two-group analysis of covariance
(ANCOVA). This design has all the advantages of post-test only design, but with internal validity due to the
odnnelling of covariates. Covariance designs can also be extended to pretest-posttest control group design.
Factorial Designs
Two- group designs are inadequate if your research requires manipulation of two or more independent variables
(treatments). In such cases, you would need four or higher-group designs. Such designs, quite popular in
experimental research, are commonly called factorial designs. Each independent variable in this design is called a
factor , and each sub-division of a factor is called a level. Factorial designs enable the researcher to examine not
only the individual effect of each a level. Factorial designs enable the researcher to examine not only the
individual effect of each treatment on the dependent variables (called main effects), but also their joint effect
(called interaction effects).
The most basic factorial design is a 2 X 2 factorial design, which consists of two treatment , each with two levels
(such as high/low or present/absent). For instance, let’s say that you want to compare the learning outcomes of
two different types of instructional techniques (in-class and online instruction), and you also want to examine
whether these effects vary with the time of instructional (1.5 or 3 hours per week for instructional time), then the
second factor will consist of three levels and you will have a 2 X 3 factorial design, On the other hand, if you
wish to add a third factor such as group work (present versus absent), you will have a 2 X 2 X 2 factorial design.
In this notation, each number represents a factor, and the value of each factor represents the number of levels in
that factor.
R X11 O

R X12 O

R X21 O

R X22 O

Figure 10.4. 2 X 2 factorial design

Factorial designs van also be depicted using a design notation, such as that shown on the right panel of Figure
10.4. R represents random assignment of subjects to treatment groups, X represents the treatment groups
themselves (the subscripts the level of each factor), and O represent observations of the dependent variable.
Notice that 2 X 2 factorial design will have four treatment groups, corresponding to the four combinations of the
two levels of each factor. Correspondingly, the2 X 3 design will have six treatment groups, and the 2 X 2 X 2
design will have eight treatment groups. As a rule of thumb, each cell in a factorial design should have a
minimum sample size of 20 (this estimate is derived from Cohen’s power calculations based on medium effect
sizes). So a 2 X 2 X 2 factorial design requires a minimum total sample size of 160 subjects, with at least 20
subjects in each cell. As you can see, the cost of data collection can increase substantially with more levels or
Sometimes, due to resource constraints, some cells in such factorial designs may not receive any treatments at all,
which are called incomplete factorial designs. Such incomplete designs hurt our ability to draw inferences about
the incomplete factors.
In a factorial design, a main effect is said to exist if the dependent variable shows a significant difference between
multiple levels of one factor , at all levels of other factors. No change in the dependent variable across factor
levels is the null case (baseline), from which main effects are evaluated. In the above example, you may see a
main effect of instructional type, instructional time, or both on learning outcomes. An interaction effect exists
when the effect of differences in one factor depends upon the level of a second factor. In our example, if the effect
of instructional type on learning outcomes is greater for 3 hours/ week of instruction time than for 1.5 hours/week,
then we can say that there is an interaction effect between instructional type and instructional time on learning
outcomes. Note that the presence of interaction effects dominate and make main effects irrelevan
t, and it is not meaningful to interpret main effects if interaction effects are significant.
Hybrid Experimental Designs
Hybrid designs are those that are formed by combining features of more established designs.
Three such hybrid designs are randomized bocks design. Solomon four-group design, and switched replications
design.
Randomized block design. This is a variation of the posttest-only or pretest-posttest control group design where
the subject population can be grouped into relatively homogeneous subgroups (called block) within which the
experiment is replicated. For instance, if you want to replicate the same posttest-only design among university
student and full time working professionals ( two homogeneous blocks),subjects In both blocks are randomly split
between treatment group (receiving the same treatment ) or control group (see Figure 10.5). The purpose of this
design in to reduce the “noise” or variance in data that may be attributable to differences between the blocks so
that the actual effect of interest can be detected more accurately.

University R X O
Student
R O

Homogeneous Groups G
Full-Time R X O
Workers
R O

Figure 10.5, Randomized blocks design.

Solomon four –group design . In this design, the sample is divided into two treatment groups and two control
groups. One treatment group and one control group receive the pretest, and the other two groups do not. This
design represents a combination of posttest-only and pretest-posttest control group design, and is intended to test
for the potential biasing effect of pretest measurement on posttest measures that tends to occur in pretest-posttest
designs but not in posttest only designs. The design notation is shown in Figure 10.6.

R O X O

R O O

R X O

R O

Figure 10.6. Solomon four –group design

Switched replication design. This is a two-group design implemented in two phases with three waves of
measurement. The treatment group in the first phase serves as the control group in the second phase, and the
control group in the first phase becomes the treatment group in the second phase, as illustrated in Figure 10.7. In
other words, the original design is repeated or replicated temporally with treatment/control roles switched
between the two groups. By the end of the study, all participants will have received the treatment either during the
first or the second phase. This design is most feasible in organizational contexts where organizational programs
(e.g., employee training) are implemented in a phased manner or are repeated at regular intervals.

R O X O O

R O O X O

Figure 10.7. Switched replication design.

Quasi-Experimental Designs
Quasi-experimental designs are almost identical to true experimental designs, but lacking one key ingredient:
random assignment. For instance, one entire class section or one organization is used as the treatment group,
while another section of the same class or a different organization in the dame industry is used as the control
group. This lack of random assignment potentially results in groups that are non-equivalent, such as one group
possessing greater mastery of a certain content than the other group, say by virtue of having a better teacher in a
previous semester, which introduces the possibility of selection bias. Quasi-experimental designs are therefore
inferior to true experimental designs in interval validity due to the presence of a variety of selection related threats
such as selection –maturation threat (the treatment and control group being differentially impact by extraneous or
historical events selection –regression threat (the treatment and control groups regressing toward the mean
between pretest and posttest at different rates). Selection-instrumentation threat (the treatment and control groups
responding differently to the measurement. Selection –testing (the treatment and control groups responding
differently to the pretest). And selection –mortality (the treatment and control groups demonstrating differential
dropout rates ). Given these selection threats, it is generally preferable to avoid quasi-experimental designs to the
greatest extent possible.
Many true experimental designs can be converted to quasi-experimental designs by omitting random assignment.
For instance, the quasi-equivalent version of pretest-posttest control group design is called nonequivalent groups
design (NEGD), as shown in Figure 10.8, with random assignment R replaced by non-equivalent (non-random)
assignment N. Likewise, the quasi-experimental version of switched replication design in called non-equivalent
switched replication design (see Figure 10.90.

N O X O

N O O

Figure 10.8. NEGD design.

N O X O O

N O O X O

Figure 10.9 Non-equivalent switched replication design.

In addition, there are quite a few unique non-equivalent designs without corresponding true experimental design
cousins. Some of the more useful of these designs are discussed next. Regression –discontinuity (RD). This is a
non-equivalent pretest-posttest design where subjects are assigned to treatment or control group based on a cutoff
score on a preprogram measure. For instance, patients who are severely ill may be assigned to a treatment group
to test the efficacy of a new drug or treatment protocol and those who are mildly ill are assigned to the control
group. In another example, students who are lagging behind on standardized test scores may be selected for a
remedial curriculum program intended to improve their performance, while those who score high on such tests are
not selected from the remedial program. The design notation can be represented as follows, where C represents
the cutoff score:

C O X O

C O O

Figure 10.10. RD design.

Because of the use of a cutoff score, it is possible that the observed results may be a function of the cutoff score
rather than the treatment, which introduces a new threat to internal validity. However, using the cutoff score also
ensures that limited or costly resource are distributed to people who need the the most rather than randomly across
a population, while simultaneously allowing a quasi-experimental treatment. The control group scores in the RD
design does not serve as a benchmark for comparing treatment group scores, given the systematic non-
equivalence between the two groups. Rather, if there is no discontinuity between pretest and posttest scores in the
control group, but such a discontinuity persists in the treatment group, then this discontinuity is viewed as
evidence of the treatment effect.
Proxy pretest design. This design, shown in Figure10.11, looks very similar to the standard NEGD (pretest-
posttest ) design , with one critical difference: the pretest score is collected after the treatment is administered. A
typical application of this design is when a researcher is brought in to test the efficacy of a program (e.g., an
educational program) after the program has already started and pretest data is not available. Under such
circumstances, the best option for the researcher is often to use a different prerecorded measure, such as students’
grade point average before the start of the program, as a proxy for pretest data. A variation of the proxy pretest
design is to use subjects’ posttest recollection of pretest data, which may be subject to recall bias, but nevertheless
may provide a measure of perceived gain or change in the dependent variable.

N O1 X O2

N O1 O2

Figure 10.11. Proxy pretest design.

Separate pretest –posttest data from the same subjects for some reason. As shown in Figure 10.12, there are four
groups in this design, but two groups come from a single non-equivalent group, while the other two groups come
from a different non-equivalent group. For instance, you want to test customer satisfaction with a new online
service that is implemented in one city but not in another. In this case, customers in the first city serve as the
treatment group and those in the second city constitute the control group. If it is not possible to obtain pretest and
posttest measures from the same customers, you can measure customer satisfaction at on e point in time,
implement the new service program and measure customer satisfaction (with a different set of customers) after
the program is implemented. Customer satisfaction is also measured in the control group at the same times as in
the treatment group, but without the new program implementation. The design is not particularly strong, because
you cannot examine the changes in any specific customer’s satisfaction score before and after the
implementation , but you can only examine average customer satisfaction scores. Despite the lower internal
validity, this design may still be a useful way of collecting quasi-experimental data when pretest and posttest data
are not available from the same subjects.

N1 O

N1 X O

N2 O

N2 O

Figurfigure 10.12. Separate pretest –posttest samples design.

Nonequivalent dependent variable (NEDV) design. This is a single-group pre-post quasi-experimental design with
two outcome, measures , where one measure is theoretically expected to be influenced by the treatment and the
other measure is not. For instance, if you are designing a new calculus curriculum for high school students, this
curriculum is likely to influence students’ posttest calculus scores but not algebra scores. However, the posttest
algebra scores may still vary due to extraneous factors such as history or maturation. Hence, the pre-post algebra
scores can be used as a control measure, while that of pre-post calculus can be treated as the treatment measure.
The design notation, shown in Figure 10.13, indicates the single group by a single N, fellowed by pretest O 1 and
posttest O 2 for calculus and algebra for the same group of students. This design is weak in internal validity, but
its advantage lies in not having to use a separate control group.
An interesting variation of the NEDV design is a pattern matching NEDV design, which employs multiple
outcome variable and a theory that explains how much each variable will be affected by the treatment. The
researcher can then examine if the theoretical predication is matched in actual observations. This pattern-matching
technique, based on the degree of correspondence between theoretical and observed patterns is a powerful way of
alleviating internal validity concerns in the original NEDV design.

O1 X O1
N
O2 O2

Figure 10.13. NEDV design.

Perils of Experimental Research
Experimental research is one of the most difficult of research designs, and should not be taken lightly, This type
of research is often best with a multitude of methodological problems. First, though experimental research
requires theories for framing hypotheses for testing, much of current experimental research is a theoretical.
Without theories, the hypotheses being tested tend to be ad hoe, possible illogical, and meaningless. Second,
many of the measurement instruments used in experimental research are not tested for reliability and validity, and
are incomparable across studies. Consequently, results generated using such instruments are also incomparable.
Third, many experimental research use inappropriate research designs, such as irrelevant stimulus across
treatment groups. Findings from such studies tend to lack internal validity and are highly suspect. Fourth, the
treatments (tasks) used in experimental research may be divers, incomparable, and inconsistent across studies and
sometimes inappropriate for the subject population. For instance, undergraduate student subjects are often asked
to pretend that they are marketing managers and asked to perform a complex budget allocation task in with they
have no experience or expertise, The use of such inappropriate tasks, introduces new threats to internal validity
(i.e., subject’s performance may be an artifact of the content or difficulty of the task setting), generates findings
that are non-interpretable and meaningless, and makes integration of findings across studies impossible.
The design of proper experimental treatments is a very important task in experimental design, because the
treatment is the meson d’ etre of the experimental method and must never be rushed or neglected. To design an
adequate and appropriate task , researchers should use revalidated tasks (by debriefing subjects after performing
the assigned task), conduct pilot tests (repeatedly, in necessary), and if doubt, using tasks that are simpler and
familiar for the respondent sample than tasks that are complex or unfamiliar.
In summary, this chapter introduced key concepts in the experimental design research method and introduced a
variety of true experimental and quasi-experimental designs. Although these designs vary widely in internal
validity, designs with less internal validity should not be overlooked and may sometimes be useful under specific
circumstances and empirical contingencies.

Q.5: In which type of research problems you will prefer to use correlation studies and when is it
appropriate to use survey studies in education ?
ANS: Descriptive Research: Definition, Characteristics, Methods, Examples and Advantages

Survey is the type of descriptive research What is descriptive research?

Descriptive research definition: Descriptive research is defined as a research method that describes the
characteristics of the population or phenomenon studied. This methodology focuses more on the “what” of the
research subject than the “why” of the research subject. The descriptive research method primarily focuses on
describing the nature of a demographic segment, without focusing on “why” a particular phenomenon occurs. In
other words, it “describes” the subject of the research, without covering “why” it happens.
For example, an apparel brand that wants to understand the fashion purchasing trends among New York buyers
will conduct a demographic survey of this region, gather population data and then conduct descriptive research on
this demographic segment. This study will then uncover details on “what is the purchasing pattern of New York
buyers,” but not cover any investigative information about “why” the patterns exits. Because for the apparel brand
trying to break into this market, understanding the nature of their market is the study’s objective.
Applications of descriptive research with examples
A descriptive research method can be used in multiple ways and for various reasons. Before getting into any
survey, though, the survey goals and survey design are crucial. Despite following these steps, there is no way to
know if one will meet the research outcome. How to use descriptive research? To understand the end objective of
research goals, below are some ways organizations currently use descriptive research today:
 Define respondent characteristics: The aim of using close-ended questions is to draw concrete
conclusions about the respondents. This could be the need to derive patterns, traits, and behaviors of the
respondents. It could also be to understand from a respondent, their attitude, or opinion about the
phenomenon. For example, understanding from millennial the hours per week they spend on browsing the
internet. All this information helps the organization researching to make informed business decisions.
 Measure data trends: Researchers measure data trends over time with a descriptive research design’s
statistical capabilities. Consider if an apparel company researches different demographics like age groups
from 24-35 and 36-45 on a new range launch of autumn wear. If one of those groups doesn’t take too well
to the new launch, it provides insight into what clothes are like and what is not. The brand drops the
clothes and apparel that customers don’t like.
 Conduct comparisons: Organization also use a descriptive research design to understand how different
groups respond to a specific product or service. For example, an apparel brand creates a survey asking
general questions that measure the brand’s image. The same study also asks demographic questions like
age, income, gender, geographical location, etc. This consumer research helps the organization
 understand what aspects of the brand appeal to the population and what aspects do not. It also helps make
product or marketing fixes or even crate a new product line to cater to high growth potential groups.
 Validate existing conditions: Researchers widely use desvriptive research to help ascertain the research
object’s prevailing conditions and underlying patterns. Due to the non-invasive research method and the
use of quantitative observation and some aspects of qualitative observation, researchers observe each
variable an conduct an in-depth analysis. Researchers also use it to validate any existing conditions that
may be prevalent in a population.
 Conduct research at different times: The analysis can be conducted at different periods to ascertain any
similarities or differences. This also allows any number of variables to be evaluated. For verification,
studies on prevailing conductions can also be repeated to draw trends.
 Observational method

The observational method is the most effective method to conduct this research, and researchers make use of both
quantitative and qualitative observations.

A quantitative observation is the objective collection of data, which is primarily focused on numbers and values.
It suggests “associated with, of or depicted in terms of a quantity.” Results of quantitative observation are derived
using statistical and numerival analysis methods. It implies observation of any entity associated with a numeric
value such as age, shape, weight, volume, scale, etc. For example, the researcher can track if current customers
will refer the brand using a simple Net Promoter Score question.
Qualitative observation doesn’t involve measurements or numbers but instead jus monitoring characteristics. In
this case, the researcher observes the respondents from a distance. Since the respondents are in a comfortable
environment, the characteristics observed are natural and effective. In a descriptive research design, the researcher
observes the respondents from a distance. Since the respondents are in a comfortable environment, the
characteristics observed are natural and effective. In a descriptive research design, the researcher can choose to be
either a complete observer, an observer as a participant, a participant as an observer, or a full participant. For
example, in a supermarket, a researcher can from afar monitor and track the customers’ selection and purchasing
trends. This offers a more in-depth insight into the purchasing experience of the customer.
 Survey research: In survey research, respondents answer through surveys or questionnaires or polls.
They are a popular market research tool to collect feedback from respondents. A study to gather useful
data should have the right survey questions. It should be a balanced mix of open-ended questions and
close ended-questions. The survey method can be conducted online or offline, marking it the go-to option
for descriptive research where the sample size is enormous.

Examples of descriptive research

Some examples of descriptive research are:

1. A specialty food group launching a new range of barbecue rubs would like to understand what flavors of rubs
are favored by different people. To understand the preferred flavor palette, they conduct this type of research
 study using various methods like observational methods in supermarkets. By also surveying while collecting
in-depth demographic information, offers insights about the preference of different markets. This can also
help tailor make the rubs and spreads to various preferred meats in that demographic. Conducting this type of
research helps the organization tweak their business model and amplify marketing in core markets.
2. Another example of where this research can be used is if a school district wishes to evaluate teachers’
attitudes about using technology in the classroom. By conducting surveys and observing their
comfortableness using technology through observational methods, the researcher can gauge what they can
help understand if a full-fledged implementation can face an issue. This also helps in understanding if the
students are impacted in any way with this change.
Some other problems and research questions that can lead to descriptive research are:
 Market researchers want to observe the habits of consumers.
 A company wants to evaluate the morale of its staff.
 A school district wants to understand if students will access online lessons rather than textbooks.
 To understand if its wellness programs enhance the overall health of the employees.

Advantages of descriptive research

Some of the significant advantages of descriptive research are:

 Data collection: A researcher can conduct descriptive research using specific methods like observational
method, case study method, and survey method. Between these three, all primary data collection methods
are covered, which provides a lot of information. This can be used for future research or even developing
a hypothesis of your research object.
 Varied: Since the data collected is qualitative and quantitative, it gives a holistic understanding of a
research topic. The informatics is varied, diverse, and thorough.
 Natural environment: Descriptive research allows for the research to be conducted in the respondent’s
natural environment, which ensures that high-quality and honest data is collected.
 Quick to perform and cheap: As the sample size is generally large in descriptive research, the data
collection is quick to conduct and is inexpensive.

CORRELATION RESEARCH
To carried out to help explain important human behaviors or to predict likely outcomes.
Purposes of co relational research
Explanatory studies It is to clarify out understanding of important phenomena by indentifying relationship among
variables. Always investigate a number of variables they believe are related to a more complex variables such as
motivation or learning. Types of co relational research.
Selecting a problem choosing a sample Selecting or developing instrument Determining procedures Collecting
and analyzing data Interpreting result Basic steps
Teacher having difficulty in mathematic subject. Teacher about to study the causes of student does not perform in
the subject. Example
What? Investigator attempt to determine the cause or consequences of differences that already exit between or
among group of individuals. Sometimes viewed, along with co relational research, as a form of associational
research, since both describe conditions that already exist.
CAUSAL COMPARATIVE RESEARCH
That two groups of individuals differ on some variable ( such as teaching style ) and then interrupt to determine
the reason for, or the result of this difference. Example
Group Differences The group difference variable in a causal comparative study is: Either a variable that cannot be
manipulated (such as ethnicity) or one that might have been manipulated but for one reasons or another has not
been (such as teaching style) Example: in the effects of a new diet on very young children
It is about one of the type of research method that using comparison between cause and effect. Comparisons can
establish whether something can be explained by the same causes or not. The difference can be a conclusion but
not for the cause of difference. Interpretation of this kind of method is limited because these studies are of value
in identified possible causes of observed variation in the 38ndividu pattern of students. This kind of research can
be used in prediction about problem such as, the different achievements of student of the class. If it about teaching
methods, it can be research either it is related with art multimedia method or the self-learning methods.