Checklist for Quasi-Experimental Appraisal Tool
Checklist for Quasi-Experimental Appraisal Tool
Checklist for Quasi-Experimental Appraisal Tool
EXPERIMENTAL STUDIES
(NON-RANDOMIZED
EXPERIMENTAL STUDIES )
Critical Appraisal tools for use in JBI Systematic Reviews
INTRODUCTION
JBI is an international research organisation based in the Faculty of Health and Medical Sciences at the
University of Adelaide, South Australia. JBI develops and delivers unique evidence-based information,
software, education and training designed to improve healthcare practice and health outcomes. With over
70 Collaborating Entities, servicing over 90 countries, JBI is a recognised global leader in evidence-based
healthcare.
________________________________________________________________________________________________
________________________________________________________________________________________________
________________________________________________________________________________________________
Critical Appraisal Tool for Quasi-Experimental Studies (Experimental Studies without random
allocation)
1. Is it clear in the study what is the ‘cause’ and what is the ‘effect’ (i.e. there is no
confusion about which variable comes first)?
Ambiguity with regards to the temporal relationship of variables constitutes a threat to the internal validity
of a study exploring causal relationships. The ‘cause’ (the independent variable, that is, the treatment or
intervention of interest) should occur in time before the explored ‘effect’ (the dependent variable, which is
the effect or outcome of interest). Check if it is clear which variable is manipulated as a potential cause.
Check if it is clear which variable is measured as the effect of the potential cause. Is it clear that the ‘cause’
was manipulated before the occurrence of the ‘effect’?
5. Were there multiple measurements of the outcome both pre and post the
intervention/exposure?
In order to show that there is a change in the outcome (the ‘effect’) as a result of the
intervention/treatment (the ‘cause’) it is necessary to compare the results of measurement before and
after the intervention/treatment. If there is no measurement before the treatment and only measurement
after the treatment is available it is not known if there is a change after the treatment compared to before
the treatment. If multiple measurements are collected before the intervention/treatment is implemented
then it is possible to explore the plausibility of alternative explanations other than the proposed ‘cause’
(the intervention of interest) for the observed ‘effect’, such as the naturally occurring changes in the
absence of the ‘cause’, and changes of high (or low) scores towards less extreme values even in the
absence of the ‘cause’ (sometimes called regression to the mean). If multiple measurements are collected
after the intervention/treatment is implemented it is possible to explore the changes of the ‘effect’ in time
in each group and to compare these changes across the groups. Check if measurements were collected
before the intervention of interest was implemented. Were there multiple pre-test measurements? Check
if measurements were collected after the intervention of interest was implemented. Were there multiple
post-test measurements?