Evidence Based Medicine

Beginners Handbook

Issued by:

Center of Evidence Based Medicine

Shifa College of Medicine
Islamabad

Version :

Date:

01 June 2006

Greetings from the EBM Task Force Members!

We welcome you to the Beginners EBM Workshop, and hope that you will enjoy the
experience.

________________________________________________________________________
CEBM-SCM
1

As you are aware EBM, is still a relatively new concept in Developing countries. As you
go through the workshop you will be convinced (if you are not already), that EBM should
play a vital role in countries with limited resources.
This manual is designed to supplement information given during the workshop. We have
not given detailed explanations regarding statistical concepts, as the aim is to teach
learners how to interpret statistical concepts rather then how to perform statistical
analysis; and to use research rather than how to do research. Therefore the little in the
manual about statistics is limited to the Glossary and Appendix.
The shift from traditional to evidence based medicine is doable , but not easy! Change
such as voluntarily finding and employing the best evidence, requires not just time and
skill but also a personal attitude and commitment to change. Ultimately physicians who
practice EBM say that it is not just better medicine for the patient but also for physicians.
Taking an evidence based approach to the care of patients is an intellectually exciting
style of practice, which leads you down a path of exploration and lifelong learning.
The task Force members welcome any suggestions that you may have for improvement
of the course and this hand book.
Good Luck,
Dr Mobeen Iqbal

________________________________________________________________________
CEBM-SCM
2

TABLE OF CONTENTS:
I.

Objectives of the manual

pg 4

II.

Introduction
What is EBM
Steps of EBM
Purpose of EBM

pg 5

III.

Formulating the Question / PICO

pg 13

IV.

The Research Pyramid

Study Designs
Levels of Evidence
Grades of Recommendation

Pg 16

V.

Finding the Best Evidence

Pg 32

VI.

Glossary of Terms

Pg 56

VII.

Appendix

Pg 61

________________________________________________________________________
CEBM-SCM
3

OBJECTIVES OF THE MANUAL:

This workbook aims to help you to find the best available evidence to answer your
clinical questions, in the shortest possible time. It will introduce the principles of
evidence based practice and provide a foundation of understanding and skills in:

Developing questions that are answerable from the literature

Searching for and identifying evidence to answer your question

________________________________________________________________________
CEBM-SCM
4

INTRODUCTION:

II

1. What is Evidence-Based Practice (EBP)?

Evidence-based practice: is integration of best research evidence with clinical expertise
and patient values1
When clinicians practice EBP:
The best available evidence, modified by patient circumstances
and preferences, is applied to improve the quality of clinical
judgements. 2

Evidence-based practice does not mean being dictated to by the literature nor is it an
attempt by journal publishers to take over the clinical world. It is just another tool you
can use to make sure your patients get the best possible care.

1- Sackett et al. 2000. Evidence based medicine. How to practice and teach EBM. Second
edition. Churchill Livingstone. London
2- McMaster Clinical Epidemiology Group 1997

________________________________________________________________________
CEBM-SCM
5

STEPS OF EBM

________________________________________________________________________
CEBM-SCM
6

Purposes of Evidence Based Medicine

Keeping Your Skills up to Date

"A well-used library is

one of the few antidotes
a general practitioner
has to the premature
senility that is liable to
overtake him. It is
astonishing with how
little reading a doctor
may practice medicine,
but it is not astonishing,
how badly he may do it"
Butler, "Equanimitas",
1901
In the years after you graduate, two things will happen:
1. Your memory of what you learned in medical school will lose its

freshness
2. New treatment methods will be found that they never taught you about in

school because they didn't exist.

If you are to remain a good doctor, or become a better one, you need to stay on
top of new developments as they occur. Evidence Based Medicine provides you
with the tools you need to find important new medical research quickly and
easily, and to work out its implications for your practice.

________________________________________________________________________
CEBM-SCM
7

Saving Time

Journals on Aging in
1974: 28

Journals on Aging in 1994: 95

As the above diagram shows, the amount of medical research now being published is so
great that it would take a very long time to read through every journal in your specialty in
paper form. Fortunately, computer indexes to the medical literature such as MEDLINE
allow you to do computer searches for the information you need quickly and easily.
The total amount of knowledge out there is far greater and often more reliable than the
clinical experience of one physician or even a group of experts. You no longer need to
read

through

masses

of

journals

in

order

to

take

advantage

of

it.

It is no longer your job to know everything, even in your chosen specialty. It is your job
to be able to find the information as and when you and your patients need it.

________________________________________________________________________
CEBM-SCM
8

Saving Lives

A detailed and exact knowledge of the outcomes of different treatments, derived from the
research, can often save lives. For example, consider the problem of whether to perform
an endarterectomy on a newly symptomatic patient with severe carotid stenosis. The
benefit of surgery in reducing the risk of a major stroke or fatality is summed up in the
following table (L. Goldstein et al, 1995):
% Patients with Major Stroke or Fatality
Surgery Group
No Surgery Number Needed to Treat
10%
19%
11
This means that you need only treat 11 patients, on average, to prevent a major stroke or
fatality - a clear and very substantial benefit. However, when asked whether they would
recommend surgery in such a case, physicians answered as follows:
Would you recommend carotid endarterectomy for
newly symptomatic patients with severe stenosis?
.

Seldom/Never Sometimes Usually/Always

All physicians

17%

28%

55%

Primary care physicians

47%
31%
20%
and internists
This seems to imply that it is at least possible that some patients are suffering a severe
stroke or fatality that can at least partly be attributed to physicians' lack of familiarity
with the medical literature.
________________________________________________________________________
CEBM-SCM
9

Supplementing Clinical Judgement

You may be asking yourself - Am I going to be replaced by a computer?

The answer is NO. The computer only gives you one of the many elements that will
enter into your clinical decision - information about how other patients in similar
situations have done. How you then apply this information to your patient is up to you.
Your patient will rarely match the patients in a research study exactly. He or she may be
younger or older than the population studies, may have other health issues, and will
certainly have his or her own views and values about medical treatment. The following
elements all help form your clinical decision:
1. The patient's situation
2. The patient's desires and values
3. Your values
4. Your experience
5. Evidence from research
EBM helps supply you with 5, evidence from research. It is up to you to judge how
applicable that information is to your patient and to decide on a course of action.

________________________________________________________________________
CEBM-SCM
10

NOTES

________________________________________________________________________
CEBM-SCM
11

Formulating the clinical question:

Clarifying the Problem Using PICO
If you dont know what you want to know, how can you know what you want or what
you got?
Spending a little time at the outset formulating your question will save a lot of time and
frustration at the end. The creation of the question is the first, crucial step in searching for
an answer. The literature search will flow from the issues raised in the question and will
be easier to perform if the question is explicit and concise. Sackett et al suggest that
clinical questions arise from the central tasks of clinical work. They suggest that in the
formulation of your clinical question, you first decide which type of clinical work you are
addressing and offer the following categories of Central Tasks:

Clinical findings: how to gather and interpret findings from the history and
physical

Etiology

Differential diagnosis

Diagnostic testing

Prognosis

Therapy

Prevention and screening

Economic analysis

Self-improvement: how to keep up to date or improve your skills

In creating your clinical question, answer the following:

Which of the central tasks am I addressing?

Why do I need the answer to this question?

What are the characteristics of the patient or patients I am interested in?

What is the specific disease or medical issue involved?

Where am I most likely to find the answer to this question: textbooks,

systematic reviews, original research reports, experts, consultants?

What is the most appropriate search strategy?

________________________________________________________________________
CEBM-SCM
12

III

As clinicians, our most fundamental questions center on diagnosis, prognosis, or therapy in

individual patients. The acronym POEM (Patient Oriented Evidence that Matters) suggests
that questions are most relevant when they relate to specific patients or clearly identified
groups of patients. To build your final question, the question that you will try to answer,
focus on the following steps:

1- Describe your own patient in as much relevant detail as you can: In a 54 year old
white male with new onset of paroxysmal atrial fibrillation and a large left atrium
2- Describe what you want to answer or ask about your specific patient:

would

anticoagulation with warfarin

3- Describe the comparison or the alternative: as opposed to no therapy at all
4- Describe your goal or desired outcome: reduce the risk of embolic stroke or reduce
mortality?
This is called a PICO question: Patient Intervention Control Outcome
When you have addressed these issues, write out the question. Writing will force you to
be more explicit. Pay special attention to the outcome. Lowering cholesterol is not the
same as preventing heart attacks. Pick an outcome that is relevant to your patient.
PICO stands for:

Patient or
population

Describes patient (age, sex, race,past

medical history, etc.)

Intervention (or
exposure)

What happens or is to be done; treatment,

diagnostic test, exposure (e.g. passive
smoking)

Comparison

Compared to what? Nothing, placebo,

another intervention

Outcomes
(preferably
clinical)

What is the effect of the intervention?

(Be specific; mortality, hospitalizations).

Before starting a search, write down the answers to these PICO questions. The key
elements in the answers will become search terms in your on-line search. (Appendix 1)

________________________________________________________________________
CEBM-SCM
13

NOTES

________________________________________________________________________
CEBM-SCM
14

Research Methods:
The Evidence Pyramid

MEDLINE and the other online medical literature databases try to be as comprehensive
as possible in their coverage. As a result, indexed material may have little direct
application to present-day medical practice.
The different types of material indexed in MEDLINE are labeled in the pyramid diagram,
with the least clinically relevant at the bottom and the most clinically relevant at the top.
The four layers above case reports and case series represent actual clinical research; the
layers below are least clinically relevant and can be useful as background resources.
The next few pages provide basic definitions and examples of clinical research designs to
help the medical student or new clinician understand how the design of a research study
may affect whether or not to accept its findings in caring for a patient.

________________________________________________________________________
CEBM-SCM
15

IV

Randomized Controlled Studies

A randomized controlled study is one in which:

1. There are two groups, one treatment group and one control group. The treatment
group receives the treatment under investigation, and the control group receives
either no treatment or some standard default treatment.
2. Patients are randomly assigned to all groups.
Assigning patients at random reduces the risk of bias and increases the probability that
differences between the groups can be attributed to the treatment.
Having a control group allows us to compare the treatment with alternative choices. For
instance, the statement that a particular medication cures 40% of cases tells us very little
unless we also know how many cases get better on their own! (Or with a different
treatment). With certain research questions, randomized controlled studies cannot be
done for ethical reasons. For instance, it would be unethical to attempt to measure the
effect of smoking on health by asking one group to smoke two packs a day and another
group to abstain, since the smoking group would be subject to unnecessary harm.
Randomized controlled trials are the standard method of answering questions about the
effectiveness of different therapies. If you have a therapy question, first look for a
randomized controlled trial, and only go on to look for other types of studies if you don't
find one.
________________________________________________________________________
CEBM-SCM
16

The Double Blind Method

A double blind study is one in which neither the patient nor the physician knows
whether the patient is receiving the treatment of interest or the control treatment.
For example, studies of treatments that consist essentially of taking pills are very
easy to do double blind - the patient takes one of two pills of identical size, shape,
and color, and neither the patient nor the physician needs to know which is which.
A double blind study is the most rigorous clinical research design because, in
addition to the randomization of subjects which reduces the risk of bias, it can
eliminate the placebo effect which is a further challenge to the validity of a study.
The placebo effect could be thought of in this way:
1. Patients who believe they are receiving a new experimental treatment tend to be
more optimistic about the outcome. This means that, when asked, they tend to
minimize health problems and give more weight to positive effects. They also
tend to take better care of themselves and comply better with the conditions of the
experiment. There is also substantial evidence that, independent of all this,
patients who have positive beliefs about their treatment do better than patients
who do not. In many situations, the placebo effect is at least as strong as any
objective effects of the treatment!
2. Doctors who believe that a patient is receiving a new experimental treatment tend
to be more optimistic about that patient's chances, evaluate their state of health
more favorably, and communicate positive expectations to the patients, who in
turn try to get better so as to prove their doctor right!
________________________________________________________________________
CEBM-SCM
17

Cohort Studies

A Cohort Study is a study in which patients who presently have a certain condition and/or
receive a particular treatment are followed over time and compared with another group
who are not affected by the condition under investigation.
For instance, since a randomized controlled study to test the effect of smoking on health
would be unethical, a reasonable alternative would be a study that identifies two groups, a
group of people who smoke and a group of people who do not, and follows them forward
through time to see what health problems they develop.
Cohort studies are not as reliable as randomized controlled studies, since the two groups
may differ in ways other than in the variable under study. For example, if the subjects
who smoke tend to have less money than the non-smokers, and thus have less access to
health care, that would exaggerate the difference between the two groups.
The main problem with cohort studies, however, is that they can end up taking a very
long time, since the researchers have to wait for the conditions of interest to develop.
Physicians are, of course, anxious to have meaningful results as soon as possible, but
another disadvantage with long studies is that things tend to change over the course of the
study. People die, move away, or develop other conditions, new and promising treatments
arise, and so on. Even so, cohort studies are generally preferred to case control studies,
since they involve far fewer statistical problems and generally produce more reliable
answers.
________________________________________________________________________
CEBM-SCM
18

Case Control Studies

Case control studies are studies in which patients who already have a certain condition
are compared with people who do not.
For example: a study on which lung cancer patients are asked how much they smoked in
the past and the answers are compared with a sample of the general population would be
a case control study.
Case control studies are less reliable than either randomized controlled trials or cohort
studies. Just because there is a statistical relationship between two conditions does not
mean that one condition actually caused the other. For instance, lung cancer rates are
higher for people without a college education (who tend to smoke more), but that does
not mean that someone can reduce his or her cancer risk just by getting a college
education.
The main advantages of case control studies are:

They can be done quickly. By asking patients about their past history, researchers
can quickly discover effects that otherwise would take many years to show
themselves.
Researchers don't need special methods, control groups, etc. They just take the
people who show up at their institution with a particular condition and ask them a
few questions.

The first study to suggest a new medical conclusion will often be a case control study,
perhaps designed to check on a hypothesis suggested by a case series. If possible,
researchers will generally try to confirm the results with a randomized controlled trial or
a cohort study.

________________________________________________________________________
CEBM-SCM
19

Case Series and Case Reports

Case series and case reports consist either of collections of reports on the treatment of
individual patients, or of reports on a single patient.
For example: one of your patients has a condition that you have never seen or heard of
before and you are uncertain what to do. A search for case series or case reports may
reveal information that will assist in a diagnosis. However, for any reasonably wellknown condition you should be able to get better evidence. Case series and case reports,
since they use no control group with which to compare outcomes, have no statistical
validity.

________________________________________________________________________
CEBM-SCM
20

Systematic Reviews and Meta-Analyses

Important medical questions are typically studied more than once, often by different
research teams in different locations.
A systematic review is a comprehensive survey of a topic in which all of the primary
studies of the highest level of evidence have been systematically identified, appraised and
then

summarized

according

to

an

explicit

and

reproducible

methodology.

A meta-analysis is a survey in which the results of all of the included studies are similar
enough statistically that the results are combined and analyzed as if it was one study. In
general a good systematic review or meta-analysis will be a better guide to practice than
an individual article.
Pitfalls specific to meta-analysis include:
1. It's rare that the results of the different studies precisely agree, and often the
number of patients in a single study is not large enough to come up with a
decisive conclusion.
2. If the authors are interested in supporting a particular conclusion, they can include
studies that support that conclusion and omit studies that do not. Do the authors

________________________________________________________________________
CEBM-SCM
21

explain in their paper exactly on what basis they included studies, and do their
reasons make sense?
3. Studies that show some kind of positive effect tend to be published more often
than those that do not. This means that if the authors include only published
studies, several weak positive studies may seem to add up to a strong positive
result. Do weak negative studies exist? This effect is known as Publication bias.

________________________________________________________________________
CEBM-SCM
22

LEVELS OF EVIDENCE & GRADES OF

RECOMMENDATION
What are we to do when the irresistible force of the need to offer clinical advice meets
with the immovable object of flawed evidence? All we can do is our best: give the
advice, but alert the advisees to the flaws in the evidence on which it is based.
The ancestor of this set of pages was created by Suzanne Fletcher and Dave Sackett 20
years ago when they were working for the Canadian Task Force on the Periodic Health
Examination [1]. They generated "levels of evidence" for ranking the validity of evidence
about the value of preventive manoeuvres, and then tied them as "grades of
recommendations" to the advice given in the report.
The levels have evolved over the ensuing years, most notably as the basis for
recommendations about the use of anti-thrombotic agents [2], have grown increasingly
sophisticated [3], and have even started to appear in a new generation of evidence-based
textbooks that announce, in bold marginal icons, the grade of each recommendation that
appears in the texts [4] in bold icons.
A final, cautionary note: these levels and grades speak only to the validity of evidence
about prevention, diagnosis, prognosis, therapy, and harm. Other strategies, described
elsewhere in the manuals pages, must be applied to the evidence in order to generate
clinically useful measures of its potential clinical implications and to incorporate vital
patient-values into the ultimate decisions.

________________________________________________________________________
CEBM-SCM
23

Oxford Centre for Evidence-based Medicine Levels of Evidence (May 2001)

Level
1a

Diagnosis
Differential
Economic and decision
Therapy/Prevention, Prognosis
diagnosis/symptom
analyses
Aetiology/Harm
prevalence study
SR (with
SR (with
SR (with
SR (with homogeneity*) SR (with
homogeneity*) of homogeneity*) of homogeneity*) of homogeneity*) of
of RCTs
inception cohort studies; Level 1 diagnostic

CDR validated in
different populations
1b

Individual RCT (with narrow

Confidence Interval)

prospective cohort
studies; CDR with 1b studies
studies from different
clinical centres

Level 1 economic studies

Individual inception
Validating** cohort study Prospective cohort study
cohort study with > 80% with good
with good follow-up****
follow-up; CDR
reference standards; or
validated in a single
CDR tested within
population
one clinical centre

Analysis based on
clinically sensible costs
or alternatives; systematic
review(s) of the evidence;
and including multi-way
sensitivity analyses

1c

All or none

All or none case-series

Absolute SpPins
and SnNouts

All or none case-series

Absolute better-value or
worse-value analyses

2a

SR (with homogeneity* )
of cohort studies

SR (with

SR (with

SR (with

SR (with

2b

homogeneity*) of homogeneity*) of homogeneity*) of homogeneity*) of

either retrospective
cohort studies or
untreated control groups
in RCTs
Individual cohort study (including Retrospective cohort
low quality RCT; e.g., <80%
study or follow-up of
follow-up)
untreated control patients
in an RCT; Derivation of

2b and better studies

Level >2 economic

studies

Retrospective cohort
Analysis based on
study, or poor follow-up clinically sensible costs
or alternatives; limited
goodreference
review(s) of the evidence,
standards; CDR after
or single studies; and
CDR or validated on derivation, or validated
including multi-way
split-sample only
only on split-sample
sensitivity analyses
or databases

2c

"Outcomes" Research; Ecological "Outcomes" Research

studies

3a

SR (with homogeneity*)
of case-control studies

3b

Individual Case-Control Study

Case-series (and poor

Expert opinion without explicit

critical appraisal, or based on
physiology, bench research or
"first principles"

Exploratory** cohort
study with

Ecological studies
SR (with

SR (with

Audit or outcomes
research
SR (with

homogeneity*) of homogeneity*) of homogeneity*) of

Case-series (and poor

quality cohort and

quality
case-control studies prognostic
)
cohort
studies***)
5

Level >2 diagnostic

studies

Expert opinion without

explicit critical appraisal,
or based on physiology,
bench research or "first
principles"

3b and better studies

3b and better studies

Non-consecutive study;
or without consistently
applied reference
standards

Non-consecutive
cohort study, or very
limited population

Expert opinion without

explicit critical appraisal,
or based on physiology,
bench research or "first
principles"

Expert opinion without

explicit critical appraisal,
or based on physiology,
bench research or "first
principles"

3b and better studies

Analysis based on limited

alternatives or costs, poor
quality estimates of data,
but including sensitivity
analyses incorporating
clinically sensible
variations.
Case-control study, poor Case-series or superseded Analysis with no
or non-independent
reference standards
sensitivity analysis
reference standard

Expert opinion without

explicit critical appraisal,
or based on economic
theory or "first
principles"

Produced by Bob Phillips, Chris Ball, Dave Sackett, Doug Badenoch, Sharon Straus,
Brian Haynes, Martin Dawes since November 1998.
________________________________________________________________________
CEBM-SCM
24

Notes
Users can add a minus-sign "-" to denote the level of that fails to provide a conclusive answer because of:

EITHER a single result with a wide Confidence Interval (such that, for example, an ARR in an RCT
is not statistically significant but whose confidence intervals fail to exclude clinically important benefit or
harm)

OR a Systematic Review with troublesome (and statistically significant) heterogeneity.

Such evidence is inconclusive, and therefore can only generate Grade D recommendations.

By homogeneity we mean a systematic review that is free of worrisome variations (heterogeneity) in the directions
and degrees of results between individual studies. Not all systematic reviews with statistically significant
heterogeneity need be worrisome, and not all worrisome heterogeneity need be statistically significant. As noted
above, studies displaying worrisome heterogeneity should be tagged with a "-" at the end of their designated level.

Clinical Decision Rule. (These are algorithms or scoring systems which lead to a prognostic estimation or a
diagnostic category. )

See note #2 for advice on how to understand, rate and use trials or other studies with wide confidence intervals.

Met when all patients died before the Rx became available, but some now survive on it; or when some patients
died before the Rx became available, but none now die on it.

By poor quality cohort study we mean one that failed to clearly define comparison groups and/or failed to measure
exposures and outcomes in the same (preferably blinded), objective way in both exposed and non-exposed
individuals and/or failed to identify or appropriately control known confounders and/or failed to carry out a
sufficiently long and complete follow-up of patients. By poor quality case-control study we mean one that failed to
clearly define comparison groups and/or failed to measure exposures and outcomes in the same (preferably
blinded), objective way in both cases and controls and/or failed to identify or appropriately control known
confounders.
Split-sample validation is achieved by collecting all the information in a single tranche, then artificially dividing
this into "derivation" and "validation" samples.

An "Absolute SpPin" is a diagnostic finding whose Specificity is so high that a Positive result rules-in the
diagnosis. An "Absolute SnNout" is a diagnostic finding whose Sensitivity is so high that a Negative result rulesout the diagnosis.

Good, better, bad and worse refer to the comparisons between treatments in terms of their clinical risks and
benefits.
Good reference standards are independent of the test, and applied blindly or objectively to applied to all patients.
Poor reference standards are haphazardly applied, but still independent of the test. Use of a non-independent
reference standard (where the 'test' is included in the 'reference', or where the 'testing' affects the 'reference')
implies a level 4 study.
Better-value treatments are clearly as good but cheaper, or better at the same or reduced cost. Worse-value
treatments are as good and more expensive, or worse and the equally or more expensive.
**

Validating studies test the quality of a specific diagnostic test, based on prior evidence. An exploratory study
collects information and trawls the data (e.g. using a regression analysis) to find which factors are 'significant'.

***

By poor quality prognostic cohort study we mean one in which sampling was biased in favour of patients who
already had the target outcome, or the measurement of outcomes was accomplished in <80% of study patients, or
outcomes were determined in an unblinded, non-objective way, or there was no correction for confounding factors.

**** Good follow-up in a differential diagnosis study is >80%, with adequate time for alternative diagnoses to emerge
(eg 1-6 months acute, 1 - 5 years chronic)

________________________________________________________________________
CEBM-SCM
25

Grades of Recommendation
A
B
C
D

consistent level 1 studies

consistent level 2 or 3 studies or extrapolations from level 1 studies
level 4 studies or extrapolations from level 2 or 3 studies
level 5 evidence or troublingly inconsistent or inconclusive studies of any level

"Extrapolations" are where data is used in a situation which has potentially clinically
important differences than the original study situation.

________________________________________________________________________
CEBM-SCM
26

NOTES

________________________________________________________________________
CEBM-SCM
27

FINDING THE BEST EVIDENCE:

EBM Searching Resources

IV

Background Resources: (give overview of a topic; print textbooks, electronic

textbooks, narrative reviews in journals)
1. Harrisons Online
2. Scientific American Medicine Online
3. MD Consult
4. Medline (for narrative review articles) a. Ovid
b. PubMed

Foreground Resources: (reports of original research or abstracts, summaries,

syntheses of primary research)
o

Secondary Sources-abstracts, summaries and syntheses of original

research
1. Cochrane Database of Systematic Reviews- systematic review of a topic
a. through Ovid
b. direct as Cochrane Collaboration
2. ACP Journal Club- review and commentary of an article
a. through Ovid
b. direct (coming soon)
3. Clinical Evidence - summarizes evidence available for common
clinical questions-includes Cochrane and ACP Journal Club report (direct)
4. Practice Guidelines

Primary Sources-reports of original research

1. OVID Medline
2. PubMed Medline

________________________________________________________________________
CEBM-SCM
28

How to Access EBM Resources:

Ovid (Medline, Best Evidence, Cochrane, DARE)--ID and Password

required

PubMed--Open access to all

Cochrane Collaboration --Password Protected (follow on-screen directions)

Clinical Evidence --Password Protected (follow on-screen directions)

Practice Guidelines --Open access to all

Steps of an EBM Search:

When you do an EBM search, you are trying to find the most useful research in the
shortest possible time. We can express the usefulness of a search as follows:

Usefulness = Relevance * Validity / Effort

Steps for using EBM to answer to a medical problem

Clarify the question and work out what search terms you need using the PICO
method (Population, Intervention, Comparison, Outcome). This method is
described in more detail later in this chapter.
Classify the problem as one of Therapy, Diagnosis, Prognosis, or Harm.
Do on-line searches for the terms that come out of the PICO analysis and combine
them appropriately using AND and OR.
Use AND or Limit Set to restrict the results of the previous step to the type of
research that applies (for instance, randomized controlled trials or cohort studies.
The type of studies you look for will depend on how you classified the problem in
step 2.
View and print out , the results of the studies you retrieved which seem useful and
relevant. and to have some reasonable kind of study design.
Carefully review the research design methods used in the studies you selected in
step 5.
Use the Evaluation methods described in the next section of this course.
Draw your conclusions and apply them to your patient(s).

________________________________________________________________________
CEBM-SCM
29

Therapy, Diagnosis, Prognosis, or Harm?

In order to apply Evidence Based medicine to your clinical problem, the first step is to
classify the problem:

Therapy
Diagnosis
Prognosis, or
Harm

These classifications are useful in helping you pick a good search strategy for finding
studies that apply to your problem, and they also help you to find out which of the studies
you find are valid and helpful. If the differences between these problem types isn't clear
to you immediately, don't worry - there are examples of each type later in the chapter.

Therapy problems are questions about what treatment, if any, to

Therapy give a patient, and what the outcomes of different treatment
options might be.
Diagnosis problems are questions about the degree to which a
particular test is reliable and clinically useful, generally asked in
order to decide whether a patient of yours would get enough
Diagnosis benefit from the test, on average, to justify having it done. Most
articles on diagnosis compare the results of the diagnostic test
being studied to the results of another standard test that is
regarded as being definitive - a 'gold standard' test.
Prognosis problems are questions about a patient's future health,
life span, and quality of life in the event that s/he chooses a
particular treatment option. For instance, how do the lifespan
Prognosis
and quality of life of an elderly patient undergoing surgery for
prostate cancer compare with those for a similar patient who
chooses not to undergo the surgery?
Harm

Harm problems are questions about the relationship between a

disease and a possible cause. For example, does a diet rich in
saturated fats increase the risk of heart disease, and if so, by how
much?

________________________________________________________________________
CEBM-SCM
30

Finding Good Therapy Studies

Finding Randomized Controlled Studies
(For pubmed details refer to Appendix 2)
For most therapy questions, you will want to look for the best kind of evidence, a
randomized controlled study. If the study can be a double blind one, so much the better.
There

are

two

common

ways

of

finding

randomized

controlled

studies:

Limit your search to EBM Reviews which will look only for articles from Cochrane,
ACP Journal Club, and Database of Abstracts of Reviews of Effectiveness (DARE)
1. Limit your search to the Publication Type Randomized Controlled Trial. This
option gives citations that are the actual reports of randomized controlled trials.
2. Using the MESH term Randomized Controlled Trials is not recommended
because as a subject heading it is used as a discussion of the method in medical
research, not the report of actual trials.
a. Doing a textword search for random: (i.e. any word that starts with the
letters random in the title or abstract of the article). This is a very
inclusive search you can use if the first type of search doesn't come up
with anything useful (also not recommended)

Finding Double Blind Studies:

A good way to find double blind studies is the MESH term Double Blind Method. If you
need a less restrictive search, try a text word search for blind.

Finding Meta-analyses and Systematic Reviews:

To find Meta-analyses, use the limit to Publication type and click on Meta Analysis
To find systematic reviews in Ovid Medline, you will need to use the limit to Publication
type

and

limit

to

both Randomized

Controlled

Trial

and

Meta

Analysis

Click on Meta analysis and hold down the Ctrl button and then scroll down to
________________________________________________________________________
CEBM-SCM
31

Randomized Controlled Trial and while holding down the Ctrl button, click on
Randomized Controlled Trial.

Therapy Example:
You have several older patients with chronic rheumatoid arthritis who have been
receiving nonsteroidal anti-inflammatory drugs (NSAIDS). The NSAID therapies place
these patients at risk for ulcers and life-threatening complications.
You would like to prescribe Misoprostol in addition to the NSAID therapy, so as to help
reduce the gastrointestinal complications. You decide to do a literature search for studies
about the effectiveness of Misoprostol in reducing these complications.
You fill out your PICO worksheet as follows:

Population

Older patients with rheumatoid arthritis

Intervention Misoprostol in addition to NSAIDS

Comparison NSAIDS alone
Outcome
Effect on gastrointestinal complications
The entries in the PICO table translate into searches as follows:

Older patients with

rheumatoid arthritis

arthritis, rheumatoid (Exploded MESH) .

Type rheumatoid arthritis, select from
MESH and then explode.
Limit to 45 years and older. Go to Limit
page, select appropriate set, scroll down to
Age Groups, select those 45 and over .
Misoprostol (MESH)

Misoprostol in
addition to NSAIDS

anti-inflammatory agents, non-steroidal

(Exploded MESH). Type NSAIDS, select
appropriate MeSH term and explode.

NSAIDS alone

(Already covered above)

Effect on
gastrointestinal
complications

Look at the titles to see which articles may

be relevant

________________________________________________________________________
CEBM-SCM
32

If OVID's EBMR is available, the fastest and most efficient way to search for valid
articles is to search for the disorder, then the intervention, combine the two sets and then
limit to EBM reviews. The system will perform the search in the EBM databases--

Cochrane, ACP Journal Club and DARE.

If none of the articles retrieved answers your question or the limit to EBM Reviews
comes up with no articles, then you will have to perform a more extensive search by
combining and then limiting.

Restricting to Randomized Controlled Trials

To restrict the search to randomized controlled trials, Go to the Limit option (at top of
Main search page), select your last search, scroll down to Publication Types and then
scroll down to Randomized Controlled Trial (it's quite a long way down the list, which
is in alphabetical order: it doesn't show up on the first screen). Select Randomized
Controlled Trial and then click on Limit Search which will take you back to the Main
Search Screen.

Restricting to Double Blind Studies

To restrict to Double Blind Studies, type in double blind, select the MESH heading
double-blind method, and do an AND with the results of your previous search. In this
particular case, all the randomized controlled studies are also double blind.

________________________________________________________________________
CEBM-SCM
33

Finding Good Diagnosis Studies

The fastest and most efficient way to search for valid articles on diagnosis is to type in
the diagnosis and use the appropriate MeSH term, type in the diagnostic test, combine,
and then limit to EBM Reviews. This will retrieve articles from ACP Journal Club which
have already been determined to be valid. If there are no appropriate articles then the
next quickest and thorough way to find articles that relate to problems of diagnosis is the
MESH heading Sensitivity and Specificity. To get this heading, type in Sensitivity: you
will get a chance to choose the full MESH heading Sensitivity and Specificity in the
next screen.
Explode Sensitivity and Specificity so as to include ROC curves and Predictive Value
of Tests. Both are more specific terms that are being used more and more nowadays
instead of, or in addition to, classic measures of the performance of a test in terms of
sensitivity and specificity.
If you find you have too many studies, you can also do an AND of your results with the
MESH heading double-blind method. This will restrict your search to double blind
studies, which are the most reliable type. If there are no double blind studies, you can do
a textword search for the word 'blind', which hopefully will give you studies that have at
least some degree of blinding.

Search Procedure
1. Search for the terms that arise out of your focused clinical question (PICO ).
2. Do a search that combines them
3. Limit that set to EBM Reviews
4. If no articles are appropriate then do the next steps.
5. Type in Sensitivity and press Enter.
6. Select the MESH term Sensitivity and Specificity.
7. Explode that term to include the narrower MeSH headings: ROC curves and
Predictive Value of Tests.
8. Do an AND between the results of step 2. and the results of step 5.
9. If you have too many studies, type in the word double, choose the MESH heading
double-blind method, and do an AND with the results of the previous step.
________________________________________________________________________
CEBM-SCM
34

Finding Good Prognosis Studies

The MESH term Prognosis is an effective way of finding studies relating to prognosis.
The best study design for answering questions about prognosis will generally be the
cohort study. For most prognosis questions of clinical interest, randomized controlled
trials are ruled out for ethical reasons.
The best way to find cohort studies is to explode the MESH term Cohort Studies.
For the above reasons, a good strategy for finding studies that relate to prognosis is to do
exploded searches for Prognosis and Cohort Studies and combine the exploded
Prognosis search and the exploded Cohort Studies search using OR.
Depending on what type of prognosis you are interested in, other terms such as:

Mortality (exploded MESH)

Morbidity (exploded MESH)

Risk (exploded MESH)

may also be useful to you in your search.

Finding Good Studies of Harm:

The MESH heading that applies most directly to studies of Harm is Risk.
The method almost always used for studies of Harm is the Cohort Study. Randomized
controlled studies of Harm questions are generally not possible for ethical reasons, and
Case Control studies or Case Series generally do not provide strong enough evidence
(though you may have to resort to them if you are desperate for any information at all
about what might be causing a certain health problem.)
This means that your best simple strategy is to do searches for the MESH terms Risk and
Cohort Study (both exploded) and then combine them using OR. If you want a slightly
broader search, you can do Textword searches for Risk and cohort, which will give you
slightly more studies.

Finding Good Meta-Analyses, Guidelines, etc.

Finding meta-analyses:
Limit to the publication type Meta Analysis.
________________________________________________________________________
CEBM-SCM
35

Broadening and Narrowing Your Search:

If You Have Too Many Studies
If you have too many studies to review in a reasonable time, take a look at the first few
studies and see which of them are potentially useful to you and which are not.

If most of them do appear to be useful,

If they mostly seem to support the same conclusion, your question may be
answered.
If there appears to be some disagreement about the conclusions, then look for a
systematic review, meta-analysis , guideline, or review article.

If most of these studies are not useful, ask why!

If they are animal or lab research, use the Limit option to limit your search to
human studies.
If they are just case reports, letters to the editor, and so on, combine your search
with one that searches for real clinical studies. Clinical Trials (Exploded) is a
good inclusive search term for real clinical studies.
If they are real human research but don't quite relate to your problem, you may
need to refine the answers to your PICO questions. If you are only interested in a
particular age group, then you can limit the search to that age group. If you are
interested in side effects of a medication, re-do the search choosing the Adverse
Effects subheading of the MESH entry for that medication. Find a way to define
exactly what you are interested in, in terms MEDLINE understands.
If they are real human research and relate to your problem but aren't of the best
design, either just skip those studies or else combine your search with a more
restrictive one on study type such as randomized controlled trials, prospective
studies, or cohort studies .
If you have a lot of studies that relate to the problem but that seem to be of poor
quality, you may want to limit your search to AIM journals. These are the core
medical journals that most health care institutions will carry, so an additional
benefit is that you will probably be able to look up the full text of any relevant
studies you find.

If You Don't Have Enough Studies:

Of course, even a single, well-designed, conclusive study that applies exactly to your
problem can be 'enough', so this really means 'if you don't have any conclusive studies'.

________________________________________________________________________
CEBM-SCM
36

You have to use your imagination a little when you try to broaden your search to find
more studies, since you can't see the studies you didn't find, and don't know what they
look like! Some ways to broaden a search are:

Consider possible alternative words for the ideas you've been searching for,
search for the different words, and combine them using OR. You say 'newborn',
someone else may say 'neonate', 'baby', or even 'infant'.

Make your MESH headings as inclusive as possible. Use Explode , Include all
Subheadings, and All Documents (rather than Restrict to Focus ).

Use less search terms. For instance, if you entered mortality in the Outcome
section of your PICO questions and made a search term out of it that you
combined with the other terms using AND, you may be missing some studies that
describe mortality but don't have it as a keyword.

Consider using a less restrictive study type - for instance, clinical trials instead of
randomized controlled trials. There may really not be any studies of the most
desirable type, or there may be studies that are of the right type but have been
mis-classified. The MESH heading randomized controlled trials, in particular,
fails to include many perfectly good randomized controlled trials.

________________________________________________________________________
CEBM-SCM
37

________________________________________________________________________
CEBM-SCM
38

NOTES

________________________________________________________________________
CEBM-SCM
39

APPENDIX 1
Centre for Evidence-Based Medicine, Institute of Health Sciences Old
Road Campus, Headington, Oxford, OX3 7LF, United Kingdom. T:
+44 (0)1865 226991; F: +44 (0)1865 226845

Educational
Prescription

Date and Place to be

presented:___________________________________________________
Patient:
_____________________________________________________________
Intervention (therapeutic, diagnostic, prognostic, causal):
Comparison ___________________________________________
Outcome/s (a change in the risk or likelihood of):
_____________________________________________________________
The Learner:
_____________________________________________________________

Presentations will cover:

1. HOW you found what you found, i.e. Search Strategies;
2. WHAT you found (the bottom line);
3. The VALIDITY and APPLICABILITY of what you found (the
critical appraisal);
4. How what you found will ALTER your MANAGEMENT of such
patients;
5. How WELL you think you DID in filling this Rx.

________________________________________________________________________
CEBM-SCM
40