The Journal of Emergency Medicine, Vol. 45, No. 4, pp.

508–519, 2013
Copyright Ó 2013 Elsevier Inc.
Printed in the USA. All rights reserved
0736-4679/$ - see front matter

http://dx.doi.org/10.1016/j.jemermed.2013.03.042

Original
Contributions

DOES THIS CHILD HAVE BACTERIAL MENINGITIS? A SYSTEMATIC REVIEW OF

CLINICAL PREDICTION RULES FOR CHILDREN WITH SUSPECTED BACTERIAL
MENINGITIS

Dina M. Kulik, MD, FRCPC,* Elizabeth M. Uleryk, BA, MLS,† and Jonathon L. Maguire, MSC FRCPC‡§jj{#**
*Division of Paediatric Emergency Medicine, Hospital for Sick Children, Toronto, Ontario, Canada, †Hospital for Sick Children, Hospital Library,
Toronto, Ontario, Canada, ‡Department of Pediatrics, St. Michael’s Hospital, Toronto, Ontario, Canada, §Keenan Research Centre, Li Ka
Shing Knowledge Institute of St. Michael’s Hospital, Toronto, Ontario, Canada, jjDivision of Pediatric Medicine and the Pediatric Outcomes
Research Team (PORT), Hospital for Sick Children, Toronto, Ontario, Canada, {Child Health Evaluative Sciences, Hospital for Sick Children
Research Institute, Toronto, Ontario, Canada, #Department of Health Policy Management and Evaluation, University of Toronto, Toronto,
Ontario, Canada, and **Department of Paediatrics, University of Toronto, Toronto, Ontario, Canada
Reprint Address: Dina M. Kulik, MD, FRCPC, Pediatric Emergency Medicine, The Hospital for Sick Children, 555 University Avenue, Toronto,
Ontario, Canada M5G 1X8

, Abstract—Background: Acute meningitis is a relatively adequate blinding, predictor reproducibility assessment,

common phenomenon in children. Identifying which chil- and meticulous follow-up of outcomes. The Bacterial Menin-
dren are most likely to have bacterial meningitis vs. self- gitis Score had the highest quality and performance and is
limiting aseptic meningitis is important, as these children the best candidate for prospective validation. Conclusions:
require investigation and antibiotic treatment. Objective: Until consistently high methodological quality and diagnos-
Our aim was to systematically identify and review the qual- tic performance are demonstrated through prospective
ity and performance of published clinical prediction rules validation, caution is warranted in the routine clinical use
(CPRs) for children with suspected bacterial meningitis. of existing CPRs for children with suspected bacterial
Methods: Medline and Embase were searched for CPRs in- meningitis. Ó 2013 Elsevier Inc.
volving children 0–18 years of age with suspected bacterial
meningitis, with cerebral spinal fluid (CSF) culture used as , Keywords—meningitis; clinical prediction rule; decision
the reference diagnostic standard. CPR quality was assessed trees; predictive value of tests; multivariate analysis
using 17 previously published items. CPR performance was
evaluated using sensitivity, negative likelihood ratio, and the
treatment frequency that would result if the rule was used. INTRODUCTION
Results: Eleven studies involving 6675 children with acute Background
meningitis fulfilled all inclusion criteria and were entered
in the study. They all describe the derivation or validation of
Acute meningitis is a relatively common and potentially
six unique CPRs. A rigorously developed, high-performing,
severe infection in childhood, with approximately 3000
and well-validated CPR ready for clinical use to guide which
children with suspected bacterial meningitis should be hos- new cases each year in children younger than 18 years
pitalized and treated with intravenous antibiotics and which of age in the United States (1–3). Fortunately, most
can be safely discharged home was not identified. Areas for infections are aseptic and resolve spontaneously, but
quality improvement for future CPR studies include pro- 6–18% are bacterial in origin and require i.v. antibiotic
spective validation using standardized inclusion criteria, treatment, hospitalization, and close monitoring (4,5).

RECEIVED: 30 March 2012; FINAL SUBMISSION RECEIVED: 3 October 2012;

ACCEPTED: 15 March 2013

508
Does This Child Have Bacterial Meningitis? 509

Accurately diagnosing bacterial meningitis in children strategy for CPRs was used with the addition of the
is imperative to avoid severe outcomes, such as neurolog- MeSH term bacterial meningitis (35,36). The reference
ical impairment or death, and minimize unnecessary lists of identified clinical prediction rule publications
antibiotic use and hospitalization in children with self- were searched manually to identify additional studies.
limiting illness (6–13). Although diagnostic techniques, Our search was limited to English publications.
such as cerebral spinal fluid (CSF) culture and poly-
merase chain reaction (PCR) are reliable, physicians Inclusion Criteria
must decide which patients to treat with i.v. antibiotics
empirically before culture or PCR results are reported Only prospective or retrospective studies that derived,
48 h later (14,15). Weighing the risk of missing a validated, or assessed the impact of CPRs were included.
true case of bacterial meningitis with unnecessarily A CPR was defined as a clinical decision-making tool that
admitting patients without bacterial meningitis for 48 h 1) includes three or more predictive variables obtained
of i.v. antibiotics remains a major challenge (16). from the history, physical examination, or simple diag-
Clinicians often review results of CSF glucose, pro- nostic tests; 2) provides the probability of an outcome
tein, white blood cell (WBC) count, Gram stain, and or suggests a diagnostic or therapeutic course of action
serum WBC count to guide diagnostic decisions (17). for an individual patient; and 3) is not a decision analysis
Gram stain is highly specific but it is falsely negative in or practice guideline (32,33,35–37)
up to 40% of cases and therefore cannot be exclusively re- Only studies involving children (term birth–18 years)
lied upon (2,18–20). Elevated CSF WBC count is more with suspected or proven bacterial meningitis were in-
common in aseptic rather than bacterial meningitis cluded. Studies that did not include positive CSF culture
(21–27). Widespread use of vaccines against Strepto- (defined as a positive bacterial pathogen isolated in the
coccus pneumonia and Hemophilus influenza type B CSF) as the outcome were not included. Studies involv-
have decreased the probability of bacterial meningitis in ing both adults and children were included if a separate
children, further decreasing the likelihood that elevated analysis was performed for children. Studies requiring
CSF WBC count is the result of bacterial meningitis the use of artificial neural networks or that assessed pre-
(28–31). dictors with no obvious goal of creating a prediction rule
Clinical prediction rules (CPRs) are potentially power- were not included. Studies that did not include positive
ful evidence-based tools for reducing uncertainty and CSF culture (defined as a positive bacterial pathogen
improving accuracy in medical decision making by stan- isolated in the CSF) as the outcome were not included.
dardizing the collection and interpretation of clinical data We did not exclude studies that included patients who
(32). They can also minimize the use of potentially harm- received antibiotics; however, no such studies met inclu-
ful diagnostic tests, such as lumbar puncture, and reduce sion criteria.
admissions and adverse events from antibiotic use. They
have been defined as clinical decision-making tools that Selection of Studies
quantify the relative importance of 3 or more variables
from history, physical examination, or simple tests to pro- Two reviewers (D. M. Kulik and J. L. Maguire) inde-
vide the probability of an outcome or suggest a single pendently assessed the inclusion of potentially relevant
diagnostic or therapeutic course of action for an individ- articles through a two-step process. First, the titles and
ual patient (32–34). abstracts from each article identified through the elec-
This study aimed to systematically identify CPRs for tronic search were assessed for inclusion. Second, pub-
children with suspected bacterial meningitis and compare lications identified as relevant by title or abstract were
their methodological quality and performance for diag- reviewed manually. Discrepancies between the two
nosing bacterial meningitis using a recently developed reviewers were discussed and included by consensus.
framework to evaluate CPRs for children (35). Discrepancies occurred <5% of the time and consensus
was reached for every discrepancy.
METHODS
Assessment of Methodological Quality
Search Strategy
The quality of the included studies was assessed using
Potentially relevant studies were identified through elec- a 17-item checklist from published guidelines for use in
tronic searches of Medline and Embase from January the derivation or validation of clinical prediction rules for
1950 up to September 2012 (Appendix 1). Because there children (see Appendix 2) (32–36,38,39). Each item was
is no medical subject heading (MeSH) that specifies clinical noted to be present (1) or absent (0). The maximum number
prediction rules, a previously developed electronic search of quality items was 17. Two reviewers (D. M. Kulik and
510 D. M. Kulik et al.

J. L. Maguire) independently abstracted data from in- RESULTS

cluded papers using a standardized data collection form.
Study Selection
Discrepancies between the reviewers were discussed and
resolved by consensus. Given the importance of rule vali-
A total of 6387 titles and abstracts were identified and
dation, hierarchy of rule validation was assessed separately
screened by the search strategy as potentially relevant
(see Assessment of Hierarchy of Rule Validation section).
(Figure 1). Of these, 379 were extracted as full text
articles and assessed for inclusion; 11 studies fulfilled
Assessment of Rule Performance
all inclusion criteria. These studies were derivation or
validation studies of six unique CPRs.
Data were extracted from each publication to construct
a 2  2 table to calculate sensitivity (Sn), specificity, neg-
Description of Studies
ative likelihood ratio (LR) and their 95% confidence in-
tervals (CIs) for the diagnosis of bacterial meningitis
Included studies were published between 2001 and 2010
(33,40). Calculations were performed using SAS 9.1
and involved a total of 6675 children (see Table 1 for
(SAS Institute Inc., Cary, NC). The predicted frequency
population characteristics for each study). The median
of empiric i.v. antibiotic treatment and hospitalization
number of children enrolled in each study was 286 (range
that would result if the rule were applied to every
74–2903). Five studies were published in general pediat-
patient (rule predicted treatment frequency) was also
ric journals, two in infectious disease journals, two in
determined. Rule predicted treatment frequency
general medicine journals, and one in each of emergency
captures both sensitivity and specificity and provides
medicine and epidemiology journals. Three studies were
a sensible quantification of the degree of overtreatment
multicentered and eight were single centered. Five
that the CPR would produce when compared with the
cohorts were from the Unite States, two were from
actual frequency of bacterial meningitis in each study.
France, three were from the Netherlands, and one was
Given that physicians are not likely to feel comfortable
multicentered across Europe (France, Poland, Turkey,
using a rule that misses more than a very small number
Spain, and Switzerland). Eight of 11 studies were per-
of children with bacterial meningitis and that such
formed in the emergency department (ED). Three studies
a rule should not substantially inflate the risk of i.v.
were completed on the pediatric ward. There was consid-
antibiotics and hospital admission, the following 4 rule-
erable variation in the population age (28 days–18 years).
performance benchmarks were used to identify high-
Three studies included patients presenting with the clini-
performing CPRs: Sn > 0.95; lower limit of sensitivity
cal suspicion for meningitis in the ED, and eight studies
95% CI > 0.95; LR < 0.1; and upper limit of the LR
included only children who had a final diagnosis of viral
95% CI < 0.1 (32,33,35–37,40,41). Sensitivity and
or bacterial meningitis. Two studies used post-discharge
LR are also independent of disease prevalence, which
follow-up methods to ensure no missed cases of bacterial
makes them useful measures for comparing CPRs from
meningitis (43,44). The frequency of acute bacterial
different populations with different frequencies of
meningitis in each study population varied between
bacterial meningitis (40,42).
0.04 and 0.48 (median 0.19).
Six studies described derivations of a rule and 5 were
Assessment of Hierarchy of Rule Validation
rule validation studies. None assessed the rule’s clinical
The degree of validation for CPRs that met inclusion cri-
teria was assessed according to the hierarchy of evidence
for CPRs published by the Evidence-Based Medicine
Working Group (32). In this hierarchy, prediction rules
that have been derived but not prospectively validated
are the lowest level of evidence (level 4), rules that
have been prospectively validated in only one sample
are level 3, rules that have been broadly validated in mul-
tiple settings are level 2, and rules that have had impact
analysis performed that demonstrated a change in clini-
cian behavior with beneficial consequences are level 1.
Although performing a meta-analysis would have
been ideal, due to heterogeneity of the populations and
predictors used in many of the rules, we believed that it Figure 1. Selection process for meningitis clinical prediction
would not be appropriate to do. rules.
 Does This Child Have Bacterial Meningitis?
Table 1. Study Characteristics

No. of
Study Setting Rule Name n Study Type Age Population Outcomes Assessment Country Sites

Bonsu, 2008 (47) ED Modified Bonsu 1107 Derivation 4 wk18 y Children diagnosed with acute Bacteria grown from CSF US 1
viral or bacterial meningitis or
children who underwent LP*
Bonsu 2004 (50) ED Bonsu 142 Derivation 5 wk18y Children diagnosed with acute Bacteria grown from CSF US 1
viral or bacterial meningitis or
children who underwent LP*
Chavanet 2007 (51) Ward Meningitest 175 Derivation 3 mos15 y Children hospitalized with LP* and Bacteria grown from CSF France 1
7 WBC/mL in CSF
Dubos 2006* (45) ED Bonsu 161 Validation 28 d16 y Children with acute meningitis Bacteria grown from CSF France 1
(CSF WBC $7/mm3)
Dubos 2006* (45) ED BMS 151 Validation 28 d16 y Children with acute meningitis Bacteria grown from CSF France 1
(CSF WBC $7/mm3)
Dubos 2006* (45) ED Oostenbrink 119 Validation 28 d16 y Children with acute meningitis Bacteria grown from CSF France 1
(CSF WBC $7/mm3)
Dubos 2010* (46) ED Meningitest 198 Validation 29 d18 y CSF WBC $7  106/L and CSF Bacteria grown from CSF 5 European 6
culture/PCR or blood culture countries
positive and procalcitonin level
done
Dubos 2010* (46) ED BMS 198 Validation 29 d18 y CSF WBC $7  106/L and CSF Bacteria grown from CSF 5 European 6
culture/PCR or blood culture countries
positive and procalcitonin level
done
Fine 2007 (27) Ward Modified BMS 696 Derivation 29 d19 y Children with a final diagnosis of Bacteria grown from CSF US 1
meningitis who had LP or CSF meningitis CSF
performed WBC $7/mm with
positive blood culture
Nigrovic 2002 (2) Ward BMS 696 Derivation 29 d19 y Children with a final diagnosis of Bacteria grown from CSF US 1
meningitis who had LP
performed
Nigrovic 2007 (48) ED BMS 2903 Validation 29 d19 y Children with a final diagnosis of Bacteria grown from CSF US 20
meningitis who had LP
performed
Oostenbrink 2001 (43) ED Oostenbrink 286 Derivation 1 mos15 y Children visiting the ED with Bacteria grown from CSF Netherlands 1
meningeal signs
Oostenbrink 2002 (44) ED Oostenbrink 74 Validation 2 mos15 y Children visiting the ED with Bacteria grown from CSF Netherlands 1
meningeal signs
Oostenbrink 2004 (49) ED Oostenbrink 226 Validation 2 mos15 y Children visiting the ED with Bacteria grown from CSF Netherlands 4
meningeal signs

BMS = Bacterial Meningitis Score; CSF = cerebrospinal fluid; ED = emergency department; LP = lumbar puncture; PCR = polymerase chain reaction; WBC = white blood cell count; US =
United States.
* Assessed the performance of more than one rule.

511
512 D. M. Kulik et al.

impact. Five were derivation or validation studies of the to all patients at risk (3 of 11). The highest-quality studies
rule created or modified by Bonsu et al. (referred to as were the Modified Bonsu derivation study (11 of 17) and
‘‘Bonsu’’ or ‘‘modified Bonsu score,’’ respectively); five the BMS validation study by Nigrovic et al. (12 of 17)
derived or validated the Bacterial Meningitis Score (47,48).
(BMS) or modified BMS; four derived or validated the
score developed by Oostenbrink et al. (referred to as Assessment of Rule Performance
‘‘Oostenbrink score’’); and 2 derived or validated the
Meningitest rule. Two studies validated multiple rules Rule performance varied considerably (Table 4). The
simultaneously (45,46). median rule sensitivity was 1 with a range of 0.83 to 1
Predictors included in the rules varied considerably, (median lower limit of the 95% CI for sensitivity was
although there were some predictors that were common 0.91, range 0.51–0.95) (Table 4). Median LR was 0,
to most rules (Table 2). Elevated CSF protein was com- with a range of 0 to 0.23 (median upper limit of the 95%
mon to five rules (BMS, modified BMS, Bonsu, modified CI was 0.16, range 0.09–0.83). Median rule-predicted
Bonsu, and Meningitest), elevated CSF neutrophil count treatment frequency was 0.48 (range 0.24–0.81).
was common to four rules (BMS, modified BMS, Bonsu, All of the studies except for the Oostenbrink validation
and modified Bonsu), history of seizure and positive CSF studies demonstrated sensitivity >0.95 and LR < 0.1
Gram stain were common to three rules (BMS, modified (2,27,43,45,47,49–51). The Oostenbrink derivation
BMS, and Meningitest). study and BMS and Meningitest validation studies had
the lower limit of the 95% CI for sensitivity > 0.95
Assessment of Methodological Quality (43,46). Only the modified BMS derivation study by
Fine had the upper limit of the 95% CI for LR < 0.1
Studies met between 5 and 12 of the 17 quality items (see (27). No derivation or validation studies had 95% confi-
Table 3). All 11 studies resulted in clinically sensible dence in both Sn > 0.95 and LR < 0.1. No studies eval-
CPRs that described a course of action to be taken and uated the performance of a score compared with clinical
provided an adequate description of the mathematical judgment alone. The lowest rule-predicted treatment fre-
technique used. The outcome was well described in 10 quencies were 0.33 and 0.41 for the modified BMS study
studies and the population was well described in 9 stud- and BMS study, respectively (27,48).
ies. The most poorly addressed CPR quality items were
predictor assessor blinding (0 of 11), reproducibility of Assessment of Hierarchy of Rule Validation
predictor variable assessment (0 of 11), prospective
design (1 of 11), adequate follow-up (2 of 11), adequate Of the six unique CPRs, five were level 4 (two were
description of the study site (3 of 11), and rule application derived with no validation and three were derived with

Table 2. Predictors Used in Each Study

Predictor BMS Modified BMS Oostenbrink Bonsu Modified Bonsu Meningitest

Seizure O O O
CSF-positive Gram stain O O O
Elevated CSF protein* O O O O O
CSF neutrophil count $1000 106/L O O O O
Blood neutrophil count $109/L O O
Purpura O
Toxic appearance O
Procalcitonin level $0.5 ng/mL O
Duration of symptoms O
Vomiting O
Cyanosis O
Altered LOC O
Meningeal irritation O
Petechia O
Elevated CRP O
Young age O O
Season
Temperature >38.4 C
WBC >6100/uL
Local disease incidence O

CRP = C-reactive protein; LOC = level of consciousness; WBC = white blood cell count.
* Cerebrospinal fluid (CSF) protein $80 mg/dL (Bacterial Meningitis Score [BMS]), $50 mg/dL (Meningitest).
Does This Child Have Bacterial Meningitis? 513

Power of Results Reported Sensible Use Reported Total

retrospective validation), none was level 3 (prospective

11

10

12
9

8
validation in one setting), and one was level 2 (prospec-
Follow Patients Adequate Reporting Technique Clinically to CI on Sn

tive validation in multiple settings) (2,27,43–47,49–51).

Yes

Yes

Yes

Yes

Yes
yes
No

No

No

No

No

6
No CPR was level 1 (impact analysis performed
demonstrating a change in clinician behavior with
Easy

Yes

Yes

Yes

Yes

Yes
No

No

No

No

No

No

5
beneficial consequences). Of the level 4 CPRs, the
highest quality study was the BMS retrospective
Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

11
validation study, which met 12 of 17 quality items and
also had strong performance (Sn 0.98, 95% CI 0.94–1;
Adequate matical
Mathe-

LR 0.03, 95% CI 0–0.11, rule-predicted treatment fre-

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

11
quency 0.41) (48). The single-level 2 rule met only 8 of
17 quality items and had fair performance (Sn 0.92,
Yes

Yes

Yes

Yes

Yes

95% CI 0.72–1, LR 0.13, 95% CI 0–0.5, rule-

No

No

No

No

No

No

5
predicted treatment frequency 0.45).
Yes

Yes

Yes

Yes

Yes
No

No

No

No

No

No

5 DISCUSSION
Applied

at Risk

This review has identified 11 studies involving the deriva-

Adequate to All

Yes

Yes

Yes
No

No

No

No

No

No

No

No

tion or validation of six different bacterial meningitis

clinical prediction rules for children. Using a recently de-
Yes

Yes
Up

No

No

No

No

No

No

No

No

No

veloped framework for evaluating the quality and perfor-

2

mance of clinical prediction rules for children, we

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured
ment of Reproducible

identified the presence or absence of 17 items generally

Predictors

considered to be of importance for high-quality clinical

prediction rules. We also evaluated the diagnostic perfor-
0

mance of these rules based on their sensitivity, LR, and

pective Described Described Defined Predictors Action Defined Outcome
Population Predictors Assess- Course Outcome Assess-

the treatment frequency that would result if the rule were

Blind

Yes

Yes

Yes

Yes

Yes
No

No

No

No

No

No

to be used (2,27,43–46,48–51).
A number of issues made it challenging to compare
Well

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

existing CPRs for children with acute meningitis. First,

No

10

three of the studies included only children who had a dis-

charge diagnosis of meningitis. This situation results in
BMS = Bacterial Meningitis Score; CI = confidence interval; Sn = sensitivity.
Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes
of

11

a circular argument where the derived CPR functions

on a population that will never exist to the clinician trying
ment of
Blind

No

No

No

No

No

No

No

No

No

No

No

to decide ‘‘does this child with acute meningeal signs

0

have bacterial meningitis?’’ We recommend that future

studies include both children who are admitted to hospital
Well

Yes

Yes

Yes

Yes

Yes

Yes
No

No

No

No

No

and those discharged to the community, with follow-up of

6

outcomes several weeks later.

Table 3. Assessment of Methodological Quality

Second, there appears to be considerable variation in the

Well

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes
No

No

frequency of meningitis between the studies (0.04–0.48,

9

median 0.185). It is unclear what accounts for this variabil-

ity, although clearly including only admitted patients, and
Well
Site

Yes

Yes

Yes
No

No

No

No

No

No

No

No

in some cases only including patients with a discharge

diagnosis of meningitis, accounts for the high prevalence
Pros-

Yes

in some of the studies. Children’s age, season of the year,

No

No

No

No

No

No

No

No

No

No

and geographical region might be factors, but comparison

Oostenbrink

Modified BMS

of existing studies shows no clear pattern. This variability

Bonsu, BMS,
Rule Name

Meningitest,

Oostenbrink

Oostenbrink

Oostenbrink
Meningitest
Bonsu

is concerning because CPRs are well-known to perform

Modified

BMS
Bonsu

better at excluding disease when the outcome frequency

BMS

BMS

is lower. We recommend that future CPR studies use con-

2008 (47)

2004 (50)

2007 (51)

2006 (45)

2010 (46)

2007 (27)

2007 (48)

2001 (43)

2002 (44)

2004 (49)
Oostenbrink

Oostenbrink

Oostenbrink
2002 (2)

sistent inclusion criteria, such as including all patients

Chavanet
Study

Nigrovic
Nigrovi
Dubos

Dubos

0–18 years of age presenting to the ED with meningeal

Bonsu

Bonsu

Total
Fine

signs, and that sampling take place throughout the year.

514 D. M. Kulik et al.

Treatment Frequency
The methodological quality of included studies was
Rule Predicted variable. As the consequences of a missed case of acute
bacterial meningitis are high, the methodological quality
0.59

0.24

0.43

0.81
0.73

0.41
0.65
0.66
0.45
0.63

0.34

0.33
0.42
0.5
in the development of a CPR that is to be used in practice
must be high. Although a number of areas of strength
were noted, several methodological weaknesses were
common in the reviewed CPR studies. Most strikingly,
all but one of the studies was retrospective. There is
Bacterial Meningitis

good evidence that retrospectively derived or validated

Frequency

CPRs perform considerably poorer when subsequently

0.24

0.11
0.12
0.13

0.48
0.48

0.04
0.29
0.44
0.42

0.18
0.19
0.1

0.1
prospectively validated (37). The single-level 2 CPR pro-
vides a nice example of this where both derivation and
retrospective validation studies suggested excellent per-
formance (Sn 1 and LR 0) but prospective validation re-
Upper 95% CI for LR

sulted in much lower performance (Sn 0.92, LR 0.13)

(43,44,49).
Other areas of methodological weakness included in-
0.11
0.18

0.83

0.09
0.14

0.5

adequate description of the study site, lack of predictor

*

*
*
*

*
*

*
*

blinding, absence of reproducibility testing of predictor

variables, inadequate follow-up of outcomes, and insuffi-
cient statistical power. There is considerable evidence
that CPRs that fail to meet these methodological stan-
Lower 95% CI on

dards are likely to be biased. Without a clear description

Sensitivity

of the study site, it is unclear to physicians whether the

0.51

0.95

0.94
0.94
0.89
0.79
0.79

0.95

0.93
0.91

0.95
0.75
0.72
0.8

study site is similar enough to their own for the results

to be applicable (35). Deficiencies in predictor blinding
have been shown to result in an overestimation of diag-
nostic performance (52). Assessment of the inter-
0.023
LR

0.23

0.03

0.13
0.02
0.04

observer reliability of predictor variables is necessary to

0
0

0
0

determine whether a rule will perform similarly when

BMS = Bacterial Meningitis Score; CI = confidence interval; LR = likelihood ratio.
Specificity

used by other physicians (33,37). Insufficient power for

0.62

0.57
0.66
0.72

0.52

0.62
0.49

0.61
0.43

0.85

0.36

0.81
0.70

0.42

statistical modeling due to an inadequate number of

outcomes for a given number of predictor variables
increases the possibility of spurious results through
Sensitivity

random effects (42,53). Inadequate follow-up of out-

0.98
0.98

0.83

0.92
0.98
0.98

comes increases the apparent rule performance by poten-

1
1

1
1

1
1

tially missing serious outcomes (37).

For clinicians to feel comfortable using a CPR for di-
Study Type

Derivation
Derivation

Derivation

Derivation
Derivation

Derivation
Validation
Validation
Validation

Validation

Validation
Validation

Validation
Validation

agnosing bacterial meningitis, the performance of such

a rule must necessarily be high. The performance of the
* Could not calculate LR 95% CI as LR = 0.

only level 2 CPR is unlikely to be sufficiently strong to

Table 4. Assessment of Rule Performance

make it a candidate for routine clinical use with both sen-

Modified Bonsu

Modified BMS

sitivity and LR- below existing performance benchmarks.

Rule Name

Oostenbrink

Oostenbrink
Oostenbrink
Oostenbrink
Meningitest

Meningitest

While several level 4 CPRs show promise, the level 4

CPR with the highest quality and performance was the
Bonsu

Bonsu
BMS

BMS

BMS
BMS

BMS, making it the strongest candidate to perform

consistently well in subsequent prospective validation
studies.
Oostenbrink 2001 (43)
Oostenbrink 2002 (44)
Oostenbrink 2004 (49)
Chavanet 2007 (51)

Nigrovic 2007 (48)

Nigrovic 2002 (2)
Dubos 2006 (45)
Dubos 2006 (45)
Dubos 2006 (45)
Dubos 2010 (46)
Dubos 2010 (46)
Bonsu 2008 (47)
Bonsu 2004 (50)

Limitations
Fine 2007 (27)
Study

There are several limitations to this systematic review.

The electronic search strategy may not have identified
all clinical prediction rules for bacterial meningitis in
Does This Child Have Bacterial Meningitis? 515

children. However, the search strategy was performed on 3. Tatara R, Imai H. Serum C-reactive protein in the differential diag-
nosis of childhood meningitis. Pediatr Int 2000;42:541–6.
multiple occasions to ensure all relevant papers were 4. Saez-Llorens X, McCracken GH Jr. Bacterial meningitis in
identified and the reference lists of all clinical prediction children. Lancet 2003;361(9375):2139–48.
rule publications were manually reviewed, which failed 5. El Bashir H, Laundy M, Booy R. Diagnosis and treatment of bacte-
rial meningitis. Arch Dis Child 2003;88:615–20.
to reveal any additional studies. The methodological
6. Lu CH, Huang CR, Chang WN, et al. Community-acquired bacte-
quality metric that we used treated each quality item rial meningitis in adults: the epidemiology, timing of appropriate
with equal weight and it is possible that some items are antimicrobial therapy, and prognostic factors. Clin Neurol Neuro-
surg 2002;104:352–8.
more important than others. Some of the methodological 7. Meyer CN, Samuelsson IS, Galle M, Bangsborg JM. Adult bacterial
quality items were subjective, however, two reviewers meningitis: aetiology, penicillin susceptibility, risk factors, prog-
reached consensus on every item for each study. Further, nostic factors and guidelines for empirical antibiotic treatment.
Clin Microbiol Infect 2004 Aug;10(8):709–17.
the methodological quality items that we used have been 8. Tunkel AR, Hartman BJ, Kaplan SL, et al. Practice guidelines for
well described in the literature but have not been rigor- the management of bacterial meningitis. Clin Infect Dis 2004;39:
ously developed or validated (32–34,37,38). However, 1267–84.
9. van de Beek D, de Gans J, Spanjaard L, Vermeulen M, Dankert J.
they have been previously used to evaluate clinical Antibiotic guidelines and antibiotic use in adult bacterial meningitis
prediction rules for children and are widely accepted in The Netherlands. J Antimicrob Chemother 2002;49:661–6.
for evaluating the quality of studies of diagnostic tests 10. Moller K, Skinhoj P. Guidelines for managing acute bacterial men-
ingitis in adults. West J Med 2000;173:223–4.
(35,36,54). Lastly, although the rule-predicted treatment 11. Feigin RD. Use of corticosteroids in bacterial meningitis. Pediatr
frequency is illustrative of the degree of overtreatment Infect Dis J 2004;23:355–7.
that would result from the use of each CPR, it remains dif- 12. Proulx N, Frechette D, Toye B, Chan J, Kravcik S. Delays in the ad-
ministration of antibiotics are associated with mortality from adult
ficult to determine how this compares with clinical judg- acute bacterial meningitis. QJM 2005;98:291–8.
ment alone, as this was not assessed in any of the studies. 13. Parasuraman TV, Frenia K, Romero J. Enteroviral meningitis. Cost
of illness and considerations for the economic evaluation of poten-
tial therapies. Pharmacoeconomics 2001;19:3–12.
CONCLUSIONS 14. Chadwick DR, Lever AM. The impact of new diagnostic methodol-
ogies in the management of meningitis in adults at a teaching hos-
In summary, a rigorously developed, high-performing pital. QJM 2002;95:663–70.
15. Poppert S, Essig A, Stoehr B, et al. Rapid diagnosis of bacterial
and well-validated CPR that is ready for clinical use to meningitis by real-time PCR and fluorescence in situ hybridization.
guide which children with suspected bacterial meningitis J Clin Microbiol 2005;43:3390–7.
should be hospitalized and treated with i.v. antibiotics and 16. Jansen GJ, Mooibroek M, Idema J, Harmsen HJ, Welling GW,
Degener JE. Rapid identification of bacteria in blood cultures by us-
can be safely discharged home was not identified. A de- ing fluorescently labeled oligonucleotide probes. J Clin Microbiol
tailed analysis of the methodological quality and perfor- 2000;38:814–7.
mance of identified CPRs suggested areas for quality 17. Spanos A, Harrell FE Jr, Durack DT. Differential diagnosis of acute
meningitis. An analysis of the predictive value of initial observa-
improvement for future CPR studies. These include pro- tions. JAMA 1989;262:2700–7.
spective validation using standardized inclusion criteria 18. Geiseler PJ. Unnecessary duplication of publication. N Engl J Med
to capture all children at risk of bacterial meningitis, 1980;22;302:1209–10.
19. Wald ER, Kaplan SL, Mason EO Jr, et al. Dexamethasone therapy
adequate blinding, predictor reproducibility assessment, for children with bacterial meningitis. Meningitis Study Group.
and meticulous follow-up of clinical outcomes after dis- Pediatrics 1995;95:21–8.
charge. Although several of the rules that have not yet 20. Marton KI, Gean AD. The spinal tap: a new look at an old test. Ann
Intern Med 1986;104:840–8.
undergone prospective validation show promise, the level 21. Sato Y, Ohta Y, Honda Y, Kaji M, Oizumi K. Diagnostic value of
4 CPR with the highest quality and performance is the atypical lymphocytes in cerebrospinal fluid from adults with entero-
BMS, making it the best candidate for prospective valida- viral meningitis. J Neurol 1998;245:598–602.
22. Baker RC, Lenane AM. The predictive value of cerebrospinal
tion. Until consistently high methodological quality and fluid differential cytology in meningitis. Pediatr Infect Dis J 1989;
diagnostic performance are demonstrated through pro- 8:329–30.
spective validation, caution is warranted in the routine 23. Negrini B, Kelleher KJ, Wald ER. Cerebrospinal fluid findings in
aseptic versus bacterial meningitis. Pediatrics 2000;105:316–9.
clinical use of existing CPRs for children with suspected 24. Berkley JA, Mwangi I, Ngetsa CJ, et al. Diagnosis of acute bacterial
bacterial meningitis. meningitis in children at a district hospital in sub-Saharan Africa.
Lancet 2001;357(9270):1753–7.
25. Freedman SB, Marrocco A, Pirie J, Dick PT. Predictors of bacterial
REFERENCES meningitis in the era after Haemophilus influenzae. Arch Pediatr
Adolesc Med 2001;155:1301–6.
1. Schuchat A, Robinson K, Wenger JD, et al. Bacterial meningitis in 26. Dunbar SA, Eason RA, Musher DM, Clarridge JE 3rd. Microscopic
the United States in 1995. Active Surveillance Team. N Engl J Med examination and broth culture of cerebrospinal fluid in diagnosis of
1997;337:970–6. meningitis. J Clin Microbiol 1998;36:1617–20.
2. Nigrovic LE, Kuppermann N, Malley R. Development and valida- 27. Fine AM, Nigrovic LE, Reis BY, Cook EF, Mandl KD. Linking
tion of a multivariable predictive model to distinguish bacterial surveillance to action: incorporation of real-time regional data
from aseptic meningitis in children in the post-Haemophilus influ- into a medical decision rule. J Am Med Inform Assoc 2007;14:
enzae era. Pediatrics 2002;110:712–9. 206–11.
516 D. M. Kulik et al.

28. Black S, Shinefield H, Fireman B, et al. Efficacy, safety and immu- 41. Stiell IG, Wells GA, Vandemheen K, et al. The Canadian CT Head
nogenicity of heptavalent pneumococcal conjugate vaccine in chil- Rule for patients with minor head injury. Lancet 2001;357(9266):
dren. Northern California Kaiser Permanente Vaccine Study Center 1391–6.
Group. Pediatr Infect Dis J 2000;19:187–95. 42. Concato J, Feinstein AR, Holford TR. The risk of determining risk
29. Kaplan SL, Mason EO Jr, Wald ER, et al. Decrease of invasive pneu- with multivariable models. Ann Intern Med 1993;118:201–10.
mococcal infections in children among 8 children’s hospitals in the 43. Oostenbrink R, Moons KG, Donders AR, Grobbee DE, Moll HA.
United States after the introduction of the 7-valent pneumococcal Prediction of bacterial meningitis in children with meningeal signs:
conjugate vaccine. Pediatrics 2004;113(3 Pt 1):443–9. reduction of lumbar punctures. Acta Paediatr 2001;90:611–7.
30. Whitney CG, Farley MM, Hadler J, et al. Decline in invasive pneu- 44. Oostenbrink R, Moons KG, Twijnstra MJ, Grobbee DE, Moll HA.
mococcal disease after the introduction of protein-polysaccharide Children with meningeal signs: predicting who needs empiric anti-
conjugate vaccine. N Engl J Med 2003;1(348):1737–46. biotic treatment. Arch Pediatr Adolesc Med 2002;156:1189–94.
31. Kyaw MH, Lynfield R, Schaffner W, et al. Effect of introduction of 45. Dubos F, Lamotte B, Bibi-Triki F, et al. Clinical decision rules to
the pneumococcal conjugate vaccine on drug-resistant Streptococ- distinguish between bacterial and aseptic meningitis. Arch Dis
cus pneumoniae. N Engl J Med 2006;354:1455–63. Child 2006;91:647–50.
32. McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, 46. Dubos F, Korczowski B, Aygun DA, et al. Distinguishing between
Richardson WS. Users’ guides to the medical literature: XXII: bacterial and aseptic meningitis in children: European comparison
how to use articles about clinical decision rules. Evidence-Based of two clinical decision rules. Arch Dis Child 2010;95:963–7.
Medicine Working Group. JAMA 2000;284:79–84. 47. Bonsu BK, Ortega HW, Marcon MJ, Harper MB. A decision rule for
33. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review predicting bacterial meningitis in children with cerebrospinal fluid
and suggested modifications of methodological standards. JAMA pleocytosis when gram stain is negative or unavailable. Acad Emerg
1997;277:488–94. Med 2008;15:437–44.
34. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction 48. Nigrovic LE, Kuppermann N, Macias CG, et al. Clinical prediction
rules. Applications and methodological standards. N Engl J Med rule for identifying children with cerebrospinal fluid pleocytosis at
1985;313:793–9. very low risk of bacterial meningitis. JAMA 2007;297:52–60.
35. Maguire JL, Kulik DM, Laupacis A, Kuppermann N, Uleryk EM, 49. Oostenbrink R, Moons KG, Derksen-Lubsen AG, Grobbee DE,
Parkin PC. Clinical prediction rules for children: a systematic Moll HA. A diagnostic decision rule for management of children
review. Pediatrics 2011;128:e666–77. with meningeal signs. Eur J Epidemiol 2004;19:109–16.
36. Maguire JL, Boutis K, Uleryk EM, Laupacis A, Parkin PC. Should 50. Bonsu BK, Harper MB. Differentiating acute bacterial meningitis
a head-injured child receive a head CT scan—a systematic review of from acute viral meningitis among children with cerebrospinal fluid
clinical prediction rules. Pediatrics 2009;124:e145–54. pleocytosis: a multivariable regression model. Pediatr Infect Dis J
37. Stiell IG, Wells GA. Methodologic standards for the development of 2004;23:511–7.
clinical decision rules in emergency medicine. Ann Emerg Med 51. Chavanet P, Schaller C, Levy C, et al. Performance of a predictive
1999;33:437–47. rule to distinguish bacterial and viral meningitis. J Infect 2007;54:
38. Reilly BM, Evans AT. Translating clinical research into clinical 328–36.
practice: impact of using prediction rules to make decisions. Ann 52. Bossuyt PM, Reitsma JB, Bruns DE, et al. Toward complete and
Intern Med 2006;144:201–9. accurate reporting of studies of diagnostic accuracy: the STARD
39. Mauer DK, Nolan J, Plaisance P, et al. Effect of active compression- initiative. Acad Radiol 2003;10:664–9.
decompression resuscitation (ACD-CPR) on survival: a combined 53. Moss M, Wellman DA, Cotsonis GA. An appraisal of multivariable
analysis using individual patient data. Resuscitation 1999;41:249–56. logistic models in the pulmonary and critical care literature. Chest
40. Hess EP, Thiruganasambandamoorthy V, Wells GA, et al. Diagnos- 2003;123:923–8.
tic accuracy of clinical prediction rules to exclude acute coronary 54. Pickering A, Harnan S, Fitzgerald P, Pandor A, Goodacre S. Clinical
syndrome in the emergency department setting: a systematic review. decision rules for children with minor head injury: a systematic
CJEM 2008;10:373–82. review. Arch Dis Child 2011;96:414–21.
Does This Child Have Bacterial Meningitis? 517

APPENDIX 1: SEARCH STRATEGY

Topic The following tables and text record the search strate-
gies and terms used.
Search strategies. We ran searches using the OVID search
MEDLINE. The search strategy for MEDLINE was com-
platform in the following databases: MEDLINE, and
pleted from 1948 to September 27, 2012. We used a com-
EMBASE. The initial search was run on August 18,
bination of MeSH and free text terms for:
2011 and AutoAlerts were received and reviewed on an
ongoing basis until article submission on October 3, 2012.

Set History Comments

1 ((derivation or clinical* or predict* or decision*) adj10 (rule* or algorithm* or tool or tools or Prediction Rule Terms
model* or score* or indicator* or validation* or criteria)).ti,ab. or decision support
techniques/
2 limit 1 to (‘‘infant (1 to 23 months)’’ or ‘‘preschool child (2 to 5 years)’’ or ‘‘child (6 to 12 Age Group Limit
years)’’ or ‘‘adolescent (13 to 18 years)’’)
3 Pediatrics/ Pediatric Specialty term
4 2 or (1 and 3) Age Group limits
5 evaluation studies.pt. or evaluation studies as topic/ or validation studies.pt. or validation Diagnostic sensitivity filter terms
studies as topic/ or ‘‘sensitivity and specificity’’/ or predictive value of tests/ or roc
curve/ or diagnostic errors/ or false negative reactions/ or false positive reactions/ or
observer variation/ or likelihood functions/ or (likelihood or likelihood ratio:).mp. or
predictive value of tests/
6 4 and 5 Diagnostic term limits
7 exp Meningitis, Bacterial/ or Meningitis/ or (bacterial adj2 Meningitis).mp. Bacterial Meningitis terms
8 6 and 7 Final results

EMBASE. The search strategy for EMBASE (1980 to 2012 week 39). We used a combination of EMBASE and free text
terms.

Set History Comments

1 (((derivation or clinical* or predict* or decision*) adj10 (rule* or algorithm* or tool or tools or Prediction Rule Terms
model* or score* or indicator* or validation* or criteria)) or (decision adj5 support adj5
techni*)).ti,ab. or decision support system/ or scoring system/
2 limit 1 to (infant <to one year> or child <unspecified age> or preschool child <1 to 6 years> Age Group Limit
or school child <7 to 12 years> or adolescent <13 to 17 years>)
3 pediatrics/ Pediatric Specialty term
4 2 or (1 and 3) Age Group limits
5 ‘‘sensitivity and specificity’’/ or diagnostic accuracy/ or ‘‘prediction and forecasting’’/ or Diagnostic sensitivity filter terms
prediction/ or predictive validity/ or observer variation/ or receiver operating
characteristic/ or roc curve/ or laboratory diagnosis/ or abnormal laboratory result/ or
likelihood functions/ or validation study/ or reproducibility/ or (likelihood or (likelihood
adj2 ratio:)).mp. or ((roc adj2 curve:) or (Receiver adj5 curve*)).mp. or ((roc adj2 curve:)
or (Receiver adj5 curve*)).mp. or ((diagnostic adj5 error:) or (false adj5 negative:) or
(false adj5 positive:)).mp.
6 4 and 5 Diagnostic term limits
7 meningitis/ or bacterial meningitis/ or (bacterial adj2 Meningitis).mp. Bacterial Meningitis terms
8 6 and 7 Final results
518 D. M. Kulik et al.

APPENDIX 2: QUALITY ASSESSMENT SCORE FOR CLINICAL PREDICTION-RULE DERIVATION

Criteria Score

Data collected prospectively 1 = yes

Study site(s) well described 1 = yes
Study population well described 1 = age, patient presentation reported
Predictors well defined 1 = predictors were unambiguous
Blinding of predictor assessors 1 = predictor assessors were blind to CSF results
Outcome well defined 1 = outcome was unambiguous
Blinding of outcome assessors 1 = outcome assessors were blind to predictors
Predictors reproducible 1 = reproducibility in children reported and K < 0.5
Adequate follow-up of outcomes 1 = outcomes assessed after hospital discharge
Rule applied to all patients at risk 1 = rule applied to all patients with features suggestive of bacterial meningitis
Adequate model power 1 = number of outcomes to potential predictors at 10 or more
Adequate reporting of results 1 = sensitivity, specificity, PPV, and NPV reported
Course of action described 1 = rule suggests CSF analysis and bacterial meningitis treatment or observation with follow-up
Clinical sensibility 1 = predictors and outcomes were clinically meaningful
Ease of bedside use 1 = reviewers could easily apply rule at the bedside
95% CI of rule properties reported 1 = yes
Mathematical technique reported 1 = recursive partitioning or logistic regression technique well described

CI = confidence interval; CSF = cerebrospinal fluid; NPV = negative predictive value; PPV = positive predictive value.
Does This Child Have Bacterial Meningitis? 519

ARTICLE SUMMARY
1. Why is this topic important?
A clinical prediction rule to augment clinical decision
making and distinguish bacterial meningitis from aseptic
meningitis in children would be extremely helpful. Ide-
ally, such a tool would allow physicians to accurately pre-
dict which children are at high risk of bacterial meningitis,
expedite appropriate management, and facilitate with-
holding i.v. antibiotics and hospital admission for those
children at very low risk.
2. What does this study attempt to show?
This study aims to systematically identify clinical pre-
diction rules (CPRs) for children with suspected bacterial
meningitis and compare their methodological quality and
performance for diagnosing bacterial meningitis using
a recently developed framework to evaluate CPRs for
children.
3. What are the key findings?
A rigorously developed, high-performing, and
well-validated CPR ready for clinical use to guide which
children with suspected bacterial meningitis should be
hospitalized and treated with i.v. antibiotics and can be
safely discharged home was not identified. Areas for qual-
ity improvement for future CPR studies include prospec-
tive validation using standardized inclusion criteria,
adequate blinding, predictor reproducibility assessment,
and meticulous follow-up of outcomes. Prospective vali-
dation of the Bacterial Meningitis Score is recommended.
4. How is patient care impacted?
Until consistently high methodological quality and di-
agnostic performance are demonstrated through prospec-
tive validation, caution is warranted in the routine clinical
use of existing CPRs for children with suspected bacterial
meningitis.