IMPROVING PATIENT CARE

Original Research

Improved Diagnostic Accuracy of Group A Streptococcal Pharyngitis With Use of Real-Time Biosurveillance
Andrew M. Fine, MD, MPH; Victor Nizet, MD; and Kenneth D. Mandl, MD, MPH

Background: Clinical prediction rules do not incorporate real-time incidence data to adjust estimates of disease risk in symptomatic patients. Objective: To measure the value of integrating local incidence data into a clinical decision rule for diagnosing group A streptococcal (GAS) pharyngitis in patients aged 15 years or older. Design: Retrospective analysis of clinical and biosurveillance predictors of GAS pharyngitis. Setting: Large U.S.-based retail health chain. Patients: 82 062 patient visits for pharyngitis. Measurements: Accuracy of the Centor score was compared with that of a biosurveillance-responsive score, which was essentially an adjusted Centor score based on real-time GAS pharyngitis information from the 14 days before a patients visit: the recent local proportion positive (RLPP). Results: Increased RLPP correlated with the likelihood of GAS pharyngitis (r2 0.79; P 0.001). Local incidence data enhanced diagnostic models. For example, when the RLPP was greater than 0.30, managing patients with Centor scores of 1 as if the scores

were 2 would identify 62 537 previously missed patients annually while misclassifying 18 446 patients without GAS pharyngitis. Decreasing the score of patients with Centor values of 3 by 1 point for an RLPP less than 0.20 would spare unnecessary antibiotics for 166 616 patients while missing 18 812 true-positive cases. Limitations: Analyses were conducted retrospectively. Real-time regional data on GAS pharyngitis are generally not yet available to clinicians. Conclusion: Incorporating live biosurveillance data into clinical guidelines for GAS pharyngitis and other communicable diseases should be considered for reducing missed cases when the contemporaneous incidence is elevated and for sparing unnecessary antibiotics when the contemporaneous incidence is low. Delivering epidemiologic data to the point of care will enable the use of real-time pretest probabilities in medical decision making. Primary Funding Source: Centers for Disease Control and Prevention and the National Library of Medicine, National Institutes of Health.
Ann Intern Med. 2011;155:345-352. For author affiliations, see end of text. www.annals.org

or communicable diseases, the risk for infection depends on local incidence (1, 2). However, clinicians rarely can access those data at the point of care and do not use formal quantitative approaches to adjust estimates of disease probability based on local incidence (3). The maturation of real-time infectious disease surveillance systems (37), coupled with the increased adoption of electronic health records, offers opportunities to improve clinical decision making by incorporating up-to-date information about local disease incidence into decision support algorithms. Even for such diseases as inuenza, for which robust surveillance exists and results are widely publicized, there are no quantitative, automated methods to adjust decision support with real-time incidence data (8). In this study, we evaluated use of real-time biosurveillance data for the care of adults and older adolescents with pharyngitis. Group A streptococcal (GAS) pharyngitis is the most common bacterial form of acute pharyngitis, with 600 million cases occurring annually worldwide (9). The disease occurs across the age spectrum, with a peak incidence in school-aged children (age 5 to 15 years). Timely antibiotic treatment reduces acute rheumatic fever, suppurative complications, symptom duration, missed work and school days, and transmission (10). Because physical examination is an unreliable method to diagnose GAS pharyngitis, the American College of Physicians (ACP) and the Centers for Disease Control and Prevention (CDC) advocate using the

Centor score to classify GAS pharyngitis risk in adults (11, 12). Clinicians assign the Centor score (0 to 4) according to the following criteria: 1 point each for presence of exudates, history of fever, presence of swollen anterior cervical lymph nodes, and absence of cough. Despite evidence-based guidelines, patients with pharyngitis are often misclassied, leading to inappropriate antibiotics for those with viral disease and undertreatment of those with bona de GAS pharyngitis. The empirical strategy proposed by the ACP leads in some cases to unnecessary antibiotic prescription (13, 14) and contributes to antimicrobial resistance. Cases of GAS pharyngitis occur sporadically, with spatial and temporal uctuations in incidence and occasional outbreaks (1518), reecting the dynamic epidemiologic nature of causative strains (19). Even when guidelines are followed, the accuracy of rapid GAS pharyngitis testing is

See also: Print Editors Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 Web-Only Appendix Tables Appendix Figure Conversion of graphics into slides
2011 American College of Physicians 345

Original Research
Context

Improved Diagnostic Accuracy of Group A Streptococcal Pharyngitis

The Centor score is commonly used to assess the risk for group A streptococcal pharyngitis in individual patients by determining the presence or absence of specific symptoms and signs.

Contribution
Modifying the Centor score by adding biosurveillance data on the local incidence of group A streptococcal pharyngitis improved diagnostic accuracy of the score.

Caution
The study was conducted among older adolescents and adults presenting to retail pharmacy clinics. Patients suspected of having sepsis were not included.

locations. These providers have demonstrated greater than 99% adherence to an established acute pharyngitis protocol: the Strep Pharyngitis Algorithm from the Institute for Clinical Systems Improvement (24). According to this algorithm, providers collect structured data on signs and symptoms, obtain rapid GAS pharyngitis tests for all patients with pharyngitis (with conrmatory testing for those who have negative results), and treat only those with positive results. The data set was limited to patients with complete information for the following variables: visit date, MinuteClinic location, age, signs and symptoms included in the Centor score, and test results.
Test Methods

Implication
Incorporation of biosurveillance data can improve clinical decision making for a common infectious disease and reduce unnecessary use of antibiotics. The Editors

All sites used the Clinical Laboratory Improvement Amendmentswaived QuickVue In-Line Strep A test (Quidel, San Diego, California). The conrmatory test was a throat culture (42%) or streptococcal DNA probe (58%). Patients were considered positive for GAS pharyngitis if the rapid or conrmatory test result was positive (25).
Statistical Analysis

affected by the pretest probability of disease, which is related to the likelihood of exposure. In temperate climates, GAS pharyngitis may peak in the winter and early spring (16), but clinicians lack accurate methods to recognize the onset, duration, or magnitude of local outbreaks from year to year. In this study, we compared the diagnostic accuracy of the Centor score adjusted with real-time contemporaneous data on local disease incidence (biosurveillance) with that of the Centor score alone to identify GAS pharyngitis. We limited the clinical analysis to patients aged 15 years or older, for whom the Centor score and ACP guidelines are intended.

METHODS
Participants and Setting

We retrospectively analyzed data collected from patients tested for GAS pharyngitis when they presented with pharyngitis to MinuteClinic, a large, national retail health chain with more than 500 sites in 26 states (20 23). MinuteClinic provided data for the following periods: 1 October 2006 to 6 February 2008 and 14 August 2008 to 31 October 2008. Patients were included if they presented with a primary symptom of pharyngitis and were tested for GAS pharyngitis or if they had symptoms of pharyngitis and were tested for GAS pharyngitis. MinuteClinic practice is to care only for patients who do not appear septic. Patients could contribute more than 1 encounter. We did not exclude patients who were pregnant, were known to have been treated for streptococcal pharyngitis in the prior month, were already receiving antibiotics, or had comorbid diseases. Nurse practitioners or physician assistants routinely collected a consistent set of historical and physical examination elements stored in a common database across all
346 20 September 2011 Annals of Internal Medicine Volume 155 Number 6

Statistical analysis was restricted to the 9 MinuteClinic markets with at least 7000 patient visits each for pharyngitis over the study period, encompassing 132 821 patient encounters across 6 states (Georgia, Indiana, Maryland, Minnesota, North Carolina, and Tennessee). For clinical analyses, we included visits from patients at least 15 years of age (n 82 062), with two thirds (n 54 981) selected randomly for derivation and the rest (n 27 081) for validation. We included data from patients younger than age 15 years (n 50 759) when calculating the overall local incidence data because these patients contribute to the epidemiologic context of the clinically analyzed population. To enable integration of contemporaneous, local epidemiologic data on GAS pharyngitis, we created a biosurveillance variable reecting disease incidence called recent local proportion positive (RLPP). This moving window was dened as follows: RLPP number of patients testing positive for GAS pharyngitis in market A in the previous 14 days/total number of patients tested for GAS pharyngitis in market A in previous 14 days. To calculate the 14-day RLPP for a patient seen on 15 October 2007, for example, we divided the number of positive GAS pharyngitis test results by the number of GAS pharyngitis tests sent in that market from 1 to 14 October 2007. We calculated 3-, 7-, and 14-day RLPPs and compared them by using Pearson correlation coefcients. Patients were grouped by Centor score (0 to 4) and RLPP (in intervals of 0.01). For example, we calculated the proportion of patients with a Centor score of 2 who tested positive for GAS pharyngitis when the RLPP was 0.30. We performed this calculation for all combinations of Centor scores (0 to 4) and RLPPs (0.10 to 0.40) when at least 40 patients shared a combination. For each Centor score, we plotted the proportion of patients testing positive as a
www.annals.org

Improved Diagnostic Accuracy of Group A Streptococcal Pharyngitis

Original Research
54 981

Table 1. Clinical Characteristics of Patients Presenting With Pharyngitis to the Retail Health Clinics (Derivation Set Patient Visits)
Characteristic Women, % Mean age (median [interquartile range]), y Age group, % 1518 y 1939 y 4059 y 60 y Fever, % Swollen anterior cervical lymph nodes, % Absence of cough, % Tonsillar exudates, % Distribution by Centor score*, % 0 1 2 3 4 Overall (n 54 981) 68 33.5 (33 [2441]) 13 58 26 3 31 61 68 22 8 33 34 18 6 Patients Positive for GAS Pharyngitis (n 13 823) 66 33.2 (33 [2540]) 10 64 24 2 45 77 76 40 3 18 34 31 15

Patients Negative for GAS Pharyngitis (n 41 158) 68 33.5 (33 [2441]) 14 56 26 3 26 55 66 16 10 38 35 14 3

GAS group A streptococcal. * To calculate the Centor score for a patient who presents with acute pharyngitis, 1 point is given for each of the following items: presence of tonsillar exudates, history of fever, presence of swollen anterior cervical lymph nodes, and absence of cough (11).

function of RLPP. Linear regression was used to determine the strength of association between RLPP and GAS pharyngitis positivity for each Centor score, and the Pearson coefcient was used for correlation. We considered the ACP guideline recommending that patients with a Centor score of 1 should not be tested or treated for GAS pharyngitis, those with a score of 2 should be tested and treated only if they test positive, and those with a score of 3 should be treated empirically (1 of 2 ACP options). For patients with a Centor score of 1 or 2, we evaluated a biosurveillance-responsive score constructed by adding 1 point to the score of patients when the RLPP exceeded the following thresholds: 0.20, 0.25, 0.30, and 0.35. When the RLPP exceeded the threshold and triggered an increase in the score, we calculated the number of patients in that group who would be correctly and incorrectly reclassied as positive for GAS pharyngitis. The assumption is that testing is 80% sensitive and 95% specic (26). We subtracted the number incorrectly reclassied from the number correctly reclassied to determine the net reclassication; we then divided the net reclassication by the number of patients to determine the net percentage reclassication at each RLPP threshold. To facilitate comparison across the different RLPP thresholds, we calculated the number of patients reclassied correctly and incorrectly for hypothetical cohorts of 1000 patients. To determine the number needed to test to identify each additional case of GAS pharyngitis, we divided the additional number of patients tested by the additional number correctly reclassied as positive for GAS pharyngitis at each threshold. We calculated national estimates of the number of patients for whom management would have been different with use of the biosurveillance-responsive approach comwww.annals.org

pared with the traditional Centor score approach. To calculate the national effect on an estimated 10.5 million annual national pharyngitis visits (27), we extrapolated the relevant distributions from the MinuteClinic data for age (62% of patients were 15 years of age), Centor score (32% of patients had a Centor score of 1), and RLPP (21% of patients presented when the RLPP exceeded 0.30). We examined the effect of reducing the score by 1 point for patients with a Centor score of 2 or 3 if the RLPP was below the following thresholds: 0.15, 0.20, and 0.25. We calculated the number of patients and number per 1000 who were correctly and incorrectly reclassied as negative for GAS pharyngitis, the net number and net percentage reclassied, and the estimated number of patients nationally whose management would have been altered. The Childrens Hospital Boston Committee on Clinical Investigation approved this database analysis. We performed statistical analyses by using JMP software, version 8 (SAS Institute, Cary, North Carolina).
Role of the Funding Source

This work was supported by the Centers for Disease Control and Prevention and by the National Library of Medicine, National Institutes of Health. The funding sources had no role in the study design, data analysis, data interpretation, writing of the manuscript, or the decision to submit the manuscript for publication.

RESULTS
Table 1 shows patient characteristics for the derivation set (n 54 981), including distribution by sex, age, clinical ndings, and Centor score. Two thirds of patients had Centor scores of 1 or 2, and 68% of patients were female.
20 September 2011 Annals of Internal Medicine Volume 155 Number 6 347

Original Research

Improved Diagnostic Accuracy of Group A Streptococcal Pharyngitis

Figure 1. Proportion of cases positive for GAS pharyngitis, by study week for 9 locations.
0.6 0.6 0.6

Maryland
0.4 0.4

NC 1
0.4

Patients Testing Positive for GAS Pharyngitis

0.2

0.2

0.2

Indiana
0 0.6 0 0.6 0 0.6

Georgia 1
0.4 0.4

NC 2
0.4

Minnesota 1

0.2

0.2

0.2

0 0.6

0 0.6

0 0.6

Georgia 2
0.4 0.4

Tennessee
0.4

Minnesota 2

0.2

0.2

0.2

01 /2 0 03 07 /2 0 05 07 /2 0 07 07 /2 00 09 7 /2 0 11 07 /2 00 7

01 /2 0 03 07 /2 0 05 07 /2 0 07 07 /2 00 09 7 /2 0 11 07 /2 00 7

7 00 11 /2

00

00

00

/2

/2

00

/2

03

05

/2

01

Week of Year

The x-axis represents the time from 1 January 2007 to 31 December 2007, and the y-axis is the weekly proportion of cases positive for GAS pharyngitis. Each graph shows the proportion of patients who tested positive each week in 1 of 9 markets. The axes have been standardized to allow comparison across markets. The horizontal line is the average across all markets (0.25) and is provided for reference and to facilitate comparison. The hash marks on the x-axis display bimonthly intervals. GAS group A streptococcal; NC North Carolina.

Thirteen percent of the patients were 15 to 18 years of age. Most patients (91%) had a single encounter, and 7% had 2 encounters. In the validation set, the distribution of patients was the same as in the derivation set for age, sex, presence of fever, presence of swollen anterior cervical nodes, absence of cough, presence of tonsillar exudates, and distribution of Centor score. The median number of patients tested per month was 7289 (interquartile range, 4365 to 8602). With regard to volume over time, 29 826 patients were tested in the rst quarter of the study, 32 143 in the second, 36 156 in the third, and 34 696 in the fourth. The number of patient visits for the entire study population was as follows: Georgia 1 (n 7777), Georgia 2 (n 9797), Maryland (n 9720), North Carolina 1 (n 12 236), North Carolina 2 (n 10 122), Indiana (n 8901), Tennessee (n 15 365), Minnesota 1 (n 30 391), and Minnesota 2 (n 27 972). The proportion of all patients testing positive varied across time and location, demonstrating no obvious predictable season for GAS pharyngitis (Figure 1). For example, during the week of 24 December 2007, the proportions positive were below 20% in 3 markets, 20% to 29% in 4, and greater than 30% in 2. The 3-, 7-, and 14-day RLPPs were strongly correlated (14 vs. 7: r2 0.79, P 0.001; 7 vs. 3: r2 0.63, P 0.001; 14 vs. 3, r2 0.48,
348 20 September 2011 Annals of Internal Medicine Volume 155 Number 6

P 0.001); thus, we used the 14-day RLPP for subsequent analyses because it provides a realistic time frame to generate reliable, contemporaneous local data on GAS pharyngitis. Overall, 25% of all patients tested positive for GAS pharyngitis in the derivation and validation sets, greater than the 17% in the original Centor study (11) but similar to ndings in Wigton and colleagues validation study (26%) (28). For patients with Centor scores of 1 to 4, the proportion testing positive is lowest when the RLPP is low and increases with rising RLPP (P values for slopes 0.001). Figure 2 illustrates the proportion testing positive plotted by RLPP, grouped by Centor score. Each point on the graph represents a group of patients with an identical Centor scoreRLPP dyad. Overall, a patient with a Centor score of 3 is more likely than a patient with a Centor score of 2 to test positive for GAS pharyngitis (43% to 25%), but this changes under particular epidemiologic conditions. To illustrate, 350 of 1053 (33% [95% CI, 30% to 36%]) patients with a Centor score of 2 test positive when the RLPP is greater than 0.35, compared with 109 of 328 (33% [CI, 28% to 38%]) with a Centor score of 3 when the RLPP is less than 0.15. The Appendix Figure (available at www.annals.org) shows a similar graphical distribution for the same analysis with use of the validation set.
www.annals.org

07

09

/2

00

Improved Diagnostic Accuracy of Group A Streptococcal Pharyngitis

Original Research

We measured the effect of adding 1 point to a Centor score of 1 when the RLPP exceeded specic thresholds. Hypothetically, testing 1000 patients when the RLPP is greater than 0.30, for example, would correctly reclassify 139 and incorrectly reclassify 41 patients as positive, a ratio exceeding 3:1 (Table 2). GAS group A streptococcal; RLPP recent local proportion positive. By extrapolation, we estimated that approximately 449 908 of the 10.5 million pharyngitis visits per year would occur among persons 15 years of age or older, with a Centor score of 1 when the RLPP exceeded 0.30. The biosurveillance approach for guiding the care of these patients would identify 62 537 additional GAS pharyngitis cases in the United States annually and lead to treatment of an additional 18 446 patients without GAS pharyngitis. The number needed to test to detect each additional case of GAS pharyngitis is 7.2. Appendix Table 1 (available at www.annals.org) displays the corresponding analyses for the validation set: 60 048 additional cases would be identied, and 18 103 patients without GAS pharyngitis would be treated. In the validation set, the number needed to test is 7.4. We examined outcomes generated by adding 1 point to the Centor score of adults and older adolescents with a score of 2 (n 18 942) at specic RLPP cutoffs (Table 2). Increasing the score incrementally when the RLPP is greater than 0.30 and empirically treating a simulated cohort of 1000 of these patients would correctly reclassify 62 but incorrectly reclassify 657 patients as positive. Through extrapolation, this approach correctly identies 29 450 additional GAS pharyngitis cases in the United States annually, but at a cost of inappropriately treating 312 077 patients without GAS pharyngitis. Next, we tabulated outcomes generated by subtracting 1 point from the Centor score of all adults and older adolescents with a score of 3 (n 10 056) or 2 (n 18 942) at dened RLPP cutoffs (Table 3). Testing 1000 hypothetical patients with a score of 3 when the RLPP is less than

Figure 2. Proportion of patients testing positive for GAS pharyngitis, by RLPP and grouped and labeled by Centor score.
Patients Testing Positive for GAS Pharyngitis
0.80 4

0.60 3

Centor Score

0.40 2

0.20

1 0

0.00 0.1 0.2 0.3 0.4

RLPP for GAS Pharyngitis

Each line represents patients with the same Centor score across varying RLPPs. The proportion of patients who tested positive increases both as the clinical score and the RLPP increase. The Pearson coefcient was used to measure the strength of correlation. The r 2 values, representing the proportion of the variation in GAS pharyngitis positivity that can be attributed to the RLPP, are 0.81 (P 0.001) for a Centor score of 4, 0.86 (P 0.001) for a Centor score of 3, 0.86 (P 0.001) for a Centor score of 2, 0.70 (P 0.001) for a Centor score of 1, and 0.47 (P 0.001) for a Centor score of 0. The slopes of the lines for Centor scores of 4, 3, 2, 1, and 0 are 1.21, 0.88, 0.64, 0.37, and 0.29, respectively. Each data point represents a median of 223 patients (range, 41 to 1152 patients; interquartile range, 115 to 518 patients). GAS group A streptococcal; RLPP recent local proportion positive.

0.20, rather than treating them empirically, would correctly reclassify 620 but incorrectly reclassify 70 patients as negative. In the United States, this could spare antibiotics for 166 616 patients (5 million doses) while missing only 18 812 cases annually. Appendix Table 2 (available at www.annals.org) shows the corresponding data from the validation set: A total of 169 637 patients would be spared antibiotics, whereas 17 632 patients would be missed.

Table 2. Reclassification Accuracy of Adjusted Centor Score Resulting From Increasing Score by Increments of 1 Point at Different
Thresholds of GAS Pharyngitis Activity*
Centor Score RLPP Threshold Cases Reclassified as Positive for GAS Pharyngitis Incorrect, n 1 0.20 (n1 0.25 (n1 0.30 (n1 0.35 (n1 0.20 (n2 0.25 (n2 0.30 (n2 0.35 (n2 13 056) 8796) 4355) 1579) 13 987) 9583) 4746) 1818) 556 370 180 65 9738 6555 3117 1150 Correct, n 1542 1119 607 216 747 537 293 122 Net Change, n (%) 985 (8) 749 (9) 427 (10) 151 (10) 8991 ( 64) 6018 ( 63) 2824 ( 60) 1028 ( 57) Cases Reclassified per 1000, n Incorrect 43 42 41 41 696 684 657 632 Correct 118 127 139 137 53 56 62 67 National Estimates of Numbers Affected, n Incorrect 63 647 40 058 18 446 7394 1 087 646 688 768 312 077 120 336 Correct 174 658 121 129 62 537 24 707 82 824 56 390 29 450 12 757

GAS group A streptococcal; RLPP recent local proportion positive. * Adjustment of Centor score to reclassify risk according to prior probability of disease as inferred by the RLPP. For patients with a Centor score of 1 or 2, the score was increased by an increment of 1 point when RLPP exceeded thresholds. Changing from 1 to 2 would result in testing n1 patients for GAS pharyngitis, and changing from 2 to 3 would result in treating n2 patients empirically.
www.annals.org 20 September 2011 Annals of Internal Medicine Volume 155 Number 6 349

Original Research
GAS Pharyngitis Activity*
Centor Score RLPP Threshold

Improved Diagnostic Accuracy of Group A Streptococcal Pharyngitis

Table 3. Reclassification Accuracy of Adjusted Centor Score Resulting From Decreasing Score by 1 Point at Different Thresholds of

Cases Reclassified as Negative for GAS Pharyngitis Incorrect, n Correct, n 344 176 35 2574 1289 233 Net Change, n (%) 1154 ( 13) 479 ( 11) 91 ( 11) 2249 (52) 1144 (55) 208 (56)

Cases Reclassified per 1000, n Incorrect 171 151 148 75 70 68 Correct 39 41 41 594 620 627

National Estimates of Numbers Affected, n Incorrect 175 950 76 433 13 741 40 971 18 812 3352 Correct 40 129 20 753 3807 324 492 166 616 30 907

0.25 (n2 0.20 (n2 0.15 (n2 0.25 (n3 0.20 (n3 0.15 (n3

8746) 4342) 847) 4334) 2080) 371)

1498 655 126 325 145 25

GAS group A streptococcal; RLPP recent local proportion positive. * Adjustment of Centor score to reclassify risk according to prior probability of disease as inferred by the RLPP. For patients with a Centor score of 2 or 3, the score was decreased by 1 point when RLPP was below threshold. Changing from 2 to 1 would result in not testing or treating n2 patients for GAS pharyngitis, and changing from 3 to 2 would result in testing n3 patients and treating them if they tested positive.

DISCUSSION
We asked the question, What if clinicians could integrate real-time incidence data into clinical decision making for patients? Retail health data on pharyngitis were ideal to address this question because clinical scores have been validated to estimate risk for GAS pharyngitis in adults, the disease is common, disease incidence varies temporally and spatially, and MinuteClinic protocol dictates that all patients with pharyngitis undergo GAS pharyngitis testing, regardless of Centor score. We show that contextualizing the Centor score by using biosurveillance data to calculate the RLPP improves prediction of positivity for GAS pharyngitis in patients presenting with pharyngitis across all clinical risk categories and is especially important when the RLPP is very high or very low. For adults and older adolescents, adjusting management on the basis of a biosurveillance approach could improve health outcomes and efciency of health care delivery. The ACP/CDC recommends not testing or treating adults who are unlikely to have GAS pharyngitis (Centor score of 0 or 1). We show that adults and older adolescents with a Centor score of 0 are unlikely to have GAS pharyngitis regardless of RLPP; even when the RLPP is elevated, their risk for GAS pharyngitis rarely exceeds 15%. However, our data suggest that during times of elevated RLPP, clinicians should consider testing adults and older adolescents who have a Centor score of 1. Adding 1 point for adults and older adolescents with a Centor score of 1 offers greater overall benet than adding 1 point to those with a score of 2. In addition, subtracting 1 point from a score of 3 when the RLPP is low spares unnecessary empirical antibiotic therapy and just slightly decreases case identication. Practice guidelines traditionally account for clinical features, and to some extent seasonality, without regard for real-time epidemiologic data on disease incidence. Some clinicians informally incorporate epidemiologic incidence data into their heuristics for medical decision making, but a clinicians knowledge of current disease trends may be
350 20 September 2011 Annals of Internal Medicine Volume 155 Number 6

prejudiced by recent or unusual cases, leading to cognitive bias and over- or underestimation of the true incidence of disease (29, 30). The propagation of retail health clinics in geographically diverse areas provided an opportunity to evaluate how local incidence data collected by using an unbiased, robust, uniform information system can improve clinical decision making. Developing formal guidelines for incorporating biosurveillance data into the diagnostic and treatment algorithms for GAS pharyngitis necessitates a discussion of the relative trade-offs between missed cases and overtreatment of patients without true bacterial infection. For example, increasing the Centor score by 1 would have resulted in 3.4 additional cases of GAS pharyngitis diagnosed for every additional negative patient treated. Nevertheless, the public health concern of antibiotic resistance must be weighed against the benets of preventing complications and morbid conditions from missed cases.
Limitations

Patients presenting to retail health clinics may be less acutely ill and have fewer comorbid conditions than patients with pharyngitis who present to other types of health care settings, and our ndings should not be applied to individuals suspected of having sepsis. Further, although all clinical and laboratory data were collected prospectively, the analyses were conducted retrospectively. Logistic concerns must be considered when the RLPP is integrated into a clinical decision-support framework. The RLPP surveillance signal stream would weaken if clinicians tested fewer low-risk patients when the RLPP is low. To preserve the accuracy of the RLPP metric, it would be imperative to maintain a stream of data through the random, intentional testing of a threshold sample of patients weekly, regardless of Centor score and RLPP. This testing approach could provide value in disease management from the population health perspective. Our data cannot inform the debate about the importance of the streptococcal carrier state because serologic
www.annals.org

Improved Diagnostic Accuracy of Group A Streptococcal Pharyngitis

Original Research

tests for GAS pharyngitis were not performed. In this analysis, all patients were symptomatic with pharyngitis, and this clinical presentation, coupled with positive results on GAS pharyngitis testing, is universally approached as true infection in clinical practice and decision-making algorithms. We did not have information about family or close contacts who tested positive for GAS pharyngitis, but we expect that clinicians would continue to consider this information, if available. Our analyses did not cover all regions of the country but did include 6 different states. Although the very large sample size is a strength of our study, the dependence on multiple providers clinical assessments may introduce some variability. However, all information was collected by nurse practitioners and physician assistants trained to evaluate pharyngitis according to a strict, computer-driven protocol. Data were not available to permit calculation of intra- or interobserver variation in the collection of clinical ndings that contribute to the Centor score. We are limited by the absence of data on locations of patients schools or jobs that might permit more rened epidemiologic modeling. Studies have shown GAS pharyngitis outbreaks differentially infecting students at different schools within a town (31). Despite these limitations, we felt that retail health data on pharyngitis provided the best available data to address the question of whether local incidence could help improve care for symptomatic patients.
Conclusions

epidemiology to individual patients (7). The $48 billion investment is intended to support a learning health system (37) with bidirectional communication between clinical and public health sites, along with delivery of clinical decision support to the point of care. Our ndings suggest that this learning health system should be capacitated to link live biosurveillance data with clinical decision making.
From Childrens Hospital Boston, Harvard Medical School, HarvardMIT Health Sciences and Technology, Boston, Massachusetts, and University of California, San Diego, La Jolla, California.
Acknowledgment: The authors thank CVS/Caremark and Minute-

Clinic for use of the data.

Grant Support: By the Mentored Public Health Research Scientist Development Award K01HK000055 from the Centers for Disease Control and Prevention (CDC), by Public Health Informatics Center of Excellence Award P01HK000088 from CDC, and by grants 1G08LM009778 and R01 LM007677 from the National Library of Medicine, National Institutes of Health. Potential Conflicts of Interest: Disclosures can be viewed at www.acponline

.org/authors/icmje/ConictOfInterestForms.do?msNum M11-0140.
Reproducible Research Statement: Study protocol and data set: Not available. Statistical code: Available from Dr. Fine. Requests for Single Reprints: Andrew M. Fine, MD, MPH, Emer-

gency Medicine-Main 1, Childrens Hospital Boston, 300 Longwood Avenue, Boston, MA 02115. Current author addresses and author contributions are available at www .annals.org.

Group A streptococcal pharyngitis occurs throughout the calendar year, and thus live biosurveillance becomes particularly helpful. Because GAS pharyngitis is a common condition affecting millions per year in the United States alone, small improvements in diagnostic accuracy have substantial impact. Incorporating real-time, biosurveillance-derived epidemiologic data into a prediction rule based on a patients signs, symptoms, location, and timing of presentation (3234) suggests the value of this epidemiologic analogue to personalized genomic medicine (35); with the biosurveillance model, the context, instead of being derived from ones genes, derives from a quantitative representation of the epidemiologic milieu. Developers of future recommendations for managing GAS pharyngitis should consider incorporating real-time epidemiologic data with clinical factors. This novel approach in clinical guideline creation could help restratify risk for patients when the contemporaneous incidence of disease is very high or very low, thereby improving diagnostic accuracy. Although our study focused on GAS pharyngitis, the results may have broad implications for other communicable diseases, for which real-time biosurveillance data might be used to more accurately deduce the likelihood that a symptomatic individual has a specic disease. The massive federal investment in health information technology (36) and the maturation of real-time biosurveillance systems present new opportunities to apply real-time
www.annals.org

References
1. Fisman DN. Seasonality of infectious diseases. Annu Rev Public Health. 2007; 28:127-43. [PMID: 17222079] 2. Wallinga J, Teunis P, Kretzschmar M. Using data on social contacts to estimate age-specic transmission parameters for respiratory-spread infectious agents. Am J Epidemiol. 2006;164:936-44. [PMID: 16968863] 3. Syndromic surveillance. Reports from a national conference, 2004. MMWR Morb Mortal Wkly Rep. 2005;54 Suppl:1-180. 4. Brownstein JS, Freifeld CC, Chan EH, Keller M, Sonricker AL, Mekaru SR, et al. Information technology and global surveillance of cases of 2009 H1N1 inuenza. N Engl J Med. 2010;362:1731-5. [PMID: 20445186] 5. Brownstein JS, Kleinman KP, Mandl KD. Identifying pediatric age groups for inuenza vaccination using a real-time regional surveillance system. Am J Epidemiol. 2005;162:686-93. [PMID: 16107568] 6. Lipsitch M, Hayden FG, Cowling BJ, Leung GM. How to maintain surveillance for novel inuenza A H1N1 when there are too many cases to count. Lancet. 2009;374:1209-11. [PMID: 19679345] 7. Mandl KD, Overhage JM, Wagner MM, Lober WB, Sebastiani P, Mostashari F, et al. Implementing syndromic surveillance: a practical guide informed by the early experience. J Am Med Inform Assoc. 2004;11:141-50. [PMID: 14633933] 8. Call SA, Vollenweider MA, Hornung CA, Simel DL, McKinney WP. Does this patient have inuenza? JAMA. 2005;293:987-97. [PMID: 15728170] 9. Carapetis JR, Steer AC, Mulholland EK, Weber M. The global burden of group A streptococcal diseases. Lancet Infect Dis. 2005;5:685-94. [PMID: 16253886] 10. Ebell MH, Smith MA, Barry HC, Ives K, Carey M. The rational clinical examination. Does this patient have strep throat? JAMA. 2000;284:2912-8.
20 September 2011 Annals of Internal Medicine Volume 155 Number 6 351

Original Research

Improved Diagnostic Accuracy of Group A Streptococcal Pharyngitis

1299-303. [PMID: 18780914] 24. Institute for Clinical Systems Improvement Health Care Guideline. Strep pharyngitis algorithm. Accessed at www.icsi.org/respiratory_illness_in_children _and_adults__guideline_/respiratory_illness_in_children_and_adults__guideline __13116.html on 29 April 2011. 25. Woodburn JD, Smith KL, Nelson GD. Quality of care in the retail health care setting using national clinical guidelines for acute pharyngitis. Am J Med Qual. 2007;22:457-62. [PMID: 18006426] 26. Corneli HM. Rapid strep tests in the emergency department: an evidencebased approach. Pediatr Emerg Care. 2001;17:272-8. [PMID: 11493831] 27. Middleton K, Hing E, Xu J. National Hospital Ambulatory Medical Care Survey: 2005 outpatient department summary. Adv Data. 2007:1-34. [PMID: 17784724] 28. Wigton RS, Connor JL, Centor RM. Transportability of a decision rule for the diagnosis of streptococcal pharyngitis. Arch Intern Med. 1986;146:81-3. [PMID: 3510600] 29. Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;78:775-80. [PMID: 12915363] 30. Kahneman D, Slovic P, Tversky A. Judgment Under Uncertainty: Heuristics and Biases. New York: Cambridge Univ Pr; 1982. 31. Cornfeld D, Hubbard JP, Harris TN, Weaver R. Epidemiologic studies of streptococcal infection in school children. Am J Public Health Nations Health. 1961;51:242-9. [PMID: 13695599] 32. Fine AM, Brownstein JS, Nigrovic LE, Kimia AA, Olson KL, Thompson AD, et al. Integrating spatial epidemiology into a decision model for evaluation of facial palsy in children. Arch Pediatr Adolesc Med. 2011;165:61-7. [PMID: 21199982] 33. Fine AM, Reis BY, Nigrovic LE, Goldmann DA, Laporte TN, Olson KL, et al. Use of population health data to rene diagnostic decision-making for pertussis. J Am Med Inform Assoc. 2010;17:85-90. [PMID: 20064807] 34. Fine AM, Nigrovic LE, Reis BY, Cook EF, Mandl KD. Linking surveillance to action: incorporation of real-time regional data into a medical decision rule. J Am Med Inform Assoc. 2007;14:206-11. [PMID: 17213492] 35. Snyderman R, Dinan MA. Improving health by taking it personally. JAMA. 2010;303:363-4. [PMID: 20103762] 36. Blumenthal D. Launching HITECH. N Engl J Med. 2010;362:382-5. [PMID: 20042745] 37. Friedman CP, Wong AK, Blumenthal D. Achieving a nationwide learning health system. Sci Transl Med. 2010;2:57cm29. [PMID: 21068440]

[PMID: 11147989] 11. Centor RM, Witherspoon JM, Dalton HP, Brody CE, Link K. The diagnosis of strep throat in adults in the emergency room. Med Decis Making. 1981;1:239-46. [PMID: 6763125] 12. Snow V, Mottur-Pilson C, Cooper RJ, Hoffman JR; American Academy of Family Physicians. Principles of appropriate antibiotic use for acute pharyngitis in adults. Ann Intern Med. 2001;134:506-8. [PMID: 11255529] 13. Bisno AL, Peter GS, Kaplan EL. Diagnosis of strep throat in adults: are clinical criteria really good enough? Clin Infect Dis. 2002;35:126-9. [PMID: 12087517] 14. McIsaac WJ, Kellner JD, Aufricht P, Vanjaka A, Low DE. Empirical validation of guidelines for the management of pharyngitis in children and adults. JAMA. 2004;291:1587-95. [PMID: 15069046] 15. Asteberg I, Andersson Y, Dotevall L, Ericsson M, Darenberg J, HenriquesNordmark B, et al. A food-borne streptococcal sore throat outbreak in a small community. Scand J Infect Dis. 2006;38:988-94. [PMID: 17148066] 16. Bisno AL. Acute pharyngitis. N Engl J Med. 2001;344:205-11. [PMID: 11172144] 17. Danchin MH, Rogers S, Kelpie L, Selvaraj G, Curtis N, Carlin JB, et al. Burden of acute sore throat and group A streptococcal pharyngitis in school-aged children and their families in Australia. Pediatrics. 2007;120:950-7. [PMID: 17974731] 18. Martin JM, Green M, Barbadora KA, Wald ER. Erythromycin-resistant group A streptococci in schoolchildren in Pittsburgh. N Engl J Med. 2002;346: 1200-6. [PMID: 11961148] 19. Kaplan EL, Wotton JT, Johnson DR. Dynamic epidemiology of group A streptococcal serotypes associated with pharyngitis. Lancet. 2001;358:1334-7. [PMID: 11684215] 20. Mehrotra A, Liu H, Adams JL, Wang MC, Lave JR, Thygeson NM, et al. Comparing costs and quality of care at retail clinics with that of other medical settings for 3 common illnesses. Ann Intern Med. 2009;151:321-8. [PMID: 19721020] 21. Rudavsky R, Pollack CE, Mehrotra A. The geographic distribution, ownership, prices, and scope of practice at retail clinics. Ann Intern Med. 2009;151: 315-20. [PMID: 19721019] 22. Pollack CE, Armstrong K. The geographic accessibility of retail clinics for underserved populations. Arch Intern Med. 2009;169:945-9. [PMID: 19468086] 23. Costello D. A checkup for retail medicine. Health Aff (Millwood). 2008;27:

MANUSCRIPT PROCESSING

AND

TURNAROUND

Annals sends about half of submitted manuscripts for peer review and publishes about 10% of submitted material. The 2010 processing and notification turnaround time for manuscripts that were rejected without external peer review was within 1 week for more than 95% of submitted manuscripts. The processing and notification turnaround time for manuscripts that were received and rejected after external peer review was within 4 weeks for 50% and within 8 weeks for 98%.

352 20 September 2011 Annals of Internal Medicine Volume 155 Number 6

www.annals.org

IMPROVING PATIENT CARE

Current Author Addresses: Dr. Fine: Emergency Medicine-Main 1,

Childrens Hospital Boston, 300 Longwood Avenue, Boston, MA 02115. Dr. Nizet: Division of Pediatric Pharmacology and Drug Discovery, UCSD School of Medicine, Cellular & Molecular Medicine East, Room 1066, 9500 Gilman Drive Mail Code 0687, La Jolla, CA 92093-5611. Dr. Mandl: Childrens Hospital Boston, 300 Longwood Avenue, Boston, MA 02115.

Author Contributions: Conception and design: A.M. Fine, K.D. Mandl. Analysis and interpretation of the data: A.M. Fine, V.N. Nizet, K.D. Mandl. Drafting of the article: A.M. Fine, K.D. Mandl. Critical revision of the article for important intellectual content: A.M. Fine, V.N. Nizet, K.D. Mandl. Final approval of the article: A.M. Fine, V.N. Nizet, K.D. Mandl. Provision of study materials or patients: A.M. Fine, K.D. Mandl. Statistical expertise: A.M. Fine. Obtaining of funding: A.M. Fine, K.D. Mandl. Administrative, technical, or logistic support: A.M. Fine, K.D. Mandl. Collection and assembly of data: A.M. Fine.

Appendix Figure. Proportion of patients in the validation set (n 27 081) testing positive for GAS pharyngitis, by RLPP and grouped and labeled by Centor score.
Patients Testing Positive for GAS Pharyngitis
0.80 4

0.60 3 0.40 2 0.20 1 0 0.1 0.2 0.3 0.4

Centor Score

0.00

RLPP for GAS Pharyngitis

Each line represents patients with the same Centor score across varying RLPPs. The proportion of patients who tested positive increases both as the clinical score and the RLPP increase. The Pearson coefcient was used to measure strength of correlation. The r 2 values, representing the proportion of the variation in GAS pharyngitis positivity that can be attributed to the RLPP, are 0.33 (P 0.012) for a Centor score of 4, 0.70 (P 0.001) for a Centor score of 3, 0.82 (P 0.001) for a Centor score of 2, 0.68 (P 0.001) for a Centor score of 1, and 0.35 (P 0.005) for a Centor score of 0. The slopes of the lines for Centor scores of 4, 3, 2, 1, and 0 are 0.99, 0.80, 0.75, 0.43, and 0.28, respectively. Each data point represents a median of 140 patients (range, 45 to 555 patients; interquartile range, 82 to 290 patients). GAS group A streptococcal; RLPP recent local proportion positive.
www.annals.org 20 September 2011 Annals of Internal Medicine Volume 155 Number 6 W-111

Appendix Table 1. Reclassification Accuracy of Adjusted Centor Score Resulting From Increasing Score by Increments of 1 Point at
Different Thresholds of GAS Pharyngitis Activity (Validation Set)*
Centor Score RLPP Threshold Cases Reclassified as Negative for GAS Pharyngitis Incorrect, n 1 0.20 (n1 0.25 (n1 0.30 (n1 0.35 (n1 0.20 (n2 0.25 (n2 0.30 (n2 0.35 (n2 8681) 3827) 1734) 633) 9378) 4145) 1969) 772) 373 160 72 26 9738 6555 3117 1150 Correct, n 974 504 236 91 747 537 293 122 Net Change, n (%) 601 (7) 344 (9) 164 (9) 65 (10) 8990 ( 64) 6018 ( 63) 2824 ( 60) 1028 ( 57) Cases Reclassified per 1000, n Incorrect 43 42 41 41 710 657 635 630 Correct 112 132 136 144 50 62 66 67 National Estimates of Numbers Affected, n Incorrect 63 118 39 543 18 103 7171 1 125 725 668 146 302 485 119 019 Correct 164 401 124 277 60 048 25 186 79 276 63 052 19 988 12 658

GAS group A streptococcal; RLPP recent local proportion positive. * Adjustment of Centor score to reclassify risk according to prior probability of disease as inferred by the RLPP. For patients with a Centor score of 1 or 2, the score was increased by an increment of 1 point when RLPP exceeded threshold. Changing from 1 to 2 would result in testing n1 patients for GAS pharyngitis, and changing from 2 to 3 would result in treating n2 patients empirically.

Appendix Table 2. Reclassification Accuracy of Adjusted Centor Score Resulting From Decreasing Score by Increments of 1 Point at
Different Thresholds of GAS Pharyngitis Activity (Validation Set)*
Centor Score RLPP Threshold Cases Reclassified as Negative for GAS Pharyngitis Cases Reclassified per 1000, n Incorrect 164 152 126 76 66 60 Correct 40 41 42 588 635 666 National Estimates of Numbers Affected, n Incorrect 169 260 75 765 11 391 42 039 17 632 2907 Correct 41 283 20 436 3797 325 246 169 637 32 268

Incorrect, n 2 0.25 (n2 0.20 (n2 0.15 (n2 0.25 (n3 0.20 (n3 0.15 (n3 4395) 2179) 388) 2169) 1044) 201) 722 330 49 165 69 12

Correct, n 175 88 16 1275 663 134

Net Change, n (%) 548 ( 12) 242 ( 11) 33 ( 8) 1110 594 (57) 122 (61)

GAS group A streptococcal; RLPP recent local proportion positive. * Adjustment of Centor score to reclassify risk according to prior probability of disease as inferred by the RLPP. For patients with a Centor score of 2 or 3, the score was decreased by 1 point when RLPP was below threshold. Changing from 2 to 1 would result in not testing or treating n2 patients for GAS pharyngitis, and changing from 3 to 2 would result in testing n3 patients and treating them if they tested positive.
W-112 20 September 2011 Annals of Internal Medicine Volume 155 Number 6 www.annals.org