Putting evidence into practice

EDITORIAL Putting Evidence into Practice C. David Naylor, MD, DPhil T he modern physician swims in a river of clinical research evidence that is unprecedented in its depth, velocity, and turbulence. Perhaps as a result, many reports during the last two decades have documented puzzling interinstitutional and inter-regional variations in practice patterns (1–3). Some variation is acceptable in the absence of definitive evidence about best practices. But when process-of-care audits are undertaken to assess practice variations (4 – 6), clinical decisions do not always reflect even clear-cut research evidence. As generators and disseminators of new knowledge, researchers and academic physicians might be expected to behave differently. It is indeed plausible that this sector of the medical profession would adopt new practices rapidly in response to evidence from randomized trials. However, in this issue of The American Journal of Medicine, Majumdar et al. put that belief to rest (7). The authors have drawn on data from the Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries (GUSTO)-1 study that was conducted from 1990 to 1993 and involved 659 hospitals across North America (8). Of these hospitals, 22 also participated in the Survival and Ventricular Enlargement (SAVE) trial (9), which showed survival benefits from the use of an angiotensin-converting enzyme (ACE) inhibitor in patients with myocardial infarction. The primary SAVE report was published in 1992, but the results had been shared earlier with participating sites. Majumdar et al. hypothesized that GUSTO-1 participants with myocardial infarction who were hospitalized at the 22 SAVE sites would be more likely to receive ACE inhibitors at discharge than were those at the other 637 hospitals. What then was the likelihood of ACE inhibitor use at discharge among patients at SAVE sites versus the rest? It was virtually identical whether considering all subjects (15% vs. 16%) or the subgroup of GUSTO-1 subjects with heart failure (30% for both sets of hospitals). Moreover, the authors controlled carefully for potential intersite differences with a fixed-effect hierarchical logistic regression that nested patients within their respective hosAm J Med. 2002;113:161–163. From the Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada. Requests for reprints should be addressed to C. David Naylor, MD, DPhil, University of Toronto, Medical Science Building, Room 2109, 1 Kings College Circle, Toronto, Ontario, Canada M5S 1A8, or medicine.dean@utoronto.ca. ©2002 by Excerpta Medica, Inc. All rights reserved. pitals, before adjusting for all relevant patient-level variables. What makes these findings dramatic is that large-scale trials such as SAVE are sometimes presumed to have positive “inoculation effects,” jump-starting the adoption of effective new treatments by introducing them to a large cohort of research-oriented practitioners in academic and community settings. Sadly, it now appears that clinicians who participate in a positive trial are as slow to adopt a highly beneficial treatment as are clinicians who do not participate. The first consolation here is the lack of an external control group. One might surmise, or at least hope, that practitioners who did not participate in studies such as GUSTO-1 or SAVE were even slower to adopt ACE inhibitors. A second consolation is that the findings of Majumdar et al. are consistent with what we have learned about clinical decision making during the last three decades. At risk of caricature, one can sketch four phases in the evolution of conventional wisdom about the relation of clinicians to research evidence. Phase 1 was the Era of Optimism. It presumed a model of passive diffusion (10). Evidence published in peer-reviewed journals would find its way into practice as clinicians filtered the medical literature and chose the most promising ideas for bedside application. Accordingly, academicians focused on training the next generation of physicians to be consumers of primary research studies, seeking, appraising, and applying evidence with machine-like efficiency. Phase 2 was the Era of Innocence Lost and Regained. As one study after another showed a disjunction of evidence and action, there was widespread criticism of physicians for failing to keep abreast of the growing medical literature. There was also increasing recognition that it was not sufficient to recreate the physician as the über consumer of clinical studies. Faith in professionalism, however, was restored through the practice guidelines movement based, in turn, on evidence synthesis and active dissemination (10,11). Meta-analyses, decision analyses, and practice guidelines would synthesize the best evidence available from multiple research studies and thereby define what a practitioner should do when confronted with a particular clinical situation. Endorsed by prestigious organizations, the resulting practice guidelines could be publicized widely and address the problem of information overload for clinicians. Phase 3, the Era of Industrialization, dawned once it 0002-9343/02/$–see front matter 161 PII S0002-9343(02)01215-9 Putting Evidence into Practice/Naylor became clear that practice guidelines were not consistently guiding practice. The new model was one of aggressive implementation rather than passive diffusion or even active dissemination (10). A veritable industry arose, focused on measuring and managing physician behavior. Although cost containment and risk management were potent drivers of the measurement-and-management movement, the industrialization of clinical quality also reflected the understandable preoccupation of the public, professional leaders, payers, and policymakers with substandard clinical decision making. Ultimately, in a brilliant countermovement, medical leaders co-opted the language of industrial quality gurus, emphasizing the need for change in management strategies that would inform and empower clinicians rather than deprofessionalize them (12). Phase 4, the Era of Information Technology and Systems Engineering, is still with us. It presupposes that we can shape clinical decisions with information tools that will make it easier for clinicians and patients to assimilate the most relevant and recent evidence. However, this era is also paying attention to research evidence of a different sort— evidence about how to change physician behavior. This evidence was usefully summarized in 1995 when Davis et al. (13) and Oxman et al. (14) conducted systematic reviews of all controlled studies of strategies designed to persuade physicians “to modify their practice performance by communicating clinical information.” Their overviews confirmed that one-off educational events or simple dissemination of educational materials had little effect on clinical decision making. Even audit and feedback had limited effect, unless undertaken on a continuous basis and coupled with feedback reminding clinicians of actions that should be taken in response to the audit results. Moreover, the evidence was compelling with respect to the sociology of behavior change. Informationbased tools were more likely to have an effect when coupled with processes that recruited opinion leaders in the local practice environment, addressed barriers to change, built a consensus in the affected professional community, and rectified specific gaps in clinical knowledge. Tellingly, multifaceted interventions showed the strongest effects on processes or outcomes of care. These findings, reinforced by subsequent research reports (15,16), have galvanized the modern systems approach to getting evidence into practice, an approach in which sophisticated information technology is a necessary but not sufficient element. Looking to the future, more research will be needed on how evidence intersects clinical practice. The classic overviews by Davis et al. (13) and Oxman et al. (14) depended in part on cross-study inferences, but those were nonrandomized comparisons with potential pitfalls. Factorial designs in studies of clinical behavior change have been uncommon, and it is seldom clear which elements in a 162 August 1, 2002 THE AMERICAN JOURNAL OF MEDICINE威 multifactorial strategy are truly essential and effective. The potential role of patients as evidence vectors is intriguing but underexplored. Not least, it may be time for education researchers to study whether we should spend more time teaching medical students about health information technology and the sociology of change management, and less time inculcating them with the time-honored principles of critical appraisal. In the interim, for clinicians depressed by this evolutionary chronicle, it may help to remember that our profession is by no means impervious to published evidence. Majumdar et al. (7) did find modest increases in the use of ACE inhibitors after publication of the SAVE study. Before publication, only 14% of GUSTO-1 participants received these drugs; after publication, the proportion rose to 18% (P ⫽ 0.001). This is consistent with research suggesting that medical practices change slowly but steadily. Indeed, there are instances, such as the utilization of carotid endarterectomy in North America (17,18), where clinical research has had rapid and dramatic effects on practice patterns. If our colleagues in surgery can assimilate evidence rapidly, there is still hope for those of us who labor in the medical vineyards! REFERENCES 1. Wennberg J. Which rate is right? N Engl J Med. 1986;314:310 –311. 2. Tu JV, Naylor CD, Kumar D, et al. Coronary artery bypass graft surgery in Ontario and New York State: which rate is right? Steering Committee of the Cardiac Care Network of Ontario. Ann Intern Med. 1997;126:13–19. 3. Di Salvo TT, Paul SD, Lloyd-Jones D, et al. Care of acute myocardial infarction by noninvasive and invasive cardiologists: procedure use, cost and outcome. J Am Coll Cardiol. 1996;27:262–269. 4. Naylor CD, Guyatt GH. Users’ guides to the medical literature. XL. How to use an article about a clinical utilization review. EvidenceBased Medicine Working Group. JAMA. 1996;275:1435–1439. 5. McAlister FA, Taylor L, Teo KK, et al. The treatment and prevention of coronary heart disease in Canada: do older patients receive efficacious therapies? The Clinical Quality Improvement Network (CQIN) Investigators. J Am Geriatr Soc. 1999;47:811–818. 6. Hemingway H, Crook AM, Feder G, et al. Underuse of coronary revascularization procedures in patients considered appropriate candidates for revascularization. N Engl J Med. 2001;344:645–654. 7. Majumdar SR, Chang W-C, Armstrong PW. Do the investigative sites that take part in a positive clinical trial translate that evidence into practice? Am J Med. 2002;113:140 –145. 8. The GUSTO Investigators. An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction. N Engl J Med. 1993;329:673–682. 9. Pfeffer MA, Braunwald E, Moye LA, et al. Effect of captopril on mortality and morbidity in patients with left ventricular dysfunction after myocardial infarction. N Engl J Med. 1992;327:669 –677. 10. Lomas J. Retailing research: increasing the role of evidence in clinical services for childbirth. Milbank Q. 1993;71:439 –475. 11. Naylor CD. Better care and better outcomes. The continuing challenge. JAMA. 1998;279:1392–1394. 12. Berwick DM. A primer on leading the improvement of systems. BMJ. 1996;312:619 –622. Volume 113 Putting Evidence into Practice/Naylor 13. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700 –705. 14. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153:1423–1431. 15. Soumerai SB, McLaughlin TJ, Gurwitz JH, et al. Effect of local medical opinion leaders on quality of care for acute myocardial infarction: a randomized controlled trial. JAMA. 1998;279:1358 – 1363. 16. Bradley EH, Holmboe ES, Mattera JA, Roumanis SA, Radford MJ, Krumholz HM. A qualitative study of increasing beta-blocker use after myocardial infarction: why do some hospitals succeed? JAMA. 2001;285:2604 –2611. 17. Simunovic M, To T, Johnston KW, Naylor CD. Trends and variations in the use of vascular surgery in Ontario. Can J Cardiol. 1996; 12:249 –253. 18. Tu JV, Hannan EL, Anderson GM, et al. The fall and rise of carotid endarterectomy in the United States and Canada. N Engl J Med. 1998;339:1441–1447. August 1, 2002 THE AMERICAN JOURNAL OF MEDICINE威 Volume 113 163