Education and debate

people in developed countries consume about 35% of their energy as fat, about 10 g/day of salt, and about 0.2 mg/day of folate, and nearly everyone is exposed to the risks of road injuries. The chief merit of the increasingly popular convenience foods is their convenience. Individuals have little influence over their composition. Even foods that are described as being healthy can be high in sugar and salt, counterbalancing any benefit from added micronutrients, such as folic acid. But discouraging the use of convenience foods is not practical; we need collective action to reduce the amounts of salt, sugar, and saturated fat in foods, and a sensible policy on portion sizes in restaurants. Of course, individuals have some choice, but for most people safety and health are minor determinantsvalue for money, convenience, and fashion rank much higher. Safety and health have to be part of the fabric of society, determined by experts with specialist knowledge, and translated into policies by governments, acting on our behalf. For years we have unquestioningly accepted the addition of iodide to table salt, to prevent thyroid disease; the prevention is silent. Charges of paternalism and pejorative labels such as nanny state, with the false innuendo that we are being controlled against our wishes, are unhelpful because effective public health needs to be integrated into the infrastructure of society. We depend on governments and professionals to ensure that our lives are as healthy and as free as possible. Governments have the main responsibility and authority for maintaining public health, through education, regulation, legislation, and taxation. Not all decisions will be right, but it is not hard to ensure that most are and, given new knowledge, to correct those that are not. An advisory role is not enough. Public health in countries such as Britain needs a stronger executive role, relatively free of short term political considerations. Watching our children (and ourselves) becoming overweight and claiming that it is all about choice is a denial of everything that public health and preventive medicine is about, and a denial of what makes a society civilised and worth belonging to. Public health and individual choice can flourish together, but the former should not be driven by the latter.
Competing interests: None declared.

Detection, verification, and quantification of adverse drug reactions

Bruno HCh Stricker, Bruce M Psaty The current system of verifying and quantifying adverse reactions to new drugs is too disparate. Epidemiological studies for testing a hypothesis have a part to play in protecting the public from the harmful effects of new drugs

Pharmacoepidemiology Unit, Department of Epidemiology and Biostatistics, Erasmus Medical Center, PO Box 1738, 3000 DR Rotterdam Bruno HCh Stricker professor of pharmacoepidemiology Cardiovascular Health Research Unit, Departments of Medicine and Epidemiology, University of Washington, Seattle, USA Bruce M Psaty professor of medicine, epidemiology, and health services Correspondence to: B HCh Stricker b.stricker@ erasmusmc.nl
BMJ 2004;329:447

Although some will question the use of the term experiment on formal grounds, most experts will likely agree that the widespread marketing of a new drug is in fact a large experiment on a population. This is especially the case when it concerns a novel molecular entity with potentially a new set of clinical experiences. As the marketing of new drugs includes the discovery of adverse effects, the publics health would be best protected by a complementary set of techniques for the detection, verification, and quantification of safety issues. Yet the current approach to this is scattered and disappointing. We discuss why healthcare professionals are not aware of all safety problems of a drug at its introduction and why pharmacoepidemiology should complement the indispensable observational method of case reporting.

adverse events with databases. We used the references from 50 of the most recent reviews, to gather the most important papers on this subject.

Sources and selection criteria

Our review is based on a search of PubMed using the terms adverse effect, adverse reaction, ADR, adverse event, adverse reaction monitoring, pharmacovigilance, cohort study, and case-control study. This yielded several thousand papers from which we excluded individual case reports and case series. From a list of cohort and case-control studies, we picked some recent examples of pharmacoepidemiological studies of

Limitations of clinical trials

Before drugs are marketed, they are extensively tested in animals and in clinical trials in humans. These tests tell much about the drugs efficacy but for several reasons relatively little about safety (box).
Systems for drug safety research and references w1-w10 are on bmj.com

44

BMJ VOLUME 329

3 JULY 2004

bmj.com

SUE SHARPLES

Education and debate

Good economical reasons preclude an endless quest for research before drugs are registered. Such research would make the development of drugs expensive, a price that ultimately is paid by the consumer.1 More effort therefore should be put into researching the safety of drugs after marketing, especially in the evidence-free zone when drugs are first launched.2 Protecting society against the adverse effects of drugs requires early detection, valid verification, and quantification, which include assessing the frequency and severity of adverse events, dose relations, the time course, and susceptibility factors.3 In daily practice many signals of a potential adverse event are not followed by a systematic process of verification and quantification. One reason for this may be the transitional period between the signal of a problem and its confirmation. At this point every sensible professional response is open to discussion. If the problem is highlighted by a single case report, attention may fade; even if similar signals are produced by other case reports, the likelihood of a causal relation is often just debated unless more evidence emerges. Subsequent well designed epidemiological studies may provide additional verification. They also fulfil the need for quantification of the adverse effect. Whether such studies are performed usually depends on the initiatives of scientific groups with an interest in the topic. Another reason why safety issues may not be pursued systematically is related to who has the responsibility. Formally, liability rests with the pharmaceutical company, which acts as marketing authorisation holder, but as it does not have an economic interest in detecting safety issues it may lack incentive to investigate problems. Additionally, many problems related to a drug are caused by class effects, such as extrapyramidal effects due to neuroleptics. Then it is not always possible to blame a particular product or company. Drug authorities should take responsibility to safeguard public health in such a situation.

Limitations of most clinical trials in highlighting a drugs safety

Homogeneous populations Most trials assess relatively healthy patients with only one disease and mostly exclude specific groups such as pregnant women, children, and elderly people Sample size Small sample size (up to 1000 patients) reduces the chance of finding rare adverse effects Limited duration Trials of short duration preclude the discovery of long term consequences such as cancer Inability to predict the real world Drug interactions can be substantial in a population as patients may take drugs concomitantly, a situation that can almost never be predicted from clinical trials

therefore remain unnoticed. One of the most common reasons for withdrawal of drugs from the market is hepatitis, which has a high background incidence making it potentially difficult to determine a causal relation.6 Voluntary reporting Despite these problems in detecting genuine adverse effects, voluntary reporting of adverse reactions by clinical observers is inexpensive and effective. Basically, there are two systems. The first is a virtual one and relates to all correspondence and short reports in the medical literature. The second consists of national and international adverse drug reaction monitoring centres. The medical literature The medical literature is probably by far the most effective system for initial detection because case reports are detailed, assessed for quality by reviewers, mostly independent from commercial incentives, and open to interested parties. The collaborative output of the 20 largest medical journals in highlighting a new problem by a single case report easily outclasses every other system. Fourteen of 18 important adverse drug reactions were shown to be detected and verified by voluntary reporting, mostly through the literature.7 Of 47 anecdotal reports published in 1963 in four major general medical journals, 35 were clearly correct, which seems to be satisfactory.8 Not all published case reports represent genuine adverse reactions as there is always the risk of false positive signals.8 Despite this, the medical literature is a highly efficient warning system for new adverse reactions, and often recognises rare events and people at high risk.9 10 Monitoring centres The second system comprises national adverse drug reaction monitoring centres and the WHO Collaborating Centre for International Drug Monitoring. These are not open to third parties such as consumers, patients, and healthcare professionals. Most of these systems work with a yellow card scheme, which can be productive if active and well qualified staff considers the detection of adverse reactions as its primary objective. Such monitoring centres are,
45

Adverse drug reactions

Detection Adverse effects cannot be detected without astute professional observers. Case reports are among the most important tools for observational research.4 All people exposed to a new drug comprise the potential catchment population for adverse effects. In a country such as the United Kingdom, with some 60 million inhabitants, a 1% cumulative exposure to a drug yearly would equate to 600 000 people using the drug at any time during that year. A rare adverse effect with an incidence of 1 in 10 000 might be detected in such a population, particularly when the adverse effect has a low background incidence, making it easily recognised. In the case of phocomelia due to thalidomide, for instance, recognition should have been easy but because of the unfamiliarity with drug safety problems at that time it took several years to identify a causal relation.5 When an adverse effect is non-specific and has an appreciable background incidence, detection is more difficult. Similarly, it may be difficult to detect an adverse effect that is indistinguishable from the disease being treatedfor example, arrhythmia as an adverse effect of antiarrhythmics. Such adverse effects may
BMJ VOLUME 329 3 JULY 2004 bmj.com

Education and debate

Databases used by pharmacoepidemiological studies to test hypotheses. Values are relative risks (95% confidence intervals) unless stated otherwise
Study Garcia Rodriguez et alw1 Jick et alw2 Database General Practice Research Database General Practice Research Database Drug Amoxicillin-clavulanic acid Appetite suppressants Adverse effect Hepatic injury Cardiac valve regurgitation Risk; comparator 6.3 (3.2 to 12.7); amoxicillin alone Risk 35/10 000 users (16.4 to 76.2) of fenfluramine or dexfenfluramine compared with 0/10 000 non-users 7.1 (2.3 to 22.0); non-use 2.8 (1.3 to 5.8); non-use 4.3 (2.4 to 7.8); non-use Risk 7.6/100 000 users (3.6 to 13.9) as against 2.1/100 000 users of oxytetracycline 3.3 (1.8 to 6.2); non-use 2.8 (1.4 to 5.7); non-use 3.0 (2.3 to 3.8); non-use 2.5 (1.04 to 6.1); non-use

Zornberg and Jickw3 Meier et alw4 Van der Linden et alw5 Derby et al
w6

General Practice Research Database General Practice Research Database General Practice Research Database General Practice Research Database Integrated Primary Care Information Project Pharmaco-morbiditeitskoppeling Tennessee Medicaid Tennessee Medicaid

Antipsychotics Postmenopausal oestrogens Fluoroquinolones Flucloxacillin

Idiopathic venous thromboembolism Systemic lupus erythematosus Achilles tendon rupture Cholestatic hepatitis

Straus et alw7 Herings et alw8 Ray et alw9 Ray et alw10

Antipsychotics Angiotensin converting enzyme inhibitors Benzodiazepines Tricyclic antidepressants (>300 mg daily)

Sudden cardiac death Hypoglycaemia Nocturnal falls Sudden cardiac death

however, often understaffed, work in isolated nonacademic environments, have all kinds of responsibilities for regulation, and are overwhelmed by reports from the pharmaceutical industry. Consequently there is little time for analysing the reports on adverse reactions sent in by doctors. The legal requirements for reporting adverse events to the Food and Drug Administration and European Medicines Evaluation Agency result in vast numbers of reports, all of which have to be processed. Many of these have a low likelihood of a causal relation. When it is likely that the literature produces really new signals in which more than 50% are causally related, most accounts of the systems that accumulate reports from industry consist of either known adverse reactions or unrelated adverse events. Considering the enormous investment in human resources in industry and government to run this system, efficiency is low. As in the end the cost will inevitably be paid by health insurance systems and patients, the price of drugs will further increase in the future. Hypothesis generating signals Over the past 30 years attempts have been made to enhance the recognition of adverse effects by data dredging or data mining. The contribution of such techniques to detecting adverse reactions has been modest. When substantial numbers of reports are available, however, comparing the proportion of reports of an adverse effect with similar drugs may provide strong hypothesis generating signals, such as for rhabdomyolysis associated with cerivastatin.11 Such comparison may even facilitate some sort of hypothesis testing relative risk assessment.12 13 The validity of such techniques, however, is probably much lower than that of formal designs of epidemiological studies. The originally embraced technique of prescription event monitoring has proved more difficult than expected.14 Use of research databases such as the General Practice Research Database for the detection of adverse effects by generating an hypothesis has had
46

only limited success. It seems better to restrict the use of such databases to hypothesis-testing epidemiological studies. Verification and quantification As soon as an adverse reaction is signalled, a hypothesis is raised that requires formal testing. Relevant questions are how often the adverse reaction occurs and how many times the risk of the adverse effect is increased by the drug. To this end the observational designs of the cohort study and the case-control study have proved useful.15 Cohort studies are particularly useful for uncommonly used drugs but are less useful if the disease being assessed is rare. Case-control studies are efficient for studying rare diseases but require that the drug is commonly used. An example of a successful case-control study was the determination of an association between vaginal carcinoma of the female offspring of women who had been treated with diethylstilbestrol during pregnancy.16 The authors were able to show the association with only eight cases and 40 controls.

Summary points
More effort should be put into researching the safety of drugs after marketing In daily practice many signals of a potential adverse event are not followed by a systematic process of verification and quantification The medical literature is probably by far the most effective system for initial detection of adverse reactions to drugs The current emphasis on the costly procedures of mandatory reporting should shift towards epidemiological studies for testing a hypothesis

BMJ VOLUME 329

3 JULY 2004

bmj.com

Education and debate

Well designed cohort or case-control studies can adequately deal with bias and confounding, the two potential criticisms of these study designs. The issue of confounding by indication is seldom a problem with rare adverse effects because such unpredictable effects are usually not associated with the indication for treatment.17 Although in such studies confounding by contraindication may play a part, it leads to a conservative estimate rather than to an overestimation of the true risk.18 The cohort and case-control designs can be used to test hypotheses in de novo field studies. Several databases facilitate the performance of such studies with prospectively gathered information on exposure to a drug and disease (see bmj.com). With these data resources, several successful pharmacoepidemiological studies have been performed (table). Unfortunately and despite the enormous growth of pharmacoepidemiology and its capabilities, most drugs are withdrawn on the bases of case reports and case series alone.6 Therefore it is time these databases are used more consistently for hypothesis testing in research concerning drug safety. In conclusion, society has the right to be safeguarded against the adverse effects of new drugs. The current emphasis on the costly procedures of mandatory reporting should perhaps shift towards epidemiological studies for testing a hypothesis. When particular drug classes and clusters of disease are involved, regulatory authorities and drug inspectorates should take the lead to fulfil their primary task of guaranteeing safe health care.
We thank J P Vandenbroucke for his critical advice in preparing this manuscript. Contributors: BHcS wrote the first draft of the article. BMP critically reviewed the manuscript. BHcS will act as guarantor for the article. Competing interests: None declared.
1 2 3 4 5 6 Henry D, Lexchin J. The pharmaceutical industry as a medicines provider. Lancet 2002;360:1590-5. Gale EAM. Lessons from the glitazones: a story of drug development. Lancet 2001;357:1870-5. Aronson JK, Ferner RE. Joining the DoTS: new approach to classifying adverse drug reactions. BMJ 2003;327:1222-5. Vandenbroucke JP. In defense of case reports and case series. Ann Intern Med 2001;134:330-4. Maio G. Zur Geschichte der Contergan-Katastrophe im Lichte der Arzneimittelgesetzgebung. Dtsch Med Wochenschr 2001;126:1183-6. Arnaiz JA, Carn X, Riba N, Codina C, Ribas J, Trilla A. The use of evidence in pharmacovigilance. Case reports as the reference source for drug withdrawals. Eur J Clin Pharmacol 2001;57:89-91. Venning GR. Identification of adverse reactions to new drugs. III: alerting processes and early warning systems. BMJ 1983;286:458-60. Venning GR. Validity of anecdotal reports of suspected adverse drug reactions: the problem of false alarms. BMJ 1982;284:249-52. Tubert P, Bgaud B, Pr JC, Haramburu F, Lellouch J. Power and weakness of spontaneous reporting: a probabilistic approach. J Clin Epidemiol 1992;45:283-6. Bgaud B, Moride Y, Tubert-Bitter P, Chaslerie A, Haramburu F. False-positives in spontaneous reporting: should we worry about them ? Br J Clin Pharmacol 1994;38:401-4. Staffa JA, Chang J, Green L. Cerivastatin and reports of fatal rhabdomyolysis. N Engl J Med 2002;346:539-40. Stricker BHCh, Tijssen JP. Serum sickness-like reactions to cefaclor. J Clin Epidemiol 1992;45:1177-84. Egberts AC, Meyboom RHB, van Puijenbroek EP. Use of measures of disproportionality in pharmacovigilance: three Dutch examples. Drug Safety 2002;25:453-8. Kubota K, Kubota N, Pearce GL, Inman WH. ACE-inhibitor-induced cough, an adverse drug reaction unrecognised for several years: studies in prescription-event monitoring. Eur J Clin Pharmacol 1996;49:431-7. Ray WA. Population-based studies of adverse drug effects. N Engl J Med 2003;349:1592-4. Herbst AL, Ulfelder H, Poskanzer DC. Adenocarcinoma of the vagina. Association of maternal stilbestrol therapy with tumor appearance in young women. N Engl J Med 1971;284:878-81. Miettinen OS. The need for randomization in the study of intended effects. Stat Med 1983;2:267-71. Feenstra J, Grobbee DE, in t Veld BA, Stricker BHCh. Confounding by contraindication in a nationwide cohort study of risk for death in patients taking ibopamine. Ann Intern Med 2001;134:569-72.

7 8 9

10

11 12 13

14

15 16

17 18

(Accepted 30 May 2004)

Making decisions about benefits and harms of medicines

Trisha Greenhalgh, Olga Kostopoulou, Clare Harries Even when good scientific data are available, peoples interpretation of risks and benefits will differ
Department of Primary Care and Population Sciences, University College London, London N19 5LW Trisha Greenhalgh professor of primary health care Department of Primary Care and General Practice, University of Birmingham, Birmingham Olga Kostopoulou Department of Health R&D/PPP national primary care postdoctoral fellow Department of Psychology, University College London, London Clare Harries lecturer Correspondence to: T Greenhalgh p.greenhalgh@ pcps.ucl.ac.uk
BMJ 2004;329:4750

Drug regulatory authorities, such as the Medicines and Healthcare Products Regulatory Agency in the United Kingdom and the Food and Drug Administration in the United States, award product licences by assessing the balance between benefit and harm. The decision to revoke a licence generally hangs on evidence of lack of efficacy or risk of serious adverse effects, taking account of the seriousness of the condition and the range of other treatments available. The authorities work at the level of the whole population. But individual patients may believe (rightly in some cases) that a particular regulatory decision is not in their own best interests, and vociferous campaigns sometimes result (box 1). Involvement of patients can be a powerful driver for improving services.5 But both lay people and professionals are susceptible to several biases when making health related decisions (box 2). What can be done to ensure that the care of individual patients is not compromised by regulatory decisions intended to protect the
BMJ VOLUME 329 3 JULY 2004 bmj.com

population as a whole, and to encourage objective and dispassionate decision making in the face of cognitive biases?

Sources and selection criteria

This article was constructed through multidisciplinary dialogue between an academic general practitioner with a keen interest in evidence based and narrative based decision making, two cognitive psychologists specialising in risk perception, and an editor with a background in medical pharmacology. The authors drew on their own disciplinary perspective, expertise, and archives. The goal was not to produce an exhaustive overview of any of our areas of expertise but to use insights from one discipline (psychology) to illuminate findings from another (drug regulatory decisions).
Two more boxes and further references (w1-w17) are available on bmj.com

47