Clinical trials to compare two treatments

JOURNAL OF ORTHODONTICS, 2017 http://dx.doi.org/10.1080/14653125.2017.1279465 STATISTICAL CORNER Clinical trials to compare two treatments Spyridon N. Papageorgiou Clinic of Orthodontics and Pediatric Dentistry, Center of Dental Medicine, University of Zurich, Zurich, Switzerland Theoretical scenario Orthodontic researchers have developed a novel way to enhance comprehensive orthodontic treatment with fixed appliances using supplemental irradiation with infrared light. The rationale is founded according to the researchers on basic biological principles and their own clinical experience with prototype appliances. In order to initiate commercial production of infrared irradiation as an adjunct to orthodontic treatment and market their own device under the name Infrabrace, they decided to test their experimental appliance with a clinical trial. The authors performed a controlled clinical trial to assess the treatment effects of Infrabrace in a parallel trial with two patient groups. Patients in the first group (henceforth, called the experimental group) were treated with fixed appliances in conjunction with a daily 30 minute use of the experimental Infrabrace appliance, while patients in the second group (henceforth, called the control group) were treated with conventional fixed appliances without Infrabrace. The authors recruited, for this study, 60 consecutive adolescent patients from the private practice of an orthodontist, who were divided into two groups of 30 patients each with similar age and sex (Table 1). Both groups were treated with the same protocol regarding fixed appliance, wire progression, treatment mechanics, and intervals between appointments. The primary outcome of the trial was overall treatment duration from appliance insertion to appliance removal in months and the secondary outcome was an assessment of occlusal outcome after treatment with the use of the Peer Assessment Rating (PAR) index (Richmond et al. 1992). Measurements were performed by a calibrated external assessor who was not involved in the treatment and was blinded to which patient belonged to which group. Data were analysed descriptively with the mean and standard deviation (SD) of each group and differences between groups were assessed with Student’s t-test for independent samples. They found that considerable differences existed in the treatment duration of the experimental group (mean = 18.5 CONTACT Spyridon N. Papageorgiou [email protected] Zurich, Plattenstrasse 11, Zurich 8032, Switzerland © 2017 British Orthodontic Society months; SD = 3.1 months) and the control group (mean = 24.9 months; SD = 4.6 months), which were statistically significant (P < 0.001; Table 1). Finally, they found no differences in final PAR score of the experimental group (mean = 3.1 points; SD = 1.4) and the control group (mean = 3.6 points; SD = 1.7), which was confirmed from the statistical analysis (P > 0.05). They concluded that Infrabrace is an effective adjunct for reducing treatment time with fixed orthodontic appliances. Based on the above trial report, which of the following statements, if any, are correct: (a) The trial robustly assessed the efficacy of Infrabrace. (b) The trial robustly assessed the efficiency of Infrabrace. (c) Trial outcomes were appropriately measured without bias. (d) Differences in the results of the two treatment groups can be attributed to Infrabrace. Answers Statement (c) is true; statements (a), (b), and (d) are false. (a) The trial robustly assessed the efficacy of Infrabrace. Efficacy in the present example would be defined as the extent to which Infrabrace produces a beneficial result under ideal conditions (Porta 2014). The authors measured the occlusal outcome of treatment with PAR scoring and found that no significant differences existed between the two groups after treatment (P > 0.05). Although it cannot be formally concluded that no difference exists in the PAR scores of the two groups (since an absence of evidence is not evidence of absence), we can be confident that the final PAR scores of the two groups seem very similar. The PAR index can be used to measure both the severity of a malocclusion and the outcome of orthodontic treatment (Richmond et al. 1992). However, no information is given about the initial PAR scores of the two groups, a fact that does not let us Clinic of Orthodontics and Pediatric Dentistry, Center of Dental Medicine, University of 2 S. N. PAPAGEORGIOU Table 1. Patient characteristics and outcomes of the given trial example. Patient characteristics Patients – n Male/female – n Age in years – mean (SD) Outcomes PAR score after treatment– mean (SD) Treatment duration in years – mean (SD) Experimental Control P value 30 17/13 13.8 (1.5) 30 14/16 14.1 (2.0) 3.1 (1.4) 3.6 (1.7) 0.187 18.5 (3.1) 24.9 (4.6) <0.001 SD, standard deviation; PAR, peer assessment rating. assess, if equally ‘difficult’ cases were included in each group. A better alternative would be to also measure the baseline PAR scores of each case prior to treatment and incorporate this in the trial, for example, by calculating the absolute or relative PAR change for each patient through treatment. We can conclude for the present trial that both the experimental and the control protocols can be used to treat patients to a similar standard (since the final PAR scores of the two groups were similar), but we cannot draw any conclusions about the efficacy of Infrabrace (since we do not know if the PAR reduction of the two groups was different). (b) The trial robustly assessed the efficiency of Infrabrace. Efficiency pertains to the effects or end results achieved in relation to the effort expended in terms of money, resources, and time (Porta 2014). In the present example, if the two groups show similar treatment effects, but the duration of treatment is drastically reduced in the Infrabrace group, it then follows that treatment efficiency is improved. Following from (a), it is clear that this statement is only partly true, since the authors did not robustly assess the efficacy of treatment with Infrabrace. It would be more appropriate to say, that if the reduction in PAR score was similar in the two groups, but the time needed to finish treatment was reduced with the use of Infrabrace, then treatment efficiency would be improved. Another alternative would be to calculate a composite outcome measurement of PAR reduction divided by duration, but this might make the trial results and their interpretation more complex. Additionally, any improvement in treatment effects or reduction in treatment duration should ideally be viewed together with any additional costs of treatment and side effects that might be associated with any adjuncts, like Infrabrace. (c) Trial outcomes were appropriately measured without bias. The authors used an external assessor to measure all trial outcomes, which minimised detection bias. It is known that measuring the results of compared interventions can be influenced by a number of factors, including an assessor’s personal preferences or present expectations from a trial that has cost plenty of time and money to the researchers. As the outcome assessor was blinded, the influence from personal preferences or expectations is minimised, while measurement errors can be appropriately handled by a priori calibration and post hoc analytic techniques (Bland and Altman 1986). (d) Differences in the results of the two treatment groups can be attributed to Infrabrace. This statement is false, since the existence of bias cannot be safely ruled out from the present trial. True baseline equivalence of experimental and the control groups prior to treatment has not been ascertained in the present trial. The outcome of orthodontic treatment might be influenced by several known or unknown factors, such as malocclusion severity, patient compliance, the genetically determined biological response of each patient, systematic diseases, smoking, and interactions with any medications taken. Ideally, a priori equivalence would be attained through random patient allocation in the experimental and control groups, which would ensure that the distribution of all known or unknown confounding factors is similar between groups. In the case of random patient allocation, any differences found between the experimental and the control group could theoretically be attributed to the supplemental use of Infrabrace. In cases where randomisation is not feasible, statistical methods that mimic randomisation like propensity score matching can also be used, although they are more complex and not without criticism (Stuart 2010). Therefore, randomisation is the best approach to ensure baseline equivalence in clinical trials of comparative effectiveness. Disclosure statement No potential conflict of interest was reported by the author. ORCID Spyridon N. Papageorgiou 3326 http://orcid.org/0000-0003-1968- References Bland JM, Altman DG. 1986. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1:307–10. Porta M. 2014. A dictionary of epidemiology. 6th ed. New York, NY: Oxford University Press. Richmond S, Shaw WC, Roberts CT, Andrews M. 1992. The PAR Index (Peer Assessment Rating): methods to determine outcome of orthodontic treatment in terms of improvement and standards. Eur J Orthod. 14:180–187. Stuart EA. 2010. Matching methods for causal inference: a review and a look forward. Stat Sci. 25:1–21. JOURNAL OF ORTHODONTICS 3 Appendix. Dataset from the given trial example. Patient Groupcode 1 1 2 1 3 1 4 1 5 0 6 1 7 1 8 1 9 1 10 0 11 0 12 1 13 1 14 0 15 1 16 1 17 0 18 1 19 1 20 1 21 1 22 0 23 0 24 0 25 0 26 1 27 0 28 0 29 0 30 1 31 0 32 1 33 0 34 0 35 1 36 0 37 1 38 0 39 0 40 0 41 0 42 1 43 0 44 0 45 1 46 0 47 0 48 1 49 0 50 1 51 1 52 1 53 1 54 0 55 0 56 0 57 1 58 0 59 0 60 0 PAR, peer assessment rating. Group Experimental Experimental Experimental Experimental Control Experimental Experimental Experimental Experimental Control Control Experimental Experimental Control Experimental Experimental Control Experimental Experimental Experimental Experimental Control Control Control Control Experimental Control Control Control Experimental Control Experimental Control Control Experimental Control Experimental Control Control Control Control Experimental Control Control Experimental Control Control Experimental Control Experimental Experimental Experimental Experimental Control Control Control Experimental Control Control Control Tx duration 18.57 15.33 16.43 20.07 23 22.1 18.47 16 15.27 19.4 27.4 24.3 16.63 28 17.63 18.47 18.73 18 16.3 15.4 16.53 26.6 22.9 29.5 29.7 16 25.57 37.23 20.63 16.33 22.73 16 25.7 25.23 16.73 20.77 15 21.93 31.03 24.2 22.17 17 25.2 26.4 18 26.77 36.5 25.56 19 20.27 23.9 19.17 17.03 22.47 24.37 23 22.07 23.57 24.47 18.57 PAR1 28 25 26 24 18 28 36 22 24 33 24 33 24 52 36 29 25 21 25 30 23 53 29 53 23 30 24 43 33 21 49 25 20 29 32 47 16 47 27 32 31 18 56 46 16 51 30 49 14 36 35 21 25 31 39 28 39 41 27 19 PAR2 8.00 3.00 2.00 4.00 2.00 3.00 2.00 3.00 3.00 2.00 4.00 2.00 2.00 2.00 2.00 2.00 4.00 3.00 2.00 3.00 4.00 11.00 2.00 2.00 4.00 2.00 3.00 2.00 5.00 2.00 2.00 2.00 3.00 3.00 6.00 9.00 2.00 2.00 5.00 2.00 3.00 3.00 7.00 4.00 4.00 19.00 4.00 3.00 4.00 2.00 5.00 2.00 4.00 2.00 6.00 2.00 4.00 11.00 6.00 6.00 Sex 1 1 0 1 0 0 0 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 0 1 0 1 1 0 0 0 1 0 1 1 1 1 0 0 0 0 0 1 0 1 1 0 1 1 1 1 0 0 1 0 1 1 1 0 0 Age 12 12 16 17 15 14 13 13 14 12 12 14 13 14 14 13 14 16 14 16 15 13 12 13 12 12 13 15 15 13 13 12 14 16 15 19 12 19 15 14 12 13 14 15 15 12 12 13 14 12 16 13 15 13 14 12 13 19 15 13