Optimizing Cutoff Scores for the Barthel Index and the

Modified Rankin Scale for Defining Outcome in Acute

Stroke Trials
Maarten Uyttenboogaart, MD; Roy E. Stewart, MSc; Patrick C.A.J. Vroomen, MD, PhD;
Jacques De Keyser, MD, PhD; Gert-Jan Luijckx, MD, PhD

Background and Purpose—There is little agreement on how to assess outcome in acute stroke trials. Cutoff scores for the
Barthel Index (BI) and modified Rankin Scale (mRS) are frequently arbitrarily chosen to dichotomize favorable and
unfavorable outcome. We investigated sensitivity and specificity of BI cutoff scores in relation to the mRS to obtain the
optimal corresponding BI and mRS scores.
Methods—BI and mRS scores were collected from 1034 ischemic stroke patients. Sensitivity and specificity were
calculated for BI cutoff scores from 45 to 100 in mRS score 1, 2, and 3 and were plotted in receiver operator
characteristic (ROC) curves.
Downloaded from http://stroke.ahajournals.org/ by guest on February 23, 2017

Results—The cutoff scores for the BI with the highest sum of sensitivity and specificity were 95 (sensitivity 85.6%;
specificity 91.7%), 90 (sensitivity 90.7%; specificity 88.1%), and 75 (sensitivity 95.7%; specificity, 88.5%) for,
respectively, mRS 1, 2, and 3. The area under the ROC curve was 0.933 in mRS 1, 0.960 in mRS 2, and 0.979 in
mRS 3.
Conclusions—The optimal cutoff scores for the BI were 95 for mRS 1, 90 for mRS 2, and 75 for mRS 3. For future acute
stroke trials that assess stroke outcome with the BI and mRS, we recommend the use of these BI cutoff score(s) with
the corresponding mRS cutoff score(s), to ensure the use of consistent and uniform end points. (Stroke. 2005;36:
1984-1987.)
Key Words: disability evaluation 䡲 outcome assessment 䡲 stroke

S everal randomized controlled acute stroke trials have

been designed to investigate effectiveness of therapeutic
interventions. A major point of discussion is how to define
Items regarding moving from wheelchair to bed and
walking on level surface are scored 0, 5, 10, or 15. The
total BI is a cumulative score of the 10 items, with a
outcome in acute stroke trials with disability and handicap maximum score of 100 corresponding with complete
scales.1– 6 The most widely used scales are the modified independence, and a minimum score of 0 corresponding
Rankin Scale (mRS) and the Barthel Index (BI). with total dependence.
The mRS has proved to be valid and reliable for defining There is little consensus on the optimal implementation of
outcome in stroke patients.7,8 Although the mRS was the BI and mRS as outcome measure in acute stroke trials. It
designed as a handicap scale,9 it should be considered a is unclear which outcome scale is preferable. Moreover, the
disability scale.10 The mRS defines 6 different grades cutoff scores distinguishing favorable and unfavorable out-
of disability, from 0 for “no symptoms at all” to 5 for “severe come are highly variable between various acute stroke trials.4
disability or bedridden, incontinent, and requiring constant This issue has great consequences for the design and inter-
nursing care and attention,” and grade 6 for death. pretation of acute stroke trials. The BI has a larger score range
The BI has also shown to be valid and reliable for and therefore more possible cutoff scores compared with the
assessing disability in stroke patients.8,11 It contains 10 mRS. Less is known which BI scores are corresponding with
items with varying weights that score activities of daily the different mRS scores. There have been only a few studies
living (ADL). The items bathing and grooming are scored that determined pivotal BI cutoff scores, and none of them
0 or 5; the items feeding, dressing, controlling bladder, were related to the mRS.12–14 In this article, we investigated
controlling bowel, getting onto and off the toilet, and which cutoff scores on the BI corresponded to mRS grades 1,
ascending and descending stairs are scored 0, 5, or 10. 2, and 3.

Received April 13, 2005; final revision received May 27, 2005; accepted June 20, 2005.
From the Departments of Neurology (M.U., P.C.A.J.V., J.D.K., G.J.L.) and Health Sciences (R.E.S.), University Medical Center Groningen, The
Netherlands.
Correspondence to Maarten Uyttenboogaart, MD, Department of Neurology, University Medical Center Groningen, PO Box 30.001, 9700 RB
Groningen, The Netherlands. E-mail [email protected]
© 2005 American Heart Association, Inc.
Stroke is available at http://www.strokeaha.org DOI: 10.1161/01.STR.0000177872.87960.61

1984
 Uyttenboogaart et al Defining Outcome in Acute Stroke Trials 1985

TABLE 1. Calculation of Sensitivity and Specificity for BI false unfavorable outcome rates were considered to be equally
Cutoff Scores in mRS 1, 2, and 3 important. This corresponds with the BI score that has the highest
sum of sensitivity and specificity.17 To investigate the relationship
mRS Reflecting mRS Reflecting between sensitivity and specificity, receiver operator characteristic
Favorable Outcome Unfavorable Outcome (ROC) curves were obtained and the areas under the curve (AUCs)
MRS ⱕ1, 2, or 3 MRS ⬎1, 2, or 3 were calculated. ROC curves plot sensitivity versus 1-specificity and
BI reflecting favorable A B visualize the optimal cutoff scores for the BI in each mRS grade. The
outcome (BI ⱖ45–100) AUC indicates the discriminative properties between favorable and
unfavorable outcome for the BI cutoff scores in the 3 mRS scores.
BI reflecting unfavorable C D
outcome (BI ⬍45–100)
A, indicates true favorable outcome; B, false favorable outcome; C, false
Results
unfavorable outcome; D, true unfavorable outcome. Population Characteristics
Sensitivity, D/(D⫹B); specificity, A/(A⫹C). From the 1034 patients, 547 (52.9%) were female. The mean
age was 69.1 years (SD 12.8 years). Median BI score was
Subjects and Methods 80 with an interquartile range from 40 to 100. The mRS score
Population and Data Collection distribution was mRS 0, 9.1%; mRS 1, 17.8%; mRS 2,
Data were obtained from the United States and Canadian Lubeluzole 13.1%; mRS 3, 19.1%; mRS 4, 29.7%; and mRS 5, 11.2%.
Ischemic Stroke Study (INT-LUB-9) and the European and Austra-
lian Lubeluzole Ischemic Stroke Study (INT-LUB-5),15,16 provided
Downloaded from http://stroke.ahajournals.org/ by guest on February 23, 2017

Sensitivity and Specificity of BI and mRS

by the Janssen Research Foundation (Beerse, Belgium). These trials
have been published respectively in 1997 and 1998. In summary, Cutoff Scores
these trials studied the neuroprotective effect of lubeluzole in acute The sensitivity and specificity for the cutoff scores of the BI
ischemic stroke. In both trials, there was no significant difference in in relation to mRS 1, 2, and 3 were calculated (Table 2) and
mortality rate (primary end point) between lubeluzole-treated pa- plotted in ROC curves (Figure).
tients and placebo-treated patients.
For mRS 1, the optimal cutoff score on the BI was 95, with
The INT-LUB-5 study included 725 stroke patients (675 ischemic
and 50 hemorrhagic), and the INT-LUB-9 included 721 patients (700 a sensitivity of 85.6% (95% CI, 82.9% to 87.9%) and a
ischemic stroke and 21 nonischemic stroke or other causes). BI and specificity of 91.7% (95% CI, 87.8% to 94.5%). For mRS 2,
mRS scores from ischemic stroke patients at 12 weeks after stroke the BI score with the highest sum of sensitivity and specific-
onset were analyzed. Dead patients were excluded because our ity was 90, with a sensitivity of 90.7% (95% CI, 88.1% to
analysis focused on disability scores of the BI and mRS. We did not
make a distinction between lubeluzole-treated and placebo-treated 92.7%) and a specificity of 88.1% (95% CI, 84.6% to 90.9%).
patients because the intention was only to study the relationship An mRS 3 agreed most with a BI score of 75, with a sensitivity
between BI and mRS scores. At 12 weeks, 519 corresponding BI and of 95.7% (95% CI, 93.3% to 97.5%) and a specificity of 88.5%
mRS scores were present in INT-LUB-9 and 515 in INT-LUB-5, (95% CI, 85.8% to 90.8%). In all 3 mRS cutoff scores,
forming a total of 1034 BI and mRS scores.
sensitivity (rate true unfavorable outcome) increased and
Analysis Methods specificity (rate true favorable outcome) decreased when BI
Outcome was dichotomized into favorable and unfavorable using 3 scores increased.
different mRS scores to obtain the corresponding BI score for each Subsequently, AUCs were calculated. The AUC for the BI
mRS score. An mRS score ⱕ1, 2, or 3 reflected favorable outcome, cutoff scores was 0.932 (95% CI, 0.916 to 0.949) in mRS 1,
and an mRS score ⬎1, 2, or 3 reflected unfavorable outcome. The BI 0.960 (95% CI, 0.949 to 0.971) in mRS 2, and 0.979 (95% CI,
cutoff scores were defined as BI ⱖ45 to 100 for favorable outcome
and as BI ⬍45 to 100 for unfavorable outcome. Sensitivity was 0.972 to 0.985) in mRS 3.
expressed as the rate of unfavorable outcome according to the mRS
and BI. Specificity was expressed as the rate of favorable outcome Discussion
according to the BI and mRS (Table 1).
The maximal distinction between favorable and unfavorable
In this study, we analyzed the optimal cutoff scores for the BI
outcome defined by the mRS is reached when the sensitivity and and the mRS. These were found to be BI 95 for mRS 1, BI 90
specificity of a BI score are maximal because false favorable and for mRS 2, and BI 75 for mRS 3. This finding may have

TABLE 2. Sensitivity, Specificity, and Sum Score for BI Cutoff Scores in mRS 1, 2, and 3
BI score 100 95 90 85 80 75 70 65 60 55 50 45
mRS 1 Sensitivity 0.927 0.856 0.786 0.706 0.660 0.620 0.573 0.529 0.480 0.433 0.390 0.349
Specificity 0.813 0.917 0.935 0.960 0.971 0.978 0.993 0.993 0.996 0.996 0.996 0.996
Sum 1.740 1.773* 1.721 1.666 1.631 1.598 1.566 1.522 1.476 1.429 1.386 1.345
mRS 2 Sensitivity 0.981 0.952 0.907 0.837 0.794 0.750 0.696 0.643 0.585 0.527 0.475 0.425
Specificity 0.651 0.809 0.881 0.939 0.966 0.978 0.993 0.993 0.998 0.998 0.998 0.998
Sum 1.632 1.761 1.788† 1.776 1.760 1.728 1.689 1.636 1.583 1.525 1.473 1.423
mRS 3 Sensitivity 1.000 1.000 1.000 0.991 0.976 0.957 0.917 0.875 0.813 0.745 0.676 0.619
Specificity 0.460 0.596 0.691 0.794 0.846 0.885 0.923 0.948 0.967 0.979 0.984 0.995
Sum 1.460 1.596 1.691 1.785 1.822 1.842‡ 1.840 1.823 1.780 1.724 1.660 1.614
*Maximum sum of sensitivity and specificity in mRS 1; †maximum sum of sensitivity and specificity in mRS 2; ‡maximum sum of sensitivity and specificity in mRS 3.
 1986 Stroke September 2005

cutoff scores could be suboptimal. The sensitivity (75.0%)

and specificity (97.8%) of these cutoff scores implicates that
25% percent of the patients would have a favorable outcome
according to the BI but an unfavorable outcome according to
the mRS. With regard to the specificity, 2.2% would have an
unfavorable outcome according to the BI but a favorable
outcome according to the mRS. By choosing a BI cutoff score
of ⱖ90 (sensitivity 90.7%; specificity 88.1%), the false
favorable outcome rate could be reduced to 9.3%, whereas
the false unfavorable outcome rate would increase to 11.9%.
Minimizing false favorable and false unfavorable outcome
rates could decrease unnecessary heterogeneity of outcome in
acute stroke trials.
Compared with our results, Celani et al found that BI ⬎90
(sensitivity 98%; specificity 97%) was a pivotal score for
which patients did not require help from another person for
everyday activities.14 Kay et al concluded that BI ⱕ80
(sensitivity 94%; specificity 80%) was the optimal cutoff
Downloaded from http://stroke.ahajournals.org/ by guest on February 23, 2017

score for self-reported dependency.12 These cutoff scores

differed with those of our study when dependency is consid-
ered to be mRS ⬎2, for which the optimal BI score was ⬍90.
These differences may be explained by the subjectivity of
self-reported dependency, which will be influenced by per-
sonal circumstances such as socioeconomic status and psy-
chological factors.
The mRS cutoff scores were used as a reference to
distinguish favorable from unfavorable outcome. Although
this is actually not a “gold standard” for dichotomizing
outcome, we think that the mRS is suitable for this purpose.
First, the mRS is a clinically relevant scale, with 6 different
easily understandable and well-defined grades. Second, the
BI is highly correlated with the mRS;19 therefore, we can
compare BI cutoff scores with the mRS. Third, the mRS
measures global disability, whereas the BI scores only ADL.
A point of criticism is that there is only a 5-point difference
between the optimal BI cutoff scores in mRS 1 and mRS 2.
These BI scores are near the maximum score of the BI. This
can be explained by the frequently observed ceiling effects of
the BI.5,6,20 Weimar et al concluded that because of the ceiling
effect, the mRS is preferable to the BI for defining outcome.5
Kwon et al showed that there was no significant difference in
BI scores between mRS 0, mRS 1, and mRS 2 because of
ceiling effects of the BI.19
If the intention of a therapeutic intervention is to obtain
excellent recovery after stroke, which could be defined as
mRS ⱕ1, the corresponding BI cutoff score was ⱖ95,
according to our results. There is consensus that mRS ⱕ2
reflects independence and mRS ⬎2 implicates dependence.21
Our study showed that a BI score ⱖ90 is the optimal cutoff
score in relation to mRS ⱕ2. In severe strokes, one could
decide to choose mRS ⱕ3 and BI ⱖ75 as cut-off scores for
favorable outcome. An example of stroke severity–related
outcome has been mentioned by Adams et al.22 They used the
ROC curves for BI cutoff scores in mRS 1, 2, and 3. mRS as primary end point, where mRS cutoff scores 0, ⱕ1,
or ⱕ2 reflected favorable outcome, depending on the baseline
consequences for the definition of outcome in acute stroke National Institutes of Health Stroke Scale (NIHSS) score.
trials. In conclusion, we determined the optimal corresponding BI
A recent acute stroke trial defined favorable outcome with and mRS cutoff scores: BI 95 for mRS 1, BI 90 for mRS 2,
an mRS ⱕ2 and BI ⱖ75.18 According to our results, these and BI 75 for mRS 3. We recommend the use of this/these BI
 Uyttenboogaart et al Defining Outcome in Acute Stroke Trials 1987

cutoff score(s) with the corresponding mRS score(s) for 11. Mahoney F, Barthel D. Functional evaluation: the Barthel Index. Md State
future acute stroke trials in which BI and mRS scores Med J. 1965;14:56 – 61.
12. Kay R, Wong KS, Perez G, Woo J. Dichotomizing stroke outcomes based
dichotomize favorable and unfavorable outcome. on self-reported dependency. Neurology. 1997;49:1694 –1696.
13. Granger C, Dewis LS, Peters NC, Sherwood CC, Barrett JE. Stroke
Acknowledgments rehabilitation: analysis of repeated Barthel Index measures. Arch Phys
This study was supported by a grant from the Catharina Heerdt Med Rehabil. 1979;60:14 –17.
Foundation. 14. Celani M, Cantisani T, Righetti E, Spizzichino L, Ricci S. Different
measures for assessing stroke outcome: an analysis from the international
stroke trial in Italy. Stroke. 2002;33:218 –223.
References 15. Diener H. Multinational randomized controlled trial of lubeluzole in acute
1. Duncan PW, Min Lai S, Keighley J. Defining post-stroke recovery: ischemic stroke. Cerebrovasc Dis. 1998;8:172–181.
implications for design and interpretation of drug trials. Neuropharma- 16. Grotta J. Lubeluzole treatment of acute ischemic stroke. The US and
cology. 2000;39:835– 841. Canadian Lubeluzole Ischemic Stroke Study Group. Stroke. 1997;28:
2. Berge E, Barer D. Could stroke trials be missing important treatment
2338 –2346.
effects? Cerebrovasc Dis. 2002;13:73–75.
17. Connell FA, Koepsell TD. Measures of gain in certainty from a diagnostic
3. Duncan P, Jorgensen HS, Wade DT. Outcome measures in acute stroke
test. Am J Epidemiol. 1985;121:744 –753.
trials: a systematic review and some recommendations to improve
18. Hacke W, Albers G, Al Rawi Y, Bogousslavsky J, Davalos A, Eliasziw
practice. Stroke. 2000;31:1429 –1438.
M, Fischer M, Furlan A, Kaste M, Lees KR, Soehngen M, Warach S; for
4. Sulter G, Steen C, de Keyser J. Use of the Barthel Index and modified
the DIAS Study Group. The desmoteplase in acute ischemic stroke trial
Rankin Scale in acute stroke trials. Stroke. 1999;30:1538 –1541.
5. Weimar C, Kurth T, Kraywinkel K, Wagner M, Busse O, Haberl R, (DIAS): a phase II MRI-based 9-hour window acute stroke thrombolysis
Diener H; for the German Stroke Data Bank Collaborators. Assessment of trial with intravenous desmoteplase. Stroke. 2005;36:66 –73.
Downloaded from http://stroke.ahajournals.org/ by guest on February 23, 2017

functioning and disability after ischemic stroke. Stroke. 2002;33: 19. Kwon S, Hartzema A, Min Lai S, Duncan P. Disability measures in
2053–2059. stroke: relationship among the Barthel Index, the Functional Inde-
6. Young F, Lees K, Weir C. Strengthening acute stroke trials through pendence Measure, and the modified Rankin Scale. Stroke. 2004;35:
optimal use of disability end points. Stroke. 2003;34:2676 –2680. 918 –923.
7. van Swieten J, Koudstaal P, Visser M, Schouten H, van Gijn J. Interob- 20. Wellwood I, Dennis MS, Warlow CP. A comparison of the Barthel Index
server agreement for the assessment of handicap in stroke patients. and the OPCS disability instrument used to measure outcome after acute
Stroke. 1988;19:604 – 607. stroke. Age Ageing. 1995;24:54 –57.
8. D’Olhaberriague L, Litvan I, Mitsias P, Mansbach HH. A reappraisal of 21. Warlow C, Dennis M, van Gijn J, Hankey G, Sandercock P, Bamford J,
reliability and validity studies in stroke. Stroke. 1996;27:2331–2336. Wardlaw J. The organization of stroke services: outcome. In: Stroke: A
9. Rankin J. Cerebral vascular accidents in patients over the age of 60. II. Practical Guide to Management. Malden, Mass: Blackwell Sciences;
Prognosis. Scott Med J. 1957;2:200 –215. 1996:746 –753.
10. de Haan R, Limburg M, Bossuyt P, van der Meulen J, Aaronson N. The 22. Adams H, Leclerc J, Bluhmki E, Clarke W, Hansen M, Hacke W.
clinical meaning of Rankin “handicap” grades after stroke. Stroke. 1995; Measuring outcomes as a function of baseline severity of ischemic stroke.
26:2027–2030. Cerebrovasc Dis. 2004;18:124 –129.
 Optimizing Cutoff Scores for the Barthel Index and the Modified Rankin Scale for
Defining Outcome in Acute Stroke Trials
Maarten Uyttenboogaart, Roy E. Stewart, Patrick C.A.J. Vroomen, Jacques De Keyser and
Gert-Jan Luijckx
Downloaded from http://stroke.ahajournals.org/ by guest on February 23, 2017

Stroke. 2005;36:1984-1987; originally published online August 4, 2005;

doi: 10.1161/01.STR.0000177872.87960.61
Stroke is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231
Copyright © 2005 American Heart Association, Inc. All rights reserved.
Print ISSN: 0039-2499. Online ISSN: 1524-4628

The online version of this article, along with updated information and services, is located on the
World Wide Web at:
http://stroke.ahajournals.org/content/36/9/1984

Permissions: Requests for permissions to reproduce figures, tables, or portions of articles originally published
in Stroke can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Office.
Once the online version of the published article for which permission is being requested is located, click
Request Permissions in the middle column of the Web page under Services. Further information about this
process is available in the Permissions and Rights Question and Answer document.

Reprints: Information about reprints can be found online at:

http://www.lww.com/reprints

Subscriptions: Information about subscribing to Stroke is online at:

http://stroke.ahajournals.org//subscriptions/