Magee 2016 Matadoc
Magee 2016 Matadoc
Magee 2016 Matadoc
Introduction
Disorders of Consciousness (DOC) describes a continuum
of acquired conditions that stem from acquired profound brain
injury. Three primary conditions can be categorized under DOC.
Coma represents a state of unarousable unresponsiveness in which
there is no evidence of self or environmental awareness (Plum &
Posner, 1983). It is usually a temporary phase that progresses to
some level of consciousness. Vegetative State (VS) is character-
ized by spontaneous arousal and sleep-wake cycles but with no
evidence of awareness of self or environment, no interaction with
others, and no sustained, reproducible, purposeful, or voluntary
behavioral responses to sensory stimuli (Jennett & Plum, 1972).
Some nerve and spinal reflex responses are preserved, present-
ing as spontaneous, non-purposeful movements that are not goal
directed. Progression from VS leads to Minimally Conscious State
Vol. 53, No. 1 3
The Rasch analysis indicated that all five items of the Principal
Subscale demonstrated good fit to the Rasch model for the individ-
ual items and the overall model fit. Both principal component and
Rasch analyses of the Principal Subscale, therefore, demonstrate
a robust unidimensional and homogeneous subscale for assessing
awareness in patients with PDOC. The final criterion, that a scale
should predict outcome, remains unknown at the current time.
Method
Design
We employed a prospective study using repeated measures to
test the reliability of subscales two and three and the internal con-
sistency of the second subscale.
Participants
A convenience sample was recruited from a specialist unit
for adults with PDOC who were admitted to the facility for the
purposes of gaining an accurate diagnosis through interdisci-
plinary assessment. Participants were required to have no con-
firmed diagnosis of VS, MCS, or Emerging at the time of recruit-
ment. Participants were required to be medically stable and be
aged between 16 and 70 years of age. Participants with known
pre-morbid hearing impairments, a previous diagnosis of musi-
cogenic epilepsy, or who emerged from DOC during recruitment
or assessment were excluded. Mental capacity assessments were
completed before recruitment, and standard procedures regard-
ing the recruitment of patients lacking capacity were followed.
Ethics was gained from the UK National Research Ethics Service
(05/Q0406/185). All participants received standard interdiscipli-
nary rehabilitation assessment and intervention during the study.
Procedures to Minimize Bias
All the assessors in this study were credentialed music therapists
who were trained in using the MATADOC. Seven assessors in total
were involved in data collection, whose experience with PDOC
populations ranged from 0.25 to 19 years (mean = 3.5 years). The
two assessors involved with each individual participant remained
consistent across that individual’s entire MATADOC assessment
(four live clinical contacts and four video ratings). Assessors were
blinded to each other’s ratings for interrater analysis. An adequate
period of time (between four and 24 weeks) elapsed between the
live session and the video observation to minimize the assessor’s
memory of the previous session. Furthermore, viewing order of
the clinical contacts was randomized, providing further interfer-
ence with any memory of the live MATADOC ratings. All data were
anonymized and entered into an EXCEL spreadsheet by a data
manager who was independent of this project. Later, data were
imported into SPSS for statistical analysis.
Analysis
Interrater and test-retest reliability were assessed using the
intraclass correlation coefficient with random effects, commonly
referred to as ICC(2), to account for there being multiple raters
(n = 7). Interrater reliability analysis used only live assessment ses-
sions but included all repeated contacts (n = 4 each), which were
assumed to be independent. This resulted in a sample size of 168
(21 x 4 x 2) observations for interrater analysis. A similar type of
assumption was made for test-retest. Test-retest reliability analysis
compared ratings of live and video observations of the same con-
tact undertaken by the same assessor (168 ratings: 21 x 4 x 2). The
intra-patient variation across multiple contacts in a PDOC popula-
tion is sufficient to support the use of this technique (see Figure 1).
The two items contained within the second subscale (MATADOC
items 6 and 7) were examined independently for internal consist-
ency. This was necessary, given that these items were structured and
scored differently to the items in the Principal and third subscales
using binary rating. This subscale warranted special examination,
given its primary utility to inform music-specific intervention plan-
ning and the arbitrary nature of its numerical scoring: “no” was
scored 1, and “yes” was scored 2. Internal consistency of scores
14 Journal of Music Therapy
Figure 1.
Matrix of live and video ratings to achieve inter-rater and test-retest ratings.
Results
Sample Characteristics
Over a 36-month period, twenty-one participants (11 male;
10 female) were recruited with a mean age of 40.3 years
(range = 19–67; SD = 15.65). All participants had profound brain
Vol. 53, No. 1 15
Reliability
We report in detail here on subscales two and three (items 6–14)
of the MATADOC, as IRR and TRT for the Principal Subscale
have already been reported (Magee et al., 2014). Calculated intra-
class correlation coefficients (ICCs) for items from subscales two
and three of the MATADOC produced mixed results. ICCs for
Table 1
Patient Cohort Recruited to the Study
Patient number Gender Age Etiology of brain damage Time since onset (months)
1 M 21 Traumatic 7
2 M 25 Traumatic 8
3 M 60 Hypoxic/ischaemic 5
4 M 23 Traumatic 10
5 M 19 Hypoxic/ischaemic 9
6 F 42 Hypoxic/ischaemic 8
7 F 19 Traumatic 13
8 M 48 Haemorrhagic 5
9 F 27 Viral 15
10 F 37 Traumatic 6
11 F 35 Hypoxic/ischaemic 6
12 F 47 Haemorrhagic 8
13 F 59 Traumatic 5
14 M 36 Hypoxic/ischaemic 6
15 M 65 Traumatic 16
16 F 58 Viral 10
17 M 44 Traumatic 7
18 M 45 Hypoxic/ischaemic 5
19 M 23 Traumatic 6
20 F 67 Hypoxic/ischaemic 5
21 F 46 Haemorrhagic 5
16 Journal of Music Therapy
IRR ranged from 0.32 to 0.74 (mean = 0.48) for seven of the nine
items, with a mean ICC of 0.41 for the two subscales overall. Item
7: Musical Response and Item 10: Choice-Making had much lower
ICCs, 0.14 and 0.17, respectively. Using the criteria proposed by
Andresen (2000), Items 8–9 and 11–12 had adequate IRR but
Items 7, 10, and 14 had poor IRR, with items 6 and 13 also falling
just inside the threshold for poor IRR (0.39).
Test-retest reliability for most of the items fell within the accept-
able range (range = 0.55–0.69, mean = 0.61) with exception of
Item 7: Musical Response (0.26) (see Table 2). The results indicate
acceptable TRR for Items 6 and 8–14. However, Item 7: Musical
Response has poor TRR.
Internal Consistency
Items 6 and 7 were examined independently for internal con-
sistency, given that they each have nonhierarchical sub-items
with binary rating. They were examined as separate rather than
combined items, as they do not examine the same construct.
Cronbach’s alpha for Item 6: Behavioural Response (with six sub-
items) was 0.51; for Item 7: Music Response (seven sub-item), it
was 0.28.These results indicate poor internal consistency for Items
6 and 7 (See Table 2).
Discussion
This study aimed to examine the reliability of the two subscales of
the MATADOC, “Subscale Two: Musical Parameter and Behavioural
Response Type” and “Subscale Three: Clinical Information to
Inform Goal Setting and Clinical Care.” These subscales were
examined separately, as they were believed to hold greater clini-
cal utility rather than psychometric strength. This contrasts with
the MATADOC Principal Subscale, which holds diagnostic utility
and so was tested for its psychometric properties separately. The
three subscales have combined strength as a comprehensive meas-
ure that can rate responsiveness to music-based auditory stimuli
in PDOC populations. Thus, this study aimed to provide a clearer
picture of the MATADOC’s overall psychometrics.
Testing for IRR and TRT for subscales two and three of the
MATADOC entire measure revealed mixed results. Four of the items
had adequate reliability, with a further two reaching just below the
Vol. 53, No. 1
Table 2
Descriptive Statistics (N = number of pairs of observations, M = mean score, SD = standard deviation), Internal Consistency (Cronbach’s α) of Derived
Scores, and Interrater and Test-Retest Reliability (estimated using intraclass correlation coefficients, ICC) for MATADOC Subscales 2 and 3
Score Q6, items summed 82 1.40 1.33 0.51 (0.37, 0.65) 0.39 (0.17, 0.54) 0.62 (0.52, 0.70)
Score Q7, items summed 84 0.36 0.67 0.28 (0.10, 0.46) 0.14 (–0.08, 0.34) 0.26 (0.11, 0.39)
Vocalization 84 –0.01 0.83 0.74 (0.63, 0.82) 0.60 (0.49, 0.69)
Nonverbal communication 84 0.37 0.75 0.63 (0.48, 0.74) 0.63 (0.53, 0.72)
Choice-making 84 0.08 0.32 0.17 (–0.04, 0.37) 0.58 (0.47, 0.67)
Motor skills 84 0.58 0.86 0.47 (0.25, 0.59) 0.61 (0.50, 0.69)
Attention to task 84 0.45 0.57 0.43 (0.21, 0.57) 0.61 (0.50, 0.69)
Intentional behaviour 84 0.52 0.83 0.39 (0.18, 0.55) 0.69 (0.60, 0.76)
Emotional response 84 0.57 1.02 0.32 (0.12, 0.50) 0.55 (0.43, 0.64)
17
Table 3
18
Third subscale: Clinical information to inform goal setting and clinical care
8: Vocalization Documents the patient’s vocal activity (including reflexive Item ratings are converted to scores that
behaviors and cognitively mediated responses). are categorized as Absent/
9: Nonverbal Rates observed responses pertaining to nonverbal social VS, Developing/MCS, or Strength/
communication communication responses. Emerging responses.
10: Choice-making Rates the ability to discriminate between a forced choice of two Holds no diagnostic value but
alternatives presented using the communication strategy used a) supports the diagnosis formed from
by the treatment team. the Principal Subscale;
11: Motor skills Rates observed physical responses to musical objects, sounds, and b) assists with uni/interdisciplinary goal
music presented. setting, and
12: Attention to task Rates volitional behaviors denoting attention (focused or c) assists with monitoring patient
sustained) during procedures involving musical stimuli, be they responsiveness against the goals that
auditory (e.g., musical sounds) or combined auditory/ have been set.
visual.
13: Intentional behaviour Rates observed goal-oriented behaviors.
14: Emotional response Rates emotionally expressive behaviors that are contingent
to musical stimuli including facial gestures, tears, and
vocalizations.
19
20 Journal of Music Therapy
Conclusion
Overall, the results need to be considered in light of this measure
being a work in progress. The entire MATADOC offers clinicians a
useful measure for documenting behaviors in a consistent manner.
It demonstrates sensitivity through its capacity to rate behaviors in
small enough increments to reflect change, a necessary function
for evaluating responsiveness in minimally responsive populations.
Vol. 53, No. 1 23
Funding
Funding for this project was made possible by the Neuro-Disability
Research Trust, Royal Hospital for Neuro-Disability, London, and
the Society for Education, Music, and Psychology Research, UK.
Acknowledgements
The authors would like to acknowledge the following at the
Royal Hospital for Neuro-Disability, London: the Departments of
Music Therapy, Speech and Language Therapy, and Occupational
Therapy, which contributed to the data collection; the Research
Department; Eirini Alexiou, Senior Music Therapist, who made a
significant contribution to data collection; and Maura Quinn, who
provided data management.
References
Andresen, E. M. (2000). Criteria for assessing the tools of disability outcomes
research. Archives of Physical Medicine and Rehabilitation, 81, S15–S20.
Boyle, M. E., & Greer, R. D. (1983). Operant procedures and the comatose patient.
Journal of Applied Behavior Analysis, 16(1), 3–12.
24 Journal of Music Therapy
Bruno, M., Vanhaudenhuyse, A., Thibaut, A., Moonen, G., & Laureys, S. (2011).
From unresponsive wakefulness to minimally conscious PLUS and func-
tional locked-in syndromes: Recent advances in our understanding of disor-
ders of consciousness. Journal of Neurology, 258(7), 1373–1384. doi:10.1007/
s00415-011-6114-x
Coleman, M. R., Davis, M. H., Rodd, J. M., Robson, T., Ali, A., Owen, A. M., &
Pickard, J. D. (2009). Towards the routine use of brain imaging to aid the clini-
cal diagnosis of disorders of consciousness. Brain: A Journal of Neurology, 132,
2541–2552. doi:10.1093/brain/awp183
Formisano, R., Vinicola, V., Penta, F., Matteis, M., Brunelli, S., & Weckel, J. W. (2001).
Active music therapy in the rehabilitation of severe brain injured patients dur-
ing coma recovery. Annali Dell’Istituto Superiore Di Sanità, 37(4), 627–630.
Giacino, J. T., Ashwal, S., Childs, N., Cranford, R., Jennett, B., Katz, D. I., … Zasler,
N. D. (2002). The minimally conscious state: Definition and diagnostic criteria.
Neurology, 58(3), 349–353.
Giacino, J. T., Kalmar, K., & Whyte, J. (2004). The JFK coma recovery scale-revised:
Measurement characteristics and diagnostic utility. Archives of Physical Medicine
and Rehabilitation, 85(12), 2020–2029.
Giacino, J. T., Schnakers, C., Rodriguez-Moreno, D., Kalmar, K., Schiff, N., & Hirsch,
J. (2009). Behavioral assessment in patients with disorders of consciousness:
Gold standard or fool’s gold? Progress in Brain Research, 117, 33–47.
Gill-Thwaites, H. (1997). The sensory modality assessment rehabilitation technique—
A tool for assessment and treatment of patients with severe brain injury in a
vegetative state. Brain Injury, 11(10), 723–734. doi:10.1080/026990597123098
Gill-Thwaites, H. (2006). Lotteries, loopholes, and luck: Misdiagnosis in the vegeta-
tive state patient. Brain Injury, 20(13–14), 1321–1328.
Gill-Thwaites, H., & Munday, R. (1999). The sensory modality assessment and reha-
bilitation technique (SMART): A comprehensive integrated assessment and
treatment protocol for the vegetative state and minimally responsive patient.
Neuropsychological Rehabilitation, 9(3–4), 305–320. doi:10.1080/096020199389392
Hirschberg, R., & Giacino, J. T. (2011). The vegetative and minimally conscious
states: Diagnosis, prognosis, and treatment. Neurologic Clinics, 29(4), 773–786.
doi:10.1016/j.ncl.2011.07.009
Jennett, B., & Plum, F. (1972). Persistent vegetative state after brain damage: A syn-
drome in search of a name. Lancet, 1(7753), 734–737.
Jones, S. J., Pato, M. V., Sprague, L., Stokes, M., Munday, R., & Haque, N. (2000).
Auditory evoked potentials to spectro-temporal modulation of complex
tones in normal subjects and patients with severe brain injury. Brain, 123(5),
1007–1016.
Laureys, S., Celesia, G. G., Cohadon, F., Lavrijsen, J., León-Carrión, J., Sannita,
W. G., … the European Task Force on Disorders of Consciousness. (2010).
Unresponsive wakefulness syndrome: A new name for the vegetative state or
apallic syndrome. BMC Medicine, 8, 68–71. doi:10.1186/1741-7015-8-68
Laureys, S., & Schiff, N. D. (2012). Coma and consciousness: Paradigms (re)
framed by neuroimaging. Neuroimage, 61(2), 478–491. doi:10.1016/j.
neuroimage.2011.12.041
Lichtensztejn, M., Macchi, P., & Lischinsky, A. (2014). Music therapy and disorders
of consciousness: Providing clinical data for differential diagnosis between
Vol. 53, No. 1 25