See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/367763584

Opportunities for Improving Motor Assessment and Rehabilitation After

Stroke by Leveraging Video-Based Pose Estimation

Article in American journal of physical medicine & rehabilitation / Association of Academic Physiatrists · February 2023
DOI: 10.1097/PHM.0000000000002131

CITATIONS READS

6 220

5 authors, including:

Ryan Roemmich
Kennedy Krieger Institute/Johns Hopkins University School of Medicine
90 PUBLICATIONS 1,512 CITATIONS

SEE PROFILE

All content following this page was uploaded by Ryan Roemmich on 13 February 2023.

The user has requested enhancement of the downloaded file.

 ANALYSIS & PERSPECTIVE

Opportunities for Improving Motor Assessment and

Rehabilitation After Stroke by Leveraging Video-Based
Pose Estimation
Kendra M. Cherry-Allen, PT, DPT, PhD, Margaret A. French, PT, DPT, PhD, Jan Stenum, PhD,
Jing Xu, PhD, and Ryan T. Roemmich, PhD

Abstract: Stroke is a leading cause of long-term disability in adults in the future of precision (i.e., patient-specific) rehabilitation.13
the United States. As the healthcare system moves further into an era Artificial intelligence in particular shows outsized promise
of digital medicine and remote monitoring, technology continues to not as a replacement for the clinician but as a valuable tool that
play an increasingly important role in post-stroke care. In this Analysis can provide insight into motor function and inform clinical
and Perspective article, opportunities for using human pose estimation— decision making.14
an emerging technology that uses artificial intelligence to track human This article looks ahead to discuss promising roles for
movement kinematics from simple videos recorded using household emerging human pose estimation technology in motor rehabil-
devices (e.g., smartphones, tablets)—to improve motor assessment itation after stroke. Pose estimation is an artificial intelligence
and rehabilitation after stroke are discussed. The focus is on the poten- technology that uses computer vision to identify and track
tial of two key applications: (1) improving access to quantitative, ob- key features of the human body (e.g., leg joints and fingers)
jective motor assessment and (2) advancing telerehabilitation for per- from simple videos that are easily recorded in the home or
sons post-stroke. clinic using common household devices (e.g., smartphones
and tablets). This technology offers clear and significant potential
Key Words: Stroke, Video, Motor, Assessment, Rehabilitation, for applications in rehabilitation, as it enables quantitative mea-
Physical Therapy, Computer Vision, Pose Estimation surement of human movement kinematics in virtually any setting
(Am J Phys Med Rehabil 2023;102:S68–S74) with minimal cost, time investment, and technological require-
ments. Here, the focus is specifically on applications of pose esti-
mation in post-stroke motor assessment and telerehabilitation.
Effective rehabilitation after stroke requires accurate as-
pproximately 795,000 incidences of stroke occur annually
A 1
in the United States alone, establishing stroke as a leading
1
cause of long-term disability in adults. Stroke often impairs
sessment of a patient’s motor abilities and subsequent delivery
of an appropriate treatment. There is a need for new methods of
post-stroke motor assessment because current methods either
2
many aspects of movement, spanning fine motor control of are subjective (e.g., clinical scales such as the Fugl-Meyer As-
the fingers3,4 to complex whole-body tasks like walking.5–7 sessment15), expensive, and inaccessible to most clinicians
Rehabilitation is an essential component of post-stroke care be- (e.g., motion capture systems) or provide only limited informa-
cause early, effective interventions can lead to significant im- tion about specific, predefined features of movement (e.g., mo-
provements in motor function.8 bile applications, wearables, and gait mats). Reliance on these
Like many fields of medicine, rehabilitation continues to methods limits our abilities to track rehabilitation outcomes
accelerate into a digital era. Interest in remote measurement and timelines of post-stroke recovery because measurement
and monitoring of patient function has increased rapidly,9 and is infrequent or data limited. There is a significant role for pose
many studies have begun to investigate the feasibility and effi- estimation to address all of these limitations by enabling objec-
cacy of telerehabilitation vs. conventional physical therapy10–12 tive, low-cost, comprehensive motor assessment for persons
(Fig. 1). Technological innovations that provide new insight post-stroke.
into patient function or advance remote delivery of care will There are also significant limitations with current ap-
undoubtedly continue to play increasingly important roles in proaches to delivery of motor rehabilitation after stroke. Post-
stroke rehabilitation is commonly delivered via in-person
From the Department of Physical Medicine and Rehabilitation, Johns Hopkins Uni- physical, occupational, and/or speech therapy. This necessi-
versity School of Medicine, Baltimore, Maryland (KMC-A, MAF, JS, RTR);
Department of Physical Therapy Education, Western University of Health Sci- tates access to reliable transportation and proximity to a ther-
ences, Lebanon, Oregon (KMC-A); Center for Movement Studies, Kennedy apy clinic. Furthermore, this model of care assumes that gains
Krieger Institute, Baltimore, Maryland (JS, RTR); and Department of Kinesiol-
ogy, University of Georgia, Athens, Georgia (JX). achieved in the clinic will translate to real-world activities. Pa-
All correspondence should be addressed to: Kendra M. Cherry-Allen, PT, DPT, PhD, tients in vulnerable and underserved populations are in need of
Western University of Health Sciences, 2665 S Santiam Hwy, Lebanon, OR more accessible approaches to rehabilitation therapy,16–19 cre-
97355.
R.T. Roemmich acknowledges funding from the RESTORE Center (via NIH grant ating a clear need to deliver therapy within real-world settings
P2CHD101913). outside of the clinic (e.g., within the home).20 Pose estimation
Margaret A. French and Jan Stenum are in training. could eventually facilitate in-home motor rehabilitation to any-
Financial disclosure statements have been obtained, and no conflicts of interest have
been reported by the authors or by any individuals in control of the content of one with access to a simple video recording device.
this article. This article is structured into three primary sections. First,
Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.
ISSN: 0894-9115 there is a brief background on human pose estimation. Second,
DOI: 10.1097/PHM.0000000000002131 there is a discussion of applications for pose estimation in

S68 www.ajpmr.com American Journal of Physical Medicine & Rehabilitation • Volume 102, Number 2 (Suppl), February 2023

Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.

Volume 102, Number 2 (Suppl), February 2023 Applications of Pose Estimation After Stroke

FIGURE 1. The number of new articles indexed on PubMed annually over the past 5 yrs resulting from the following searches (from top to bottom):
“stroke,” “stroke rehabilitation,” “stroke remote,” “stroke telerehabilitation,” and “pose estimation.” Percentages shown above each bar plot indicate
growth from 2017 to 2021.

improving quantitative post-stroke motor assessment. Finally, human movement tracking.21–28 In these fields, it is often used
there are proposed applications for leveraging pose estimation for applications like movement pattern recognition,29–31 but
to advance telerehabilitation after stroke. pose estimation has not yet been used widely in clinical set-
tings or rehabilitation science (with notable limitations of cur-
WHAT IS HUMAN POSE ESTIMATION? rent algorithms reviewed in Seethapathi et al.32).
Human pose estimation is an artificial intelligence tech- Three important barriers have precluded clinical applica-
nology that uses computer vision to identify and track key fea- tions of pose estimation. First, it is not known how well these
tures of the human body from simple videos. A simplified way approaches track and measure movement in clinical popula-
of thinking about pose estimation is as “motion capture in your tions (e.g., persons post-stroke) where motor deficits, assistive
pocket,” where two- or three-dimensional human movement devices, and out-of-plane compensatory movements may make
kinematics can be generated from videos recorded by a com- tracking and analysis more difficult. There is a need for
mon smartphone or tablet device. This technology has rapidly large-scale validation and feasibility studies to demonstrate
gained traction in data science and neuroscience communities the clinical potential of this technology. Second, there is a need
with a wide array of different software options for performing for accessible software that can be used with minimal technical

© 2023 Wolters Kluwer Health, Inc. All rights reserved. www.ajpmr.com S69

Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.

Cherry-Allen et al. Volume 102, Number 2 (Suppl), February 2023

expertise. Current algorithms require at least some degree of tively in clinical settings without access to expensive, research-
computer programming acumen to install and execute. Third, grade motion capture equipment. Furthermore, accessible tools
there is a paucity of data and awareness about potential clinical for quantitative in-home measurement of movement kinemat-
applications of pose estimation. ics are not currently available. There is clear potential for pose
Only a handful of studies have explored clinical applica- estimation to offer new approaches for precise measurement of
tions in humans (recently reviewed in Stenum et al.33), al- motor function post-stroke using accessible, affordable tech-
though this number will likely grow rapidly in the coming nologies that are commonly available within the home and
years. Existing applications of pose estimation in stroke popu- clinic (e.g., smartphones and tablets).
lations are in their nascency and have focused on estimating Third, many post-stroke motor assessments are performed
spatiotemporal gait parameters in small samples of persons using validated tasks that necessitate clinic visits because they
post-stroke.34,35 Many other computer vision-based ap- require interaction with specific objects (e.g., Nine-Hole Peg
proaches have been used to measure movement in persons Test,43 Purdue Pegboard,44 and Box and Block Test45). Beyond
post-stroke (some examples reviewed in Souza et al.36), al- the necessity of an in-person clinic visit, these types of assess-
though most of these require specialized equipment (e.g., a ments have several other important limitations. Some clinics
Microsoft Kinect device) that is inherently less accessible than may have access to only some of these assessment materials
video-based techniques like pose estimation. but not others; training is often required to administer the as-
Of note is the fact that most existing pose estimation algo- sessment appropriately; and some assessments can take con-
rithms require intensive computing capabilities. Depending on siderable time to administer (especially if the patient dem-
the duration and complexity of the video recording and desired onstrates severe impairment and completes subtasks of the
output (e.g., hand-only tracking vs. full-body tracking), a graph- assessments very slowly). There is a need to improve the flexi-
ics processing unit may be necessary for time-efficient analysis. bility of motor assessments to promote accessibility in a wider
Real-time movement tracking is also available in some algo- range of environments. Validation of pose estimation ap-
rithms (e.g., OpenPose25) but is particularly computationally proaches for tracking movements involved in these clinical stan-
intensive. Therefore, although only a simple video is needed dards against ground-truth measurements and commonly used
as an input into the pose estimation algorithm, current algo- clinical assessments could lead to the development of remote as-
rithms often require significant computing power to perform sessments that approximate these object-based tests without the
the quantitative kinematic tracking. Fortunately, there are free need for the objects themselves or an in-person clinic visit.
resources available (e.g., Google Colaboratory) that can pro-
vide this computing power remotely if the user does not own
a graphics processing unit. What Aspects of Movement Could Be Measured?
Pose estimation has the potential to capture aspects of
POTENTIAL APPLICATIONS OF HUMAN POSE post-stroke motor impairment that are observable in movement
kinematics. Although many devices exist for measuring move-
ESTIMATION IN POST-STROKE ment kinematics outside of the clinic (e.g., wearables, com-
MOTOR ASSESSMENT puter vision-based gaming systems), pose estimation could
provide significant advantages over existing remote monitor-
Why Use Pose Estimation to Measure Movement ing devices in that the data are inherently “raw” (i.e., there is
After Stroke? considerable flexibility in what can be measured) and, impor-
There are several reasons that automated video-based mo- tantly, no equipment is required beyond a simple video record-
tor assessments could significantly improve care for persons ing device. This subsection discusses three primary areas of
post-stroke. First, frequent motor assessment is critical for potential application: (1) fine motor control of the hand and
tracking recovery and rehabilitation progress after stroke. Cur- fingers, (2) arm movements, and (3) gait (Fig. 2).
rent standards for post-stroke motor assessment (e.g., Fugl- Many persons post-stroke exhibit kinematic deviations in
Meyer Assessment15,37 and Action Research Arm Test38,39) rely movements of the paretic hand3,4,46–53 that result from impair-
upon subjectively rated ordinal scales that require a trained clini- ments in fine motor control. Deficits in control can be quanti-
cian to manually inspect and rate the performance of many sim- fied by measuring the ability to perform skilled and/or func-
ple movement tasks and aspects of motor function or impair- tional movements with the fingers39,45,54–56 or by testing the
ment. Many of these are largely kinematic in nature and could ability to individuate movements of one finger from an-
be captured using pose estimation, including assessments of other.3,4,46,47,57 Many key aspects of these assessments—
movement speed, amplitude, and range of motion. The reliance flexion and extension of individual fingers, closing the fingers
upon the time and expertise of the clinician limits the frequency and thumb in a precision grip, opening of the hand, and move-
with which motor assessments can be performed, thereby limit- ment of the fingers in isolation of one another—can be cap-
ing information about the trajectory of the patient’s recovery tured in movement kinematics and could lend well to assess-
and/or rehabilitation. Pose estimation provides an avenue for ment via pose estimation. It was recently shown that several
fast, accurate, and objective measurement of motor function in relevant tasks—hand opening and closing, hand pronation and
persons post-stroke. supination, and finger tapping—can be tracked accurately using
Second, persons post-stroke often exhibit a wide variety of pose estimation in healthy young adults,58 although this ap-
motor deficits that vary in type and severity.5,6,37,40–42 Many proach has not yet been validated in persons post-stroke.
persons post-stroke exhibit impairment in fine motor control Beyond confirming existing assessments, video-based
and gait, but these impairments are difficult to assess quantita- pose estimation also holds promise for filling existing gaps in

S70 www.ajpmr.com © 2023 Wolters Kluwer Health, Inc. All rights reserved.

Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.

Volume 102, Number 2 (Suppl), February 2023 Applications of Pose Estimation After Stroke

FIGURE 2. A, General workflow for using pose estimation for measurement of clinically relevant movement kinematics. B, Example applications for
using pose estimation to track movement kinematics with clinical relevance for persons post-stroke.

post-stroke dexterity assessment. Clinical assessments of dex- point, and joint individuation of shoulder, elbow, and wrist.
terous hand movements often use ecologically sound tasks Interestingly, joint individuation was the best predictor of kine-
(e.g., precision grip and object manipulation) at the expense matic properties such as reaching path curvature and end-point
of granularity. On the other hand, laboratory-based kinematic/ error.68 Video-based pose estimation may be an accessible tool
kinetic assessments with higher granularity often ignore key for capturing these kinematic features. Furthermore, such anal-
aspects of real-world dexterous control. For example, most as- yses may provide richer kinematic information than can be ob-
sessments of finger individuation rely on devices that only as- tained in most current clinical settings, allowing clinicians and
sess movement in one or two dimensions (e.g., finger flexion researchers to directly relate these measures with clinical as-
and extension) but omit other movements that are key aspects sessments. For example, the Fugl-Meyer Assessment of Upper
of hand dexterity (e.g., abduction/adduction and circumduc- Extremity impairment after stroke underscores impairment in
tion). One study using the Cyber Glove (Virtual Technologies, out-of-synergy joint movements (e.g., joint extension, shoulder
Palo Alto, CA) showed that abduction and adduction were elevation/retraction/abduction/rotation, forearm pronation/supination,
more impaired than flexion/extension in persons post-stroke.4 wrist circumduction).15 These assessments often cannot be fully
However, the Cyber Glove also has limited resolution. For in- captured by laboratory-based motion capture systems, and clin-
stance, it does not accurately assess middle finger abduction/ ical scores lack granularity and are based on subjective visual in-
adduction, and the measured ranges of motion for all the finger spection. Pose estimation may have the potential to extract de-
joints were consistently much smaller than those assessed with- tailed kinematic information with a higher level of granularity
out wearing the glove.4 Moreover, individuation impairment than Fugl-Meyer scores, although the accuracy of this approach
assessed using the Cyber Glove is only minimally correlated has yet to be tested. However, also note that pose estimation will
with clinical assessment of hand function.4,59 not be able to capture other important aspects of hand dexterity,
Because of the lack of granularity of these assessment including sensory feedback,69 and force production.70
tools, it is difficult to determine the relationship between indi- Finally, more than half of persons post-stroke have resid-
viduation and other hand functions. One reason for this dif- ual gait impairments even after prolonged rehabilitation.71 Gait
ficulty is that motor control variables omitted from these dysfunction is heterogeneous and idiosyncratic after stroke:
assessments—such as movement direction, velocity, and spa- deficits are apparent in a variety of leg joint movements (e.g.,
tial and temporal coordination across fingers—may play essen- stiff knee and foot drop5,6,72–76), spatiotemporal gait parame-
tial roles in functional everyday activities. Another possible ters (e.g., shortened paretic stance time and asymmetric step
reason is that clinical assessments often quickly reach a ceiling lengths5,7,77,78), and measures of global gait function (e.g.,
when residual deficits can only be detected by kinematic/ slowed walking speed6,78). Many clinical facilities rely on ei-
kinetic measures. These fine-grained analyses become more ther subjective visual inspection of gait or devices like wear-
informative with respect to dexterous control, where the motor ables or instrumented gait mats that provide only limited and
repertoires push the boundary regions of the neural/biomechanical predefined gait metrics (e.g., spatiotemporal parameters but
constraints.60 Detection of these subtle impairments is critical little/no information about whole-body gait kinematics). How-
in determining the true recovery of those repertoires vs. com- ever, it is possible that many (if not all) of the deficits in kine-
pensation after stroke. For example, persons post-stroke have matics and spatiotemporal gait parameters mentioned above
demonstrated impairment in anticipatory shaping of hand pos- could be measured using movement tracking via pose estima-
ture in a reach-and-grasp task and exhibited a compensatory tion. Indeed, recently developed pose estimation–based gait
strategy of increasing metacarpophalangeal joint flexion to ad- analysis workflow demonstrated accurate measurement of a
just to different object shapes.61 wide variety of gait parameters in healthy adults79 and others
Movements of the paretic arm are also impaired in many have shown preliminary data suggesting that similar ap-
persons post-stroke.41,62–66 Persons post-stroke commonly proaches can accurately estimate selected gait parameters in
show kinematic deviations in reaching movements,41,67 includ- persons post-stroke.34,35 Similar to the other domains of move-
ing impairments in velocity, path curvature, index finger end- ment that we have discussed previously, there is a clear need for

© 2023 Wolters Kluwer Health, Inc. All rights reserved. www.ajpmr.com S71

Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.

Cherry-Allen et al. Volume 102, Number 2 (Suppl), February 2023

larger validation studies in persons post-stroke; however, early cators of patient recovery with improved accuracy when com-
results suggest exciting potential for assessing post-stroke gait pared with reliance on clinical scores.
using pose estimation-based gait analysis. Second, pose estimation stands to advance the field of
post-stroke rehabilitation by allowing clinicians and re-
searchers to “see behind the curtain” into how people move
POTENTIAL APPLICATIONS OF HUMAN in the real world and better understand the real-world impact
POSE ESTIMATION IN REHABILITATION of clinical interventions. Because pose estimation requires only
AFTER STROKE a simple video that can be recorded using household devices,
The ability to measure movement kinematics via pose es- there is significant potential for measuring patient kinematics
timation has the potential for significant impact on post-stroke during natural behaviors directly in the home or other commu-
rehabilitation practice. First, it can provide clinicians with a nity setting. Improving real-world performance is the ultimate
readily accessible tool to gather quantitative data about move- objective of rehabilitation, and clinicians and researchers oper-
ment quality, thus generating a more comprehensive picture ate under the assumption that a person’s in-clinic movement
of a patient’s movement status and changes to that status that abilities directly reflect how he/she moves in daily life. How-
occur during rehabilitation. Traditionally, precise kinematic ever, recent evidence challenges this assumption. Indeed, clin-
data were unavailable to clinicians as they required expensive ical motor capacity and real-world motor performance of the
and sophisticated three-dimensional motion analysis hardware paretic arm have been found to be incongruent in persons
and software to obtain. Typical clinical outcomes focus on post-stroke.88 Pose-estimation movement analysis obtained
whether a person can execute a functional task (e.g., Can a per- during real-world activity may help clinicians understand to
son pick up a cup to drink? How many steps does a person take what extent movement and movement improvements exhibited
in a day?), with only sparse observation-based information in the clinic reflect movement in the home and contribute to
gathered about how a person completes the task (e.g., move- more naturalistic accounts of meaningful changes during
ment quality). However, during a recent “Stroke Recovery post-stroke rehabilitation.89
and Rehabilitation Roundtable,” experts agreed that there is Finally, pose estimation–based movement analysis can be
an urgent need to include movement quality measures (e.g., ki- incorporated into a growing realm of rehabilitation: telereha-
nematics) in stroke rehabilitation trials.80 bilitation (Fig. 3). Telerehabilitation—the delivery of rehabili-
Several studies support the need for inclusion of move- tation in the home via videoconferencing—has emerged as a
ment quality measures in post-stroke rehabilitation. For in- promising mode to overcome access to in-person care.90
stance, upper extremity kinematics in persons post-stroke Home-based stroke rehabilitation is safe and provides impor-
(e.g., movement time, trajectory length, directness, smooth- tant insight about environmental factors that influence mobil-
ness, and trunk displacement) were sensitive to change over ity. As in in-person post-stroke rehabilitation, a critical element
time, correlated with single time point upper extremity of effective telerehabilitation is the ability to collect quantita-
Fugl-Meyer scores, and associated with clinically meaningful tive data about an individual’s movement.91 A number of re-
improvements.81–84 Similarly, lower extremity kinematics— cent technologies have aimed to meet this need. For instance,
particularly those of the paretic leg—are correlated with mea- body sensors can identify the type and quantity of movement
sures of gait function and may be used to identify individuals during practice or daily routines.92 Microsoft Kinect and vir-
with continuing gait deficits,85 despite appearing fully “recov- tual reality systems are able to track reaching and grasping
ered” on traditional scales of gait function (gait speed).86,87 movements within a defined area.93,94 Pose estimation–based
Thus, quantitative movement analysis is an important clinical movement analysis can be used in conjunction with these other
tool for assessing pathologies and could provide real-time indi- remote monitoring tools but has the advantage of not requiring

FIGURE 3. Conceptual diagram showing a clinician-centered approach for using pose estimation to measure in-home movement kinematics and, in
turn, leveraging these data to facilitate in-home telerehabilitation.

S72 www.ajpmr.com © 2023 Wolters Kluwer Health, Inc. All rights reserved.

Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.

Volume 102, Number 2 (Suppl), February 2023 Applications of Pose Estimation After Stroke

any equipment beyond the patient’s own recording device to 8. Dromerick AW, Geed S, Barth J, et al: Critical Period after Stroke Study (CPASS): A phase II
clinical trial testing an optimal time for motor recovery after stroke in humans.
capture the data. Together, these technologies will allow fre- Proc Natl Acad Sci U S A 2021;118:e2026676118
quent, precise data acquisition that can be used to individualize 9. Nascimento LMSD, Bonfati LV, Freitas MB, et al: Sensors and systems for physical
care and improve post-stroke rehabilitation. rehabilitation and health monitoring—A review. Sensors (Basel) 2020;20:4063
10. Seron P, Oliveros MJ, Gutierrez-Arias R, et al: Effectiveness of telerehabilitation in physical
therapy: A rapid overview. Phys Ther 2021;101:1–18
WHAT’S NEXT? 11. Turolla A, Rossettini G, Viceconti A, et al: Musculoskeletal physical therapy during the
COVID-19 pandemic: Is telerehabilitation the answer? Phys Ther 2020;100:1260–4
Before widespread implementation of pose estimation 12. Levy CE, Silverman E, Jia H, et al: Effects of physical therapy delivery via home video
tools for measuring movement after stroke, there is a need to telerehabilitation on functional and health-related quality of life outcomes. J Rehabil Res Dev
2015;52:361–70
understand (1) how well these tools can capture common post-
13. Adans-Dester C, Hankov N, O’Brien A, et al: Enabling precision rehabilitation interventions
stroke motor deficits and (2) best practices regarding how videos using wearable sensors and machine learning to track motor recovery. NPJ Digit Med 2020;
should be recorded to capture specific deficits. This validation 3:121
and feasibility work is ongoing,34,35 but significantly more re- 14. Fogel AL, Kvedar JC: Artificial intelligence powers digital medicine. NPJ Digit Med 2018;1:5
15. Fugl Meyer AR, Jaasko L, Leyman I: The post stroke hemiplegic patient, I: A method for
search is needed to optimize the use of pose estimation for evaluation of physical performance. Scand J Rehabil Med 1975;7:13–31
post-stroke motor assessment. In particular, there is an unmet 16. Eberhardt MS, Pamuk ER: The importance of place of residence: Examining health in rural
need for studies of pose estimation applications for capturing and nonrural areas. Am J Public Health 2004;94:1682–6
motor deficits in the paretic hand and arm. Specifically, it re- 17. Pearson TA, Lewis C: Rural epidemiology: Insights from a rural population laboratory.
Am J Epidemiol 1998;148:949–57
mains to be determined how sensitive pose estimation is in 18. Joubert J, Prentice LF, Moulin T, et al: Stroke in rural areas and small communities. Stroke
capturing digit individuation and movements of exceptionally 2008;39:1920–8
small ranges of motion. 19. Ellis C, Hyacinth HI, Beckett J, et al: Racial/ethnic differences in poststroke rehabilitation
outcomes. Stroke Res Treat 2014;2014:950746
There are also a variety of human factors and infrastruc- 20. Mayo NE: Stroke rehabilitation at home: Lessons learned and ways forward. Stroke 2016;47:
ture barriers to clinical implementation of these technologies 1685–91
that are discussed at length in a previous review.33 Briefly, pose 21. Mathis A, Mamidanna P, Cury KM, et al: DeepLabCut: Markerless pose estimation of
estimation technologies need to become more user-friendly for user-defined body parts with deep learning. Nat Neurosci 2018;21:1281–9
22. Insafutdinov E, Pishchulin L, Andres B, et al: Deepercut: A deeper, stronger, and faster
persons without technical backgrounds, provide easily inter- multi-person pose estimation model, in: European Conference on Computer Vision.
pretable outcome measures with direct clinical relevance, and New York, Springer, 2016:34–50
become more accessible to groups that do not have access to 23. Pishchulin L, Insafutdinov E, Tang S, et al: DeepCut: Joint subset partition and labeling for
multi person pose estimation, in: Proceedings of the IEEE Conference on Computer Vision
high-powered computing resources. Clinical applications of and Pattern Recognition. Piscataway, NJ, IEEE, 2016:4929–37
pose estimation technologies will likely not become widely 24. Insafutdinov E, Andriluka M, Pishchulin L, et al: ArtTrack: Articulated multi-person tracking
used until these barriers—which are largely addressable— in the wild, in: Proceedings of the IEEE Conference on Computer Vision and Pattern
are overcome. Recognition. Piscataway, NJ, IEEE, 2017:1293–301
25. Cao Z, Hidalgo Martinez G, Simon T, et al: OpenPose: Realtime multi-person 2D pose
estimation using part affinity fields. IEEE Trans Pattern Anal Mach Intell 2019;43:172–86
26. Cao Z, Simon T, Wei SE, et al: Realtime multi-person 2D pose estimation using part affinity
CONCLUSIONS fields, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Pose estimation has the potential to make a significant im- 2017:1302–10
pact on the way that movement is measured and care after 27. Andriluka M, Pishchulin L, Gehler P, et al: 2D human pose estimation: New benchmark and
state of the art analysis, in: 2014 IEEE Conference on Computer Vision and Pattern
stroke is delivered. The ability to capture clinically relevant Recognition. 2014:3686–93
movement kinematics using a household video recording de- 28. Toshev A, Szegedy C: DeepPose: Human pose estimation via deep neural networks, in:
vice could lead to the development of widely accessible tools 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014:1653–60
29. Sokolova A, Konushin A: Pose-based deep gait recognition. IET Biometrics 2019;8:134–43
for quantitative post-stroke motor assessment that could be per- 30. Guo Y, Deligianni F, Gu X, et al: 3-D canonical pose estimation and abnormal gait recognition
formed in virtually any setting, including directly in the home with a single RGB-D camera. IEEE Robot Autom Lett 2019;4:3617–24
or clinic. Furthermore, there is opportunity to leverage pose es- 31. Nakano N, Sakura T, Ueda K, et al: Evaluation of 3D markerless motion capture accuracy
timation tools to deliver telerehabilitation interventions that tar- using OpenPose with multiple video cameras. Front Sports Act Living 2020;2:50
32. Seethapathi N, Wang S, Saluja R, Blohm G, Kording KP: Movement science needs different
get movement quality directly in patient homes. It is antici- pose tracking algorithms. 2019. arXiv Prepr arXiv190710226. Available at: https://arxiv.org/
pated that these technologies will play a prominent role in the abs/1907.10226. Accessed August 3, 2019
future of digital medicine and remote monitoring of motor 33. Stenum J, Cherry-Allen KM, Pyles CO, et al: Applications of pose estimation in human health
and performance across the lifespan. Sensors (Basel) 2021;21:7315
function in persons post-stroke.
34. Lonini L, Moon Y, Embry K, et al: Video-based pose estimation for gait analysis in stroke
survivors during clinical assessments: A proof-of-concept study. Digit Biomark 2022;6:9–18
35. Moro M, Marchesi G, Odone F, et al: Markerless gait analysis in stroke survivors based on
REFERENCES computer vision and deep learning: A pilot study. Proc ACM Symp Appl Comput 2020:
1. Virani SS, Alonso A, Benjamin EJ, et al: Heart Disease and Stroke Statistics—2020 Update: 2097–104
A report from the American Heart Association. Circulation 2020;141:e139–596 36. de Souza JT, Naves ELM, de Sá AAR: Computer vision devices for tracking gross upper limb
2. Lawrence ES, Coshall C, Dundas R, et al: Estimates of the prevalence of acute stroke movements in post-stroke rehabilitation. Res Soc Dev 2021;10:e57910616143
impairments and disability in a multiethnic population. Stroke 2001;32:1279–84 37. Gladstone DJ, Danells CJ, Black SE: The Fugl-Meyer Assessment of Motor Recovery after
3. Raghavan P, Petra E, Krakauer JW, et al: Patterns of impairment in digit independence after Stroke: A critical review of its measurement properties. Neurorehabil Neural Repair 2002;16:
subcortical stroke. J Neurophysiol 2006;95:369–78 232–40
4. Lang CE, Schieber MH: Reduced muscle selectivity during individuated finger movements in 38. Lyle RC: A performance test for assessment of upper limb function in physical rehabilitation
humans after damage to the motor cortex or corticospinal tract. J Neurophysiol 2004;91: treatment and research. Int J Rehabil Res 1981;4:483–92
1722–33 39. Yozbatiran N, Der-Yeghiaian L, Cramer SC: A standardized approach to performing the action
5. Olney SJ, Richards C: Hemiparetic gait following stroke. Part I: Characteristics. Gait Posture research arm test. Neurorehabil Neural Repair 2008;22:78–90
1996;4:136–48 40. Peterson CL, Cheng J, Kautz SA, et al: Leg extension is an important predictor of paretic leg
6. Knutsson E: Gait control in hemiparesis. Scand J Rehabil Med 1981;13:101–8 propulsion in hemiparetic walking. Gait Posture 2010;32:451–6
7. Knutsson E, Richards C: Different types of disturbed motor control in gait of hemiparetic 41. Cirstea MC, Levin MF: Compensatory strategies for reaching in stroke. Brain 2000;123:
patients. Brain 1979;102:405–30 940–53

© 2023 Wolters Kluwer Health, Inc. All rights reserved. www.ajpmr.com S73

Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.

Cherry-Allen et al. Volume 102, Number 2 (Suppl), February 2023

42. Langhorne P, Coupar F, Pollock A: Motor recovery after stroke: A systematic review. 68. Zackowski KM, Dromerick AW, Sahrmann SA, et al: How do strength, sensation, spasticity
Lancet Neurol 2009;8:741–54 and joint individuation relate to the reaching deficits of people with chronic hemiparesis?
43. Mathiowetz V, Weber K, Kashman N, et al: Adult norms for the nine hole peg test of finger Brain 2004;127:1035–46
dexterity. Occup Ther J Res 1985;5:24–38 69. Sobinov AR, Bensmaia SJ: The neural mechanisms of manual dexterity. Nat Rev Neurosci
44. Tiffin J, Asher EJ: The Purdue Pegboard: Norms and studies of reliability and validity. 2021;22:741–57
J Appl Psychol 1948;32:234–47 70. Rotella MF, Nisky I, Koehler M, et al: Learning and generalization in an isometric visuomotor
45. Mathiowetz V, Volland G, Kashman N, et al: Adult norms for the Box and Block Test of task. J Neurophysiol 2015;113:1873–84
manual dexterity. Am J Occup Ther 1985;39:386–91 71. Mayo NE, Wood-Dauphinee S, Ahmed S, et al: Disablement following stroke.
46. Xu J, Haith AM, Krakauer JW: Motor control of the hand before and after stroke. Disabil Rehabil 1999;21:258–68
Clin Syst Neurosci 2015;1:271–89 72. Kerrigan DC, Karvosky ME, Riley PO: Spastic paretic stiff-legged gait: Joint kinetics.
47. Xu J, Ejaz N, Hertler B, et al: Separable systems for recovery of finger strength and control Am J Phys Med Rehabil 2001;80:244–9
after stroke. J Neurophysiol 2017;118:1151–63 73. Chen G, Patten C, Kothari DH, et al: Gait deviations associated with post-stroke hemiparesis:
48. Kamper DG, Fischer HC, Cruz EG, et al: Weakness is the primary contributor to finger Improvement during treadmill walking using weight support, speed, support stiffness, and
impairment in chronic stroke. Arch Phys Med Rehabil 2006;87:1262–9 handrail hold. Gait Posture 2005;22:57–62
49. Cruz EG, Waldinger HC, Kamper DG: Kinetic and kinematic workspaces of the index finger 74. Stanhope VA, Knarr BA, Reisman DS, et al: Frontal plane compensatory strategies associated
following stroke. Brain 2005;128:1112–21 with self-selected walking speed in individuals post-stroke. Clin Biomech 2014;29:518–22
50. Carey JR, Kimberley TJ, Lewis SM, et al: Analysis of f MRI and finger tracking training in 75. Sulzer JS, Gordon KE, Dhaher YY, et al: Preswing knee flexion assistance is coupled with hip
subjects with chronic stroke. Brain 2002;125:773–88 abduction in people with stiff-knee gait after stroke. Stroke 2010;41:1709–14
51. Li S, Latash ML, Yue GH, et al: The effects of stroke and age on finger interaction in 76. Takebe K, Basmajian JV: Gait analysis in stroke patients to assess treatments of foot drop.
multi-finger force production tasks. Clin Neurophysiol 2003;114:1646–55 Arch Phys Med Rehabil 1976;57:305–10
77. Hsu AL, Tang PF, Jan MH: Analysis of impairments influencing gait velocity and asymmetry
52. Lang CE, DeJong SL, Beebe JA: Recovery of thumb and finger extension and its relation to
of hemiplegic patients after mild to moderate stroke. Arch Phys Med Rehabil 2003;84:
grasp performance after stroke. J Neurophysiol 2009;102:451–9
1185–93
53. Raghavan P: The nature of hand motor impairment after stroke and its treatment.
78. Olney SJ, Griffin MP, McBride ID: Temporal, kinematic, and kinetic variables related to gait
Curr Treat Options Cardiovasc Med 2007;9:221–8
speed in subjects with hemiplegia: A regression approach. Phys Ther 1994;74:872–85
54. Berardi A, Saffioti M, Tofani M, et al: Internal consistency and validity of the Jebsen-Taylor
79. Stenum J, Rossi C, Roemmich RT: Two-dimensional video-based analysis of human gait
hand function test in an Italian population with hemiparesis. NeuroRehabilitation 2019;45:
using pose estimation. PLoS Comput Biol 2021;17:e1008935
331–9
80. Kwakkel G, Van Wegen EE, Burridge JH, et al: Standardized measurement of quality of upper
55. Wolf SL, Catlin PA, Ellis M, et al: Assessing Wolf Motor Function Test as outcome measure
limb movement after stroke: Consensus-based core recommendations from the Second Stroke
for research in patients after stroke. Stroke 2001;32:1635–9
Recovery and Rehabilitation Roundtable. Int J Stroke 2019;14:783–91
56. Barreca S, Gowland CK, Stratford P, et al: Development of the Chedoke Arm and Hand
81. Alt Murphy M, Willén C, Sunnerhagen KS: Responsiveness of upper extremity kinematic
Activity Inventory: Theoretical constructs, item generation, and selection. Top Stroke Rehabil
measures and clinical improvement during the first three months after stroke.
2004;11:31–42
Neurorehabil Neural Repair 2013;27:844–53
57. Wolbrecht ET, Rowe JB, Chan V, et al: Finger strength, individuation, and their interaction:
82. Demers M, Levin MF: Do activity level outcome measures commonly used in neurological
Relationship to hand function and corticospinal tract injury after stroke. Clin Neurophysiol
practice assess upper-limb movement quality? Neurorehabil Neural Repair 2017;31:623–37
2018;129:797–808
83. Van Dokkum L, Hauret I, Mottet D, et al: The contribution of kinematics in the assessment of
58. Cornman HL, Stenum J, Roemmich RT: Video-based quantification of human movement upper limb motor recovery early after stroke. Neurorehabil Neural Repair 2014;28:4–12
frequency using pose estimation: A pilot study. PLoS One 2021;16:e0261450 84. Mawase F, Cherry-Allen K, Xu J, et al: Pushing the rehabilitation boundaries: Hand motor
59. Lang CE, Schieber MH: Differential impairment of individuated finger movements in impairment can be reduced in chronic stroke. Neurorehabil Neural Repair 2020;34:733–45
humans after damage to the motor cortex or the corticospinal tract. J Neurophysiol 2003;90: 85. Padmanabhan P, Rao KS, Gulhar S, et al: Persons post-stroke improve step length symmetry
1160–70 by walking asymmetrically. J Neuroeng Rehabil 2020;17:1–14
60. Kutch JJ, Kuo AD, Bloch AM, et al: Endpoint force fluctuations reveal flexible rather than 86. Shin SY, Lee RK, Spicer P, et al: Quantifying dosage of physical therapy using lower body
synergistic patterns of muscle cooperation. J Neurophysiol 2008;100:2455–71 kinematics: A longitudinal pilot study on early post-stroke individuals. J Neuroeng Rehabil
61. Raghavan P, Santello M, Gordon AM, et al: Compensatory motor control after stroke: An 2020;17:15
alternative joint strategy for object-dependent shaping of hand posture. J Neurophysiol 2010; 87. Shin SY, Lee RK, Spicer P, et al: Does kinematic gait quality improve with functional gait
103:3034–43 recovery? A longitudinal pilot study on early post-stroke individuals. J Biomech 2020;105:
62. Krakauer JW: Arm function after stroke: From physiology to recovery. Semin Neurol 2005;25: 109761
384–95 88. Waddell KJ, Strube MJ, Bailey RR, et al: Does task-specific training improve upper limb
63. Kamper DG, McKenna-Cole AN, Kahn LE, et al: Alterations in reaching after stroke and their performance in daily life post-stroke? Neurorehabil Neural Repair 2017;31:290–300
relation to movement direction and impairment severity. Arch Phys Med Rehabil 2002;83: 89. Dobkin BH, Martinez C: Wearable sensors to monitor, enable feedback, and measure
702–7 outcomes of activity and practice. Curr Neurol Neurosci Rep 2018;18:1–8
64. Roby-Brami A, Feydy A, Combeaud M, et al: Motor compensation and recovery for reaching 90. Dicianno BE, Parmanto B, Fairman AD, et al: Perspectives on the evolution of mobile
in stroke patients. Acta Neurol Scand 2003;107:369–81 (mHealth) technologies and application to rehabilitation. Phys Ther 2015;95:397–405
65. Murphy MA, Willén C, Sunnerhagen KS: Kinematic variables quantifying upper-extremity 91. Dobkin BH: A rehabilitation-internet-of-things in the home to augment motor skills and
performance after stroke during reaching and drinking from a glass. Neurorehabil Neural exercise training. Neurorehabil Neural Repair 2017;31:217–27
Repair 2011;25:71–80 92. Appelboom G, Camacho E, Abraham ME, et al: Smart wearable body sensors for patient
66. Wagner JM, Rhodes JA, Patten C: Reproducibility and minimal detectable change of self-assessment and monitoring. Arch Public Heal 2014;72:28
three-dimensional kinematic analysis of reaching tasks in people with hemiparesis after 93. Webster D, Celik O: Systematic review of Kinect applications in elderly care and stroke
stroke. Phys Ther 2008;88:652–63 rehabilitation. J Neuroeng Rehabil 2014;11:108
67. Cirstea MC, Mitnitski AB, Feldman AG, et al: Interjoint coordination dynamics during 94. Laver KE, George S, Thomas S, et al: Virtual reality for stroke rehabilitation.
reaching in stroke. Exp Brain Res 2003;151:289–300 Cochrane Database Syst Rev 2015;2015:CD008349

S74 www.ajpmr.com © 2023 Wolters Kluwer Health, Inc. All rights reserved.

View publication stats Copyright © 2023 Wolters Kluwer Health, Inc. All rights reserved.