See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/228720023

A Tutorial on Motion Capture Driven Character Animation

Article · January 2008

CITATIONS READS
2 838

2 authors, including:

Baihua Li
Loughborough University
65 PUBLICATIONS   615 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Football player detection, tracking and action recognition View project

3D geometric modeling of objects View project

All content following this page was uploaded by Baihua Li on 04 April 2016.

The user has requested enhancement of the downloaded file.

Proceedings of the Eighth IASTED International Conference
Visualization, Imaging, and Image Processing (VIIP 2008)
September 1-3, 2008 Palma de Mallorca, Spain

A TUTORIAL ON MOTION CAPTURE DRIVEN CHARACTER ANIMATION

Ben Bradwell and Baihua Li
Department of Computing and Mathematics, Manchester Metropolitan University, UK
[email protected], [email protected]

ABSTRACT of image ambiguity and noise, markerless un-encumbrance

Motion capture (MoCap) is an increasingly important MoCap is still in its infancy in many aspects [3, 4, 5].
technique to create realistic human motion for animation.
However MoCap data are noisy, the resulting animation is The most reliable and commercially available systems are
often inaccurate and unrealistic without elaborate manual marker-based MoCap [6, 7], such as Vicon [8] and Motion
processing of the data. In this paper, we will discuss Analysis [9]. In a marker-based system, to reduce the
practical issues for MoCap driven character animation, difficulty of image processing, retro-reflective markers are
particularly when using commercial toolkits. We highlight attached on a subject at tightly clothed key sites, such as
open topics in this field for future research. MoCap joints and limbs, as shown in Fig.1. During MoCap, the
animations created in this project will be demonstrated at movements of performers are recorded by multiple cameras
the conference. in real-time with an unrestricted background. 3D marker
trajectories are extracted from the high contrast images.
KEY WORDS Various motion parameters can be derived from the concise
Marker-based motion capture, character animation, facial marker data and used for motion analysis, animation, etc.
animation. The measurement accuracy of such a system is at a level of
a few millimeters in a control volume spanning meters in
1. Introduction linear extent. The application prospect of marker-based
MoCap has largely motivated many excellent work in
Humans are very acute at recognising plausible motion. related fields, such as motion warping, blending,
This makes the task of generating aesthetically pleasing, retargeting, mapping, synthesizing, noise filtering, motion
human-like character animation a challenging problem. representation and data compression [3, 6, 7, 10, 11, 12].
Though manually key-framing continues to play an
essential and complementary role in real-time animation,
realism and productivity issues of key-framing are
becoming critical as the complexity and volume of content
increases in the fields of film, television and computer
games.

Motion capture (MoCap) is a relatively new technique

widely used for biomedical studies, sports analysis and
entertainment industry [1, 2, 3]. MoCap is the process of
measuring the movement of a performer (typically, but not Fig.1 Maker-based full body MoCap
always, human) in real-time and interpreting the
information in a compact and computer-usable manner.
Ideally, the capture of motion data should be easily MoCap can be highly productive to generate realistic and
available, inexpensive, and non-invasive. Various MoCap complex character animations. However, raw MoCap data
techniques have been developed, such as mechanical are very noisy [6, 7]. Though, significant progress has been
trackers, electro-mechanical trackers and electromagnetic made and many toolkits for MoCap handling have been
systems [2]. Due to the problems of intrusive sensors, developed, the resulting animation often contains artifacts
limited workspace, low accuracy and sampling rate, and requires time consuming elaborate handwork of highly
application of these systems has been largely restricted. skilled animators. In this tutorial, we present some practical
The state-of-the-art technology is video-based optical issues on creating character animation using marker-based
MoCap, which detects motion from standard video cameras MoCap data. We also identify open problems in this field
using advanced image processing and computer vision for future research.
techniques [3]. In this category, markerless MoCap
facilitates wide application prospect and has drawn great
attention in the last decade. Though much progress has
been made in this area, because of the inherent challenges

630-125 356
2. Character Animation offset data obtained from real MoCap. Though rescaling
can be done with the aid of “Talent Figure Mode” in the
2.1 Character modeling “Motion Capture” rollout of 3ds Max, we found it was very
Low number polygon models, composed by triangles or time consuming and could generate unexpected mesh
quadrilaterals, are widely adopted because of the good distortion in rigging process, as shown in Fig. 2. To solve
support for efficient rendering by current graphics this problem, the BIP format was adopted. The BIP file
hardware. Meshes can be generated interactively though a stores limb rotation data which are less dependent to
graphical interface using various primitives and modifiers, physical body measurement.
such as extrusion, symmetry, tweaking, subdivision
(NURMS in 3ds Max), and transformations. Joint meshes To preserve the fidelity of human movement in animation,
are special segments for attaching joints to an underlying MoCap data are usually captured at 60~120Hz. However,
skeleton. They also prevent mesh collision caused by the animation generated from MoCap on a frame-by-frame
deformation of adjacent segments. basis, at a high sampling-rate, can cause very slow
performance and may be unnecessary in applications. An
2.2. MoCap data pre-processing alternative way is to reduce the MoCap data to sparse key-
frames. We found this method makes the animation
MoCap marker data are usually noisy [6, 7]. Missing data compact, effective, and a lot easier to edit and personalize.
can be caused by occlusion. “Ghost” points may be In 3ds Max, the “Key Reduction” in the Motion Capture
involved from the workspace or caused by image analysis Conversion Parameters menu can be used to detect MoCap
algorithms due to imaging ambiguity. The data point are keys based on the motion intensity and interpolate the in-
unlabeled from 3D reconstruction. To infer the underlying betweens. However, the key reduction may cause artifacts
physical structure, an identification procedure is often like jittering movements and “foot-slide”. Jittering
required to know which point in massive MoCap data movements can be improved by tweaking (see Section 2.4).
corresponds to which marker attached to the subject. Even Foot-slide can be solved via “Footstep Extraction”.
with the most sophisticated system [8, 9], auto-labeling Footstep Extraction extrapolates footsteps so that keys that
may fail for complex movements [10]. In addition, motion make up the foot placement and movement will be turned
blur and measurement error can generate jagged marker into footsteps to maintain correct foot-toe-ground contact.
trajectories resulting in wobbly motion in animation. It is
cumbersome to manipulate and very difficult to alter noisy 2.3 MoCap rigging
MoCap data, especially when the key “essence” of the
motion is not distinguished from the large amount of To animate the character mesh, a complete bone hierarchy,
potentially irrelevant details and noise. In practice, we need or “skeleton”, is required with full simulation of Inverse
to re-capture a movement until an acceptable quality is Kinematics. Most commercial animation tools have pre-
achieved. For the MoCap data used in this project, built skeletons or third party plug-ins, such as Biped in 3ds
identification errors were corrected manually or using the Max.
algorithm developed in [10]. Missing data gaps were filled
by interpolation or splines; marker trajectories were
smoothed by various low-pass filters.

Fig. 3 Biped skeleton and posed Biped in character mesh

The first stages of rigging are to correctly position, scale

Fig. 2 Distorted mesh and gestures with CSM
and rotate the bones/joints of the skeleton within the
character mesh, as shown in Fig. 3. Subsequently, the
Common MoCap data formats include CSM, BVH and
skeleton should be linked to the polygon model, known as
BIP. The CSM file stores frame-by-frame 3D coordinates
rigging. Rigging starts from a root node, e.g. the pelvis, and
of identified body markers in world coordinate system. The
spans out through all the children in the hierarchy. In 3ds
BHV format contains hierarchy and motion data. The
Max, rigging will generate a series of structural vertex-
hierarchy section provides information on joint-to-joint
bone links and “Envelopes”, as shown in Fig. 4. Rigging
connections and offsets of child-segments from their
can be achieved manually in 3ds Max using two modifiers:
parent. The motion section describes the movements of
“Skin” and “Physique”. The Skin modifier provides many
these joints. These two formats require rescaling the
advanced features to aid complex rigging, such as
rigging skeleton to conform with absolute marker or joint

357
Manually Vertex Weighting and good flexibility over 3.1 Face modeling form range data
“Envelope” control. The main drawback of the Skin
modifier is the lack of deformation functions of its rigging A neutral face scan was obtained using a structure-light
Envelopes, therefore it cannot provide realistic mesh based range scanner manufactured by InSpeck [15], as
deformations around joints. The Physique modifier is shown in Fig. 5 left. The scan data contain measurement
designed particularly for rigging the Biped to a humanoid errors and holes caused by light reflections during the scan
character. It allows deformation models to be applied to the process. The data were filtered and some small holes were
rigging Envelopes. fixed using software tools embedded in the InSpect system.
The distorted vertices were fixed manually with the aid of
“Soft Selection” in 3ds Max. The original scan was also
very large, almost 50,000 polygons. This amount of detail
is unmanageable for real-time facial animation. We used
the “MultiRes” modifier in 3ds Max to reduce the polygon
count to about 2200 as shown in Fig. 5 right.

Fig. 4 Rigging links and Envelopes

Envelopes are used to define the region of motion influence

of a link in the skeleton hierarchy and control deformation
styles. Each envelope is comprised of an inner and an outer
bound (ring). Vertices within the inner bound are assigned
a weight value of 1, indicating the strongest influence. Fig. 5 Noisy dense range scan (left) and clean-up low-
Vertices between the inner and outer bounds are assigned a polygon mesh model (right)
fall-off value between 1~0, linearly or non-linearly related
to the distance of the vertex from the inner bound. An The face texture was captured by a synchronized CCD
example of a Biped’s upper arm envelope created by camera with the scan. The InSpect EM software was then
Physique is shown in Fig.4 right. After rigging, MoCap used to UV map the texture to the mesh. The texture
data in the BIP format can be loaded and applied to the coordinate information can be correctly preserved with the
character using the “Motion Capture” rollout. use of the MultiRes modifier to reduce polygon count.

2.4 MoCap Tweaking 3.2 Facial rigging

To animate basic movements for head, jaw and eyelid, a
It is often necessary to correct errors and refine the original facial bone rig was created using “Bones System” in 3ds
MoCap animation, such as smoothing jittering movements Max, as shown in Fig.6.
or correcting unrealistic limb rotations as shown in Fig.2. It
may also be necessary to fine turn the blending MoCap
clips, or modify the MoCap to allow the avatar to run along
a wall perpendicular to the ground for example. All these
can be achieved by MoCap tweaking that creates new
animation layers over the original MoCap layer. Novel
motion can be configured separately at each layer, and all
animation layers can be stored in a BIP file for future
editing or reuse in other trials. On completion of the
MoCap tweaking, all layers can be merged together to
create desired effects.

3. Facial animation

Character animation concerning emotion, style and intent

can be enhanced using facial animation. Representative
methods of facial animation include key-frame based
wrapping and morphing, parameterized models, physics
models, performance-based methods, and more recently Fig. 6 Facial bone hierarchy
image-based techniques, see reviews in [13, 14]. In this
tutorial, morphing method was used.
To rotate the bones, various controllers can be created
using circles, splines etc. The controllers are linked to the

358
bones via orientation constraints. Fig. 7 shows the skull shown in Fig. 9. Then with the aid of the ‘Morpher’ tool,
(red), neck (black), upper jaw (purple) low jaw (yellow) we blended multiple morphing targets to create novel faces.
and eyelid (grey) bones, and circle controllers used to
control the rotation of these bones. We attempted to use morphing to animate blinking eyelids.
However it caused serious mesh distortion. Therefore, we
used bones and the Skin modifier to animate eyelids, which
produced a much better result.

4. Data driven character animation

Maya, 3ds Max, Blender, Autodesk and MotionBuilder are
popular modeling and animation applications with
integrated MoCap toolkits. We chose 3ds Max for this
project because of its comprehensive industrial standard
functions for low polygon modeling, handy character
Fig. 7 Bones and controllers rigging and animation layering without using plug-ins or
scripts. Maya has similar functions to 3ds Max. However
The facial bone rigging was done using the ‘Skin’ modifier its “Full Body IK System” does not have as many easy to
due to its advanced feature of Manual Vertex Weighting. use features as the “Biped” in 3ds Max. Blender integrates
Firstly Envelopes were used to create a rough outline of the a number of advanced deformation tools good for realistic
vertices that will be influenced by the bones. The rigging organic modeling. However it is a high-polygon modeler. It
was then refined by manually assigning influence weights is possible to use MoCap data within Blender, but a large
to the vertices in the mesh. Fig. 8 shows the region of the amount of scripting is required since there are no built-in
mesh influenced by the jaw bone. Areas shaded in varying tools for MoCap data handling. Autodesk MotionBuilder
shades of red/orange/yellow indicate decreasing influence, PLE is a freeware featured on MoCap. It could be a good
and areas shaded in blue indicate the fall-off and weak choice for home learners.
influence.
In the past, MoCap data was limited to studio access.
Nowadays free MoCap databases are available to the public
[16, 17]. Some of the MoCap data used in this project were
captured from a Vicon system installed at (omitted for
blind review). Some are obtained from freely available
MoCap databases [16]. We have applied a wide range of
MoCap data to a number of avatar models. Some examples
are present below.

4.1 MoCap animation 1: Humanoid Robot

Fig. 8 Weighted rigging of the lower-jaw bone
The intention of this animation was to evaluate: 1) how
3.3 Morphing MoCap data can be used on a mechanical character with
limb proportions completely diffident from those of the
MoCap performers. 2) how MoCap clips, captured at
different location/orientation and from different subjects,
can be blended seamlessly.

The humanoid robot is shown in Fig. 10. Physique modifier

was used for rigging. Several MoCap clips, e.g. walking,
running, jumping and head scratching, were downloaded
from [16]. MoCap blending was achieved in 3ds Max using
Fig.9 Neutral face with three morph targets
“Motion Flow Mode”. MoCap clips were linked with
specified transitions and/or interpolated in-betweens. These
To animate facial expressions, morphing technique was
transitions were saved as scripts for reusing in other
employed to transfer geometry between morph targets.
animations. When MoCap data was applied, by default the
Ideally morph targets of expressions should be created from
character was positioned at the origin. We must reset a start
scan data, and the scan data must be meshed using the same
location and rotation, especially if scene objects have
structure. There is still a lack of software tools for
already been created or MoCap clips need to be blended at
automatic and precise mesh fitting. In this case, the “Soft
different locations. To avoid un-characteristic deformations
Selection” tool was used to modify vertices in the scanned
of the robot, we need to carefully refine the rigging process.
neutral face to create scary, frown and smile targets, as

359
 create hair animation without using procedural techniques,
and 3) simulate non-rigid motion for cloth.

Modeling: The character mesh used in this animation was

also freely downloaded [19]. We rigged the mesh with
Biped for MoCap animation. We added hair and cloth
features to the mesh model. To speed up rendering, the hair
and dress were modeled using polygons rather than
procedural methods. The hair geometry was created from
Fig. 10 Rendered still robots from animation several small mesh segments, attached together to produce
a layering effect, as shown in Fig 12 top. Transparency
mapping [20] was then used to merge layered textures of
4.2 MoCap animation 2: Male Dancer diffuse, alpha and specular to produce a realistic look for
the hair, as shown in Fig 12 bottom.
The intention of this animation was to: 1) animate a
downloaded, pre-rigged mesh, 2) modify the mesh with
procedural hair and shading, and 3) create facial
expressions using key-framing official bones.

Modeling: The character mesh used in this animation was

freely downloaded [18], as shown in Fig.11 left. It was
fully rigged with a Biped and additional bones to control
facial movements. Modifications were performed including
creation of procedural hair and using shaders for the
character’s skin and clothing, as shown in Fig.11 right.

Fig. 12 Polygon hair (top) and fully transparency mapped

hair mesh (bottom)
Fig.11 Original model (left), procedural hair (middle) and
Male Dancer (right)

Body and facial animation: The animation was created

using the combination of MoCap, keyframe and procedural
animation techniques. Two MoCap clips were blended
together to animate the character walking and then dancing.
Multi-layer tweaking was applied to fix incorrect limb
rotations and to prevent limbs from passing through one
another. Keyframing of facial bones was used to generate
various face gestures. Expression animation was created via
morphing. Fig. 13 Rendered character in motion (left), dress rigging
(middle) and collapsing dress (right)
Dynamic hair: A procedural modifier: “Hair and Fur” was
used to produce realistic looking hair, and to simulate hair Dress dynamics: Cloth dynamics was generated by rigging
dynamics caused by gravity and body movement. However, the dress to the thigh bones of the Biped using the Skin
it was found that the use of procedural modeling is modifier and Manually Vertex Weighting feature of the
computationally expensive and extremely slow for Skin modifier to provide more control over individual
rendering, about 11 hours on a 3.2 GHz Pentium Core2Duo vertices. Rigging of the lower dress to the left thigh is
with 2 GB of RAM, and an Nvidia 8800 series graphics shown in Fig.13 middle. However the dress mesh may
accelerator card. incorrectly collapse into itself due to fast and close leg
movements, as shown in Fig.13 right. To solve this
4.3 MoCap animation 3: Female Dancer problem, we rigged parts of the dress to the Biped’s spine
and calf bones with small weightings. A smooth, fluid
The aim for this animation was to: 1) rig a downloaded motion simulation was achieved. The dress moves
mesh using the Skin modifier, instead of Physique, 2) realistically corresponding to character’s legs without using
complex deformation modifiers. However, at some points

360
 in the animation, the character’s legs still pass through the References
dress due to ineffective collision detection, which would
require further fine turning to rectify. [1] C. Bregler, Motion capture technology for
entertainment, IEEE Signal Processing Magazine,
Body motion: The animation was created by blending 24(6), 2007, 158-160.
walking and dance MoCap clips, similar to the Male [2] D. Sturman, A brief history of motion capture for
Dancer. Keyframing and tweaking techniques were applied computer character animation, SIGGRAPH 94: Course
to the MoCap trials to remove MoCap errors and to 9.
incorporate some personalized features. A rendered image [3] M. Gleicher & N. Ferrier, Evaluating video-based
from the animation is shown in Fig. 13 left. motion capture, Proc. of Computer Animation, 2002,
75-80.
Another important topic of MoCap driven animation is lip- [4] T. Moeslund, A. Hilton & V. Kruger, A survey of
syncing facial animation, which has not been addressed in advances in vision-based human motion capture and
this project. We will work on this in the future. analysis, Computer Vision and Image Understanding,
104(2), 2006, 90-126.
4.4 Environmental modeling [5] L. Wang, W. Hu & T. Tan, Recent developments in
human motion analysis, Pattern Recognition, 36(3),
The ground was modeled using simple flat polygons and 2003, 585-601.
textured with a diffuse map. In ‘Humanoid Robot’ trial [6] J. Richards, The measurement of human motion: a
differed from the other two, a fog atmospheric effect was comparison of commercially available systems,
generated on the surface of the ground plane. This was Human Movement Science, 18 (5), 1999, 589–602.
achieved using a thin layer of mist, which both adds depth [7] D. Washburn. The quest for pure motion capture.
to the surface and compliments the rest of the scene. The Game Developer, 8(12), 2001, 24–31.
sky was constructed from a hollow sphere and then UVW [8] Vicon Motion System, www.vicon.com.
mapped with a sky. The lighting effect was provided from [9] Motion Analysis Corporation,
an omni-directional light source. To make the light appear www.motionanalysis.com.
more like a sun, a lens flare effect was applied to the [10] B. Li, Q. Meng & H. Holstein. Articulated pose
camera. identification with sparse point features. IEEE Trans.
Systems, Man and Cybernetics, 34(3), 2004, 1412-
5. Conclusion 1422.
[11] A. Kirk, J. O'Brien & D. Forsyth. Skeletal parameter
MoCap has been proven an effective way to generate estimation from optical motion capture data. IEEE
expressive character animation. MoCap technique has been Conf. on Computer Vision and Pattern Recognition,
widely recognized by animators. Handy toolkits for MoCap 2005, 782-788.
handling have been integrated within animation software [12] H. Shin, J. Lee, M. Gleicher & S. Shin. Computer
by the developers. However, many issues still exist. puppetry: an importance-based approach. ACM
Currently, most MoCap systems are very expensive and are Transactions on Graphics, 20(2), 2001, 67-94.
not affordable for average users. MoCap data [13] J. Noh & U. Neumann. A survey of facial modelling
standardization (e.g. marker attachment protocol, format) and animation techniques. Technical report, Univ. of
and usability need to be addressed. Comprehensive MoCap Southern California, 1998.
databases including a wide range of motion categories and [14] Z. Deng & J. Noh, Computer Facial Animation: A
configurations are still very limited. Motion synthesis from Survey, Data-Driven 3D Facial Animation, Springer
a restricted number of MoCap samples is highly expected London, 2007
to extend applications of MoCap. Great strides on the [15] Inspeck Inc, http://www.inspeck.com
development of new and improved methods for capturing [16] Universidade Lusófona MovLab,
and processing MoCap data, motion retargeting, tweaking, http://movlab.ulusofona.pt/FilesMocap.php?categoria=
deformation handling and real-time rendering, are 1
demanded to make MoCap a more viable tool for animation [17] CMU Graphics Lab Motion Capture Database,
production. In the future, mocap.cs.cmu.edu.
[18] Highend3d, “Character Rig With Facials and
Morphs”,http://www.highend3d.com/3dsmax/downloa
Acknowledgement ds/character_rigs.
Some MoCap data used in this tutorial were captured by a [19] 3dtotal, http://www.3dtotal.com
optical MoCap system — Vicon 512, installed at the Dept. [20] Ordix Interactive Tutorial, Transparency Mapping,
of Computer Science, Univ. of Wales, Aberystwyth. www.ordix.com/pfolio/tutorial/hair/transpmap01.ht

361

View publication stats