Now
for the Animation
I
finally have my system set up and my models created. It is time to create
some real-time animation. The time-tested animation production method
is to take a track of audio dialog and go through it, matching the visemes
in your model set to the dialog. Then, in second pass, go through it
and add any emotional elements you want. This, as you can imagine, is
pretty time consuming. Complicating the matter is that there are not
many off-the-shelf solutions to help you out. The job requires handling
data in a very special way and most commercial animation packages are
not up to the task without help.
Detecting
the individual phonemes within an audio track is part of the puzzle
that you can get help with. There is an excellent animation utility
called Magpie Pro from Third Wish Software that simplifies this task.
It can take an audio track and analyze it for phoneme patterns you provide
automatically. While not entirely accurate, it will at least get you
started. From there you can manually match up the visemes to the waveform
until it looks right. The software also allows you to create additional
channels for things such as emotions and eye movements. All this information
can be exported as a text file containing the transition information.
This in turn can be converted directly to a game-ready stream of data.
You can see Magpie Pro in action in Figure 6.
|
|
|
Figure
6. Magpie Pro simplifies the task of isolating
phoneme patterns in your audio track.
|
Wire
Me Up, Baby
With
all the high-tech toys available these days, it may seem like a waste
to spend all this time hand-synching dialog. What about this performance
capture everyone has been talking about? There are many facial capture
devices on the market. Some determine facial movements by looking at
dots placed on the subject’s face. Others use a video analysis method
for determining facial position. For more detailed information on this
aspect, have a look at Jake Rodgers’s article “Animating Facial Expressions”
in the November 1998 issue of Game Developer . The end result
is a series of vectors that describe how certain points on the face
move during a capture session. The number of points that can be captured
varies based on the system used. However, typically you get from about
eight to hundreds of sensor positions in either 2D or 3D. The data is
commonly brought into an animation system like Softimage or Maya and
the data points drive the deformation of a model. Filmbox by Kaydara
is designed specifically to aid in the process of capturing, cleaning
up, and applying this form of data. Filmbox can also apply suppressive
expressions, inverse kinematic constraints, and perform audio analysis
similar to Magpie Pro.
This
form of motion capture clearly can speed up the process of generating
animation information. However, it’s geared much more toward traditional
animation and high-end performance animation. In this respect it doesn’t
really suit the real-time game developer’s needs. It’s possible to drive
a real-time character by using the raw motion capture data to drive
a facial deformation model. However, for a real-time game application,
I do not believe this is currently feasible.
In
order to convert this stream of positional data into my limited real-time
animation system, I would need to analyze the data and determine what
visemes and emotions the performer is trying to convey. You need a filtering
method that will take the multiple sample points and select the viseme
or muscle action that is occurring. This is really the key to making
motion capture data usable for real-time character animation. This area
of research, termed gesture recognition, is pretty active right now.
There is a lot of information out there for study. However, Quantumworks’
Geppetto provides gesture recognition from motion capture data to drive
“muscle macros” as both a standalone and a plug-in for Filmbox.
Where
Do We Go from Here?
Between
viseme-based and muscle-based facial animation, you can see that there
are a lot of possible approaches and creative areas to explore. In fact,
the whole field has really opened up to game development in terms of
opportunities for game productions as well as tool developers. Games
are going to need content to start filling up those new DVD drives and
I think facial animation is a great way to take our productions to the
next level.
For
Further Information
•
Ekman, P. and W. Friesen. Manual for the Facial Action Coding System.
Palo Alto, Calif.: Consulting Psychologist Press, 1977.
•
Faigin, Gary. The Artist’s Complete Guide to Facial Expression.
New York: Watson-Guptill Publications, 1990.
•
Goldfinger, Eliot. Human Anatomy for Artists. New York: Oxford
University Press, 1991.
•
Landreth, C. “Faces with Personality: Modeling Faces That Exude Personality
When Animated.” Computer Graphics World (February 1996): p. 58(3).
•
Waters, Keith. “A Muscle Model for Animating Three-Dimensional Facial
Expression,” SIGGRAPH Vol. 21, N. 4 (July 1987): pp. 17-24.
Facial
Animation
http://mambo.ucsc.edu/psl/fan.html
Gesture
Recognition
http://www.cs.cmu.edu/~face
Performance
Animation Society
http://www.pasociety.org
Magpie
Pro
http://thirdwish.simplenet.com
Filmbox
http://www.kaydara.com
Geppetto
http://www.quantumworks.com
Acknowledgements
Thanks to Steve Tice of Quantumworks Corporation for the skull model
and the use of Geppetto as well as insight into muscle-based animation
systems. The W. C. Fields image is courtesy of Virtual Celebrity Productions
LLC (http://www.virtualceleb.com)
created using Geppetto. The female kiss image is courtesy of Tom Knight
of Imagination Works.
When
not massaging the faces of digital beauties or doing stunt falls in
a mo-cap rig, Jeff can be found flapping his own lips at Darwin 3D.
Send him some snappier dialogue at jeffl@darwin3d.com.
________________________________________________________
[Back
to] Decisions, Decisions
|
|
|
|
Features
