Animation, of course, is vastly more complicated than bitmaps. Even so, animation layering offers modern animators some of the same benefits that bitmap layers gave to painters -- freedom to experiment, ease of working with dense data, and the ability to keep the complex texture and nuance of the original source material.
Layers let you apply targeted fixes on top of dense data using the kind of simple and elegant curves you'd want from a traditional rig.
For example, if you like the base motion from a mocap file but want to exaggerate the arc of a swinging arm, you could key just the offsets into a new layer to exaggerate the motion without having to touch all the keys in the original arc.
You'd only need a handful of keys to provide the extra oomph, and you wouldn't be responsible for touching the zillions of keys in the underlying move. And just as in Photoshop, you could keep your fix around to make it easy to tweak, maybe dial it up or down with a weighting value, or, once you were satisfied, collapse it down onto the original animation for a simpler scene.
This all sounds pretty attractive. Unfortunately, you have to commit to some serious study if you want to access this kind of power. Animation tools are opaque at the best of times, and the additional complication of non-linear animation blending on top of the familiar hassles of curve management and rigging makes for an intimidating mix.
It doesn't help much that 3ds Max's and Maya's animation layering tools are both afterthoughts, tacked onto existing systems rather than designed from the ground up. They both leave a lot to be desired in the UI department. (Softimage XSI's approach, thankfully, is more neatly integrated. See Figure 1.)
Despite the differences in UI, the basic strategy is the same in all three packages. You start with some dense source data. Usually it's mocap data, but it could be a baked simulation, a ragdoll sequence imported from Endorphin, or even conventional animation that's been collapsed down to pure FK keys for convenient sharing.
The "layers" of animation you add are either adding to or just replacing the translation and rotation numbers baked into your original data. It's not wizardry at all, although like everything in the animation world, it does take some doing.
For example, if your source animation is a bit anemic, you could add exaggerations to the key poses by keying offsets (add a bit to the X rotation here, take a bit off the Z translation here ... ).
In other cases, you might need to completely replace the underlying animation. You can do this by adjusting the relative weights of the new and old animation layers, or by brute force by going into the original keys and zeroing out the bad bits then adding better keys on a new layer. It's conceptually pretty simple. The complexity arises from managing all the different fixes and layers without going crazy.
Of course, straight layering is still old-school FK animation. The skeptics are probably wondering why you'd want to go to all this trouble if you also have to give up goodies like constraints and IK -- but thankfully, you don't have to forego the finer things. You can simply add your IK or constraints into new layers.
Well, "simply" might be overstating it, thanks to the Neanderthal UI, but it's definitely possible to drive the final character with a combination of baked FK keys, constraints, IKs, and expressions.
However, if you go down that road very far, you'll face a dilemma that has haunted riggers ever since the first caveman IKed a club to the head of a giant ground sloth. In the conventional rigging world, rigs that allow for possible tweaks and offsets are sometimes too complex to use effectively, but rigs that are too stripped down are sometimes too limiting.