Introduction

Much of the work in game AI has focused on the ‘big' problems: path planning, squad planning, goal-directed behavior, etc. The result is characters that are capable of increasingly intelligent behavior. However, acting intelligently and acting aware and sentient is not the same thing. But if we are to create the kind of compelling and emotional characters upon which the next generation of computer games will be based, we must solve the latter problem, namely how to build characters that seem aware and sentient.

An important theme of the work of the Synthetic Characters Group at the MIT Media Lab was to understand and build the kind of AI needed to create a sense of an inner life. Our belief, presented most cogently by Damian Isla, was that this sense of an inner life arose not out of the large motion and behavior of the character but out of what Isla termed the low-level motion and behavior.

Examples of such behaviors include: the shift in gaze, and widening eyes as a result of perceived motion just on the periphery, the slight stiffening of a cat's tail that presages a predatory pounce, the slinking gait of a fearful dog in expectation of being punished; the catch in breath in response to a startling noise. In traditional animation, these movements and behaviors would be labeled as secondary anticipatory actions. And yet, as Isla puts it, “much of the low-level animation described is significant precisely because it is indicative of some kind of emotional or knowledge state internal to the character. If a character frowns and continually glances towards a door, we might infer that it is because the character is anxious about someone soon coming through it.” [Isla]

	Damian Isla's work on expectations addressed the problem of forming and validating expectations about the likely location of objects that have been observed in the past, but aren't currently visible. His dog would show surprise, confusion, satisfaction, or frustration based on whether the world matched its expectations.

The shift in glance conveys that the character is aware of its environment, that it possesses an expectation about what is to happen, and an attitude with respect to the event as to whether it is going to be good or bad. All of which serve to create a sense of a plausible and comprehensible inner life.

In this article, we wish to focus on “anticipatory AI”, that is, the AI needed to support the anticipatory behaviors that prepare the eye and the mind of the observer for what is to follow because it is these very behaviors that are at the crux of building convincing and compelling characters. We begin by discussing why anticipatory behaviors are so important in nature, as well as, in animation. We will then focus on 3 types of behavior that serve this function of preparing the observer for what is to follow, and discuss the AI implications of each. These include:

1. Making perception perceivable: what a character is observed to perceive gives cues as to what it will do, and why, and ultimately, what it will feel.
2. Making expectations perceivable: from perception to action to outcome, a character's expectation with respect to what they are observing, the expected outcome of their actions, and their attitude with respect to the actual outcome of their actions should be made manifest to the observer.
3. Making impending changes in motivational state perceivable: dramatic changes in motivational state should be telegraphed to the observer so as to prepare them for the change.

Anticipation: Preparing the Eye and the Mind

The importance of anticipatory movements, or more generally anticipation was one of the earliest and most important discoveries of classical animation. To paraphrase two of the pioneers, Frank Thomas and Ollie Johnson: by anticipating the action, the animator allows the audience to focus on how the action is being done rather than in trying to figure out what is being done. In other words, anticipatory movements prepare the eye and the mind of the audience for what is to follow. [Thomas]

As with many of the rules of animation, anticipation has its roots in nature. For example, anticipatory movements may arise out of the physics of the movement like the wind-up before the throw, or the coil of a rattlesnake before it strikes. Or they may arise out of behavior, like the retraction of the lips that precede a snarl, or the raised hackles on the back of the neck, both of which signal an impending attack. Indeed, some anticipatory movements are so predictable that animals use them as so-called “intention movements” to predict what another animal intends to do.

The biological value of attending to intention movements is clear. By doing so, animals can reliably predict what another animal is about to do and plan its response proactively. In some cases, this may allow it to avoid conflict, in other cases it may allow it to predict the zag in the zigzag and meet the prey mid-zag.

We are no different than other animals in our conscious and subconscious reliance on the perception of anticipatory behaviors as important cues into the mind of other beings, be they other people, animals or animated characters.

	In "Alpha Wolf," we focused on making the emotional state of the wolf pups manifest to the user via expressive motion, posture and rendering.

Anticipation: Making Perception Perceivable

One of the most important and telling anticipatory behaviors is that of perception. Virtually all animals orient before acting so as to better perceive the focus of their attention. Cats will move their ears in response to a noise before visually orienting. Dogs, being highly olfactory will sniff the ground or the air. Seeing animals perform behaviors such as these provides us with important cues as to what they are attending in the world. In addition, the manner in which perceptual behaviors are performed often provides clues about how the animal feels about what perceives, or expects to perceive. We then use these cues to predict what the animal is going to do and/or fee next.

Since the characters that populate our games are almost always visual creatures, their visual behaviors from gaze to glance are typically the most important cues that people use to infer what a character is thinking, and about to do. Indeed, these behaviors are the canary in the mine when it comes to providing the foundation for the appearance of motivated behavior. Get it right and you are 80% there. Get it wrong and no matter how good the animation, the subsequent behavior will not seem motivated.

In our work at the Media Lab, we typically made the AI dependent on information that was acquired through the “look-at” so as to force ourselves to get it right. Indeed, we went so far as to implement synthetic vision in a number of our systems. That is, the scene would be rendered from the perspective of the character, and relevant information would be extracted from the resulting image via image processing. By encoding information in the image via false coloring, and by taking advantage of the power of today's generation of graphics cards, this approach was remarkably fast and powerful.

This sensory honesty, as Burke and Isla called it, insured that the animal would only act on what it could realistically sense. In addition, because the “look-at” was a fundamental part of the system as opposed to a cosmetic effect, it was a priority for everyone to ensure that the behavior was correct, and as a result, it was both believable and compelling.

Making the perceptual acts observable and believable to the audience is the first thing to get right with respect to creating a sense of an inner life. Seeing the turn of the head, the cock of the ears, the sniff of the nose is more than simply a cosmetic effect. A system may model the diffusion of a scent trail but if the user doesn't see the simulated dog sniff the ground and cast back and forth in order to acquire the scent trail, the resulting behavior will come across as artificial. It is especially important that the character react to those perceptual events that the user expects to be salient to it. For example, a character should start when it hears a sudden noise unless there is a good reason for it not to.

Before we move on, there are three quick points to make. First, making perception perceivable to the audience is more than just a character issue. As the Disney animators know, it has important implications for staging (how the user's eye is directed to the important action in the frame), for example, via a close-up camera shot. Second, what a given character attends to in a given situation is highly character and context specific. An audience will expect a young recruit in his first battle to attend to completely different stimuli than the grizzled sniper whose focus is exclusively on her next target, having seen it all before.

Thus, what a character appears to perceive must match the audience's expectation of what such a character should perceive given its personality and context. Third, how the perceptual acts are performed is every bit as important as performing them in the first place. It is in the “how”, that is, in the quality of motion that the character's expectations are revealed to audience. That is, what they expect to see and how they expect they will feel when they see it. And this brings us to the second topic, namely making expectations perceivable.