Constructing Artificial Emotions: A Design Experiment

By Daniel Cook

Bacchus: The Orgiastic Ritual Game

Bacchus is a multiplayer dancing game with a religious theme. The selling point is its ability to evoke intense emotions.

Imagine if you will, a decrepit theater filled with writhing, dancing people. The lights flare and swoop in time and the people chant in unison. A massive screen shows a mirror image of the hall like some surrealistic portal into an alternate universe. Instead of blokes and lasses in street clothes, the onscreen spirits are clad in ornate ritualistic garb. The movements on each side of screen are eerily synchronized. The pitch of the chant rises.

The screen zooms in on a girl in the center of the room. The crowd, as one, turns and watches her figure on the screen. She begins to dance. At first her movement is controlled and intricate. The screen pulsates and she yells to its beat. The room takes up her words and amplifies them, giving them god-like resonance. Bass mixed with reverb mixed with primal, guttural passion. Her dance becomes wild. The pace increases and she begins to confess.

The theater reacts. Each word she utters shimmers on screen, merging with ghostly photos from her past. In a beat, the entire room witnesses her sorrow over the death of her mother, her time alone in an empty apartment, and her first kiss. An inhumanly beautiful electronic chorus rises, matches and turns her words into a song. Her movements become a blur. Her glowing eyes are ecstatic. At the peak, her spirit on the large screen explodes in light and the girl collapses to the floor in fervent religious swoon.

The crowd goes wild. The screen zooms out and the next god dancer is chosen.

Later, the girl writes to her online friends that the night she danced was the single most powerful spiritual and emotional experience in her entire life. It was the night she was touched by a higher power while playing a video game.

Your browser may not support display of this image.
William-Adolphe Bouguereau - The Youth of Bacchus (1884)

Bacchus: The Gedankenexperiment

The game Bacchus is a thought experiment, not a real game. It exists merely to explore, in one design, several effective, yet rarely-used techniques for inducing emotion through gameplay. It happens to have a religious theme, but I’m primarily interested in exploring how designed experiences can yield intense player emotions.

The game designer’s palette of emotion has traditionally been limited to boredom, frustration, and triumphant mastery. There is very little published research on how to evoke a broader range of emotions and designers have very few practical or theoretical tools at their disposal in the quest to create meaningful, emotional experiences for their players. Designers interested in evoking emotion fall back on:

  1. Stealing techniques from non-game media. “And then we show a movie of the faithful heroine being stabbed by the evil villain!”
  2. Copious handwaving. “See, this pink pulsating blob represents ‘Feelings’”, explains the designer to the confused player.

The resulting experiences are far more emotionally simplistic than we might dream of creating.

To expand beyond the present constraints, I set forth a personal challenge. What if you wanted to create a game that pushes the player through a sequence of emotions, from joy to sorrow, to perhaps even religious ecstasy? What current or future techniques would you use? Is it even possible for a game to evoke a rich palette of emotions?

In order to build a game that induces such a complex emotional spectrum, we need to dig into the fundamentals of evoking emotions in games. It turns out that many folks in the scientific community have been studying tangentially related problems for quite some time.

What Is Covered In This Essay

This essay has five parts

With each technique, we’ll cover the theory, how you can put the theory to use, how technology can help, and some of the limitations.


The Two Factor Theory of Emotion

Let’s begin with some basic cognitive science. The framework I’ll be leaning on throughout our investigation of artificial emotions is a well-known cognitive theory called the Two Factor Theory of Emotion, by psychologists Stanley Schachter and Jerome Singer. The theory states that in order for an emotion to be felt, two factors must be present:

Simply put, when your body reacts physically to some stimuli and you mind assigns meaning to your physical state, you synthesize an emotional response.

An intriguing experiment

The Two Factor Theory of Emotion is certainly mildly intriguing as an analytic description of how emotion works, but it has a far more practical application in the realm of game design. In the process of proving their theories on emotion, researchers spent much of their effort on figuring out how to dissect the component aspects of emotion and reassemble them into new emotions of their choice. In effect, they figured out how to reconstitute artificial emotions within their subjects. Their experiments provide us with practical examples of how we might build our own systems of generating artificial emotion in our players.

It turns out that the physiological changes that accompany many emotions, such as fear and lust, are remarkably the same. There is a wide range of stimuli, including loud noises, intense memories or even a fear of heights that activate the sympathetic nervous system, prepping the body for action in the face of stress. Your heart rate elevates. Your palms become sweaty. Your alertness increases and body hair stands on end. Different stimuli, same response.

Due to the ambiguity of the physical response you rely on your brain to determine what all this activity actually means. Should your run, should you fight, should you laugh? In a heartbeat, you brain need to figure out what is happening and synthesize the correct response. In this moment, your carefully calibrated gray matter can be tricked.

One of the colorful experiments that demonstrate this effect was performed by psychologists Dutton and Aron in 1974. They wanted to see if they could alter the context of a situation so that the subject would instead experience lust instead of fear. In their study, a highly attractive young woman approached a sample of young men and asked them to fill out a survey. The experiment had two components.

Almost twice as many men (60%) gave the girl a call if they had been surveyed on the dangerous bridge than on the safe path. Due to the strong contextual signals in the form of presence of the attractive woman, the men misinterpreted the fear-driven activation of their sympathetic nervous system as authentic lust.

A potion for emotion

Other experiments validated the theory by inducing both happiness and anger in their subjects. These studies suggest the following general recipe for concocting artificial emotions.
Your browser may not support display of this image.

By evoking both states in the player, the mental and the physical, designers can greatly increase the likelihood that players will experience the desired emotional response to a game.

With our theory in hand, let’s look at the set of practical techniques that help us generate the artificial emotions at the heart of Bacchus.

Technique 1: Active recall of emotional memories

“Each word she utters shimmers on screen, merging with ghostly photos from her past. In a beat, the entire room witnesses her sorrow over the death of her mother, her time alone in an empty apartment, and her first kiss.”

In Bacchus, the player recalls intensely personal emotional moments as part of a public confession. It turns out that this is a great technique for evoking a both a physiological response and a set of cognitive labels.

Theory: When you experience an intense emotion, a primitive portion of the brain called the amygdala kicks in and ensures that you store a vivid, emotionally charged memory. When you recall the memory, your brain also ensures that you remember the emotional element.

Your browser may not support display of this image.

Remembering an emotional event causes that you to re-experience that associated emotion.

Your browser may not support display of this image.

For example, when war veterans recall a traumatic experience, they often experience elevated heart rate, perspiration and other signs of panic. The memory of the veteran’s traumatic event triggers a replica of the physiological response that occurred during the event. The emotional panic they feel is very real, very physical and easily measured. Other extreme examples of this phenomenon include many phobias such as fear of small spaces, flying, etc., all of which are typically rooted in some traumatic experience.


The basic system of recording and recalling emotional memories that underlie these reactions is present in every single one of us. In normal circumstances, the recall of emotional memories is a healthy and helpful aspect of basic human cognition. Storing emotions in good and bad times and then recalling them instantaneously if a similar situation occurs is a great evolutionary benefit. When you see the mountain lion the second time around, your body is instantly primed for flight. You don’t have to think or analyze the situation. You simply feel the correct course of action. The upside of all those millions of years of evolution for game developers? Your players have spent their lives collecting a deep pool of strong emotional memories that is just waiting for you to tap into with the appropriate game design.

Technology: In Bacchus, the confession is a game mechanic that encourages the players to tap into their emotional memories. Confessions are effective because the player self-selects memories with strongest, most pertinent emotional content and voluntarily shares them. For most people, this has the effect of turning on an emotional fire hose.

In order to make the recall as intense as possible, we want to augment the verbal recall with a few simple technologies. Human memory is stored like a sparse set of data points in a connected network. When a person remembers, they pull on related data points to flesh out the memory. You can increase recall by feeding the user related data points, thus lighting up more bits of their memory and increasing the clarity of the final result.

The classic example is a group of people remembering an event. A single person might remember a few key elements and have a fuzzy memory of the event. However, when a group of people gets together, each one contributes a small piece of additional information that helps light up a highly detailed map of the memory. Technology can serve a similar role.

The result of this particular system is the intense recall of very personal memories. With intense recall comes the highly desirable stronger physiological reaction.

Benefits: The main benefits of using the recall of personal emotions is that you are almost always going to get a solid emotional response, especially in newer players. You are digging up the raw materials of their most personal emotions and the results can be explosive.

Limitations: The downsides of using emotional memories to generate physiological responses are substantial.

Technique 2: Evoking emotional memories with relevant stimuli

“[T]he onscreen spirits are clad in ornate ritualistic garb… The room takes up her words and amplifies them, giving them god-like resonance. Bass mixed with reverb mixed with primal, guttural passion. An inhumanly beautiful electronic chorus rises, matches and turns her words into a song. Her movements become a blur. Her glowing eyes are ecstatic. At the peak, her spirit on the large screen explodes in light…”

In Bacchus, the player is surrounded by rich, evocative visuals and symbols. These also play an important role in priming the player to feel emotion. This technique builds upon some of the elements of emotional memories, but uses relevant stimuli instead of the blunt trauma of actual recall. We broadly evoke the player’s existing experiences with religion.

Theory: Once upon a time, I had a workmate that had an irrational fear of house plants. When she was a child, an aunt played tag with her in a greenhouse. At some point, the game stopped being a game. The girl desperately wanted the aunt to stop, but the aunt assumed the girl was still playing. The intensely emotional memory of being hunted, terrified, and surrounded by clinging, suffocating plants was seared into my workmate’s memory. This is in keeping with our discussion of emotional memories.

What I found fascinating is that recalling the specific event was not the only trigger for her phobia. Instead, stimuli peripherally connected with the memory, such as particular shades of green, a swaying vine, or a delivery of flowers with a bit too much leafy foliage on Valentine’s Day could set off a panic attack.

Emotional memories need not be triggered by recalling the exact event that embedded them. Instead, rich contextually coherent descriptions of similar situations can trigger those same pathways, though perhaps to a lesser degree. You may feel joy when you remember your first kiss with that shy girl from next door. But you may also feel joy if you hear the story of someone much like you who kisses the shy young princess. Enough nodes in your memory are triggered for you to react to the related story.

There is a rather broad cognitive theory of media at play here. In short:


It can be a strange concept to wrap your head around. When you read a romantic book, your response is as much due to your past experience with romance as it is to the contents of the book. It is very likely that another person, one who has led an unnaturally lonely life, might read the same book and not be moved at all.

As a traditional author, your goal is to describe experiences that are relevant or highly correlated with experiences that your audience might possess. This is one very powerful technique for creating meaningful, emotional impactful media. This isn’t anything new. Much of what game artists and writers do involves the creation of relevant stimuli.. All those detailed graphics, booming sound effects and cliched story lines? That’s relevant stimuli, baby.

Technology: According to our little theory and building up on the lessons of emotional memories, you can predict the characteristics of highly effective relevant stimuli:

Bacchus doesn’t rely on plot or well-rendered NPCs for its relevant stimuli. Instead it focuses on making the fantastical ‘spiritual’ experiences in the game personally relevant and highly detailed.

Your browser may not support display of this image.

The detail comes in the form of traditional imagery and sound effects that references common culturally relevant spiritual symbols. The ornate costumes worn by the player avatars are intentionally laden with various religious icons, glowing colors, angel wings, etc. The choral sounds are intended to recall childhood experiences with church choirs or even pop culture representations of spirituality found in movies or on TV.

Movies and books are typically limited to piling on the detail and hoping that it strikes a chord. Games have the opportunity to make all these details much more personal and relevant to the moment at hand.

The primary technique you see in Bacchus is known as "avatar mapping", where the player's actions are mapped onto an in-game character. It stops being about watching a costumed religions icon and much more about interacting and participating on a personal level with a religious situation.

All this is a crapshoot. If I’m lucky, I’ll actually trigger the recall of an actual spiritual experience.

Benefits: The use of related stimuli is the gold standard for inducing artificial emotions across practically every media known to man. Great novels, paintings, movies, and poems rely on their intense portrayals a human experience that is not our own, but close enough to tap into our personal experiences. As a creator, this is technique is quite cost effective and most of us have been trained in its application.

Limitations: Relevant stimuli is amazingly powerful, but has some limitations when it comes to use in games.

Technique 3: Biofeedback

"She begins to dance. At first her movement is controlled and intricate. The screen pulsates and she yells to its beat."

Not all techniques at our disposal create both a physiological response and cognitive labels. There are some that just do one or another. Such techniques can be used as building blocks in a larger system.

One well-studied technique that affects that body is bio feedback. This is particularly interesting to game developers since it is a fundamentally interactive technique and offers deep opportunities for mastery-focused gameplay.

Theory: In the classical model of human behavior, there is the somatic nervous system which controls voluntary actions like moving your arm and the autonomic nervous system which attempts to maintain homeostasis by automatically adjusting such things as body temperature or heart rate.

A surprising number of automatically controlled systems can in fact be influenced consciously. The most obvious one is breathing, as seen by pearl divers holding their breath. Other systems can be controlled indirectly by consciously adjusting related systems. For example by staying stationary, slowing your breathing and thinking calming thoughts, you can slow heart rate.

If only we could encourage the player to directly control their physiological state, they could

consciously put themselves in a state that was conducive to feeling the desired artificial emotions. Unfortunately, people are generally quite poor at recognizing and attaining mastery over systems such as the autonomic nervous system that have poorly-visible second order effects that are only loosely connected to the original action. Most people couldn’t tell you their heart rate and even fewer could tell you how they could consciously speed it up or slow it down.

We see this problem of controlling second order effects pop up in games all the time. Suppose you add a switch that unleashes an AI monster, that then steps on a switch that opens a door off screen. The end result is that users complain that they have no idea why things are happening. The chain of events between cause and effect is too long and confusing for the user to form a testable mental model of the system.

In order to teach the user how to control second order interactions, the game designer has to provide lots of clear, concise feedback and plentiful rewards for the right actions. Biofeedback applies these game design principles to the task of influencing the autonomic nervous system.


At the heart of Bacchus are the biofeedback monitors that each player uses.

Technology: All of these are additional control mechanisms for the game that can be used to help facilitate the feeling of emotions. Instead of pressing a button to advance the game, the player instead focuses on putting their body in a state that statistically correlates with happiness. The game recognizes the hint of a smile, the increased heart rate, the increased pitch of the repeated phrases and the avatar on the screen responds with grand flourishes of sparkles and other visible indications of success. We use biofeedback to create short, tight feedback loops that reward the activities we, as the designer, desire.

First-time players would simply be astonished that the game knows that they are feeling bored or irritable. More advanced players know that the pulsing lights on the big screen are meters showing them key indicators of their physical state. They use these as feedback that guide them towards reaching the appropriate state to enjoy the game.

Limitations: There are several major limitations of biofeedback as a control mechanism

Technique 4: Setting social norms

“Later, the girl writes to her online friends that the night she danced was the single most powerful spiritual and emotional experience in her entire life. It was the night she was touched by a higher power while playing a video game.”

Ultimately, the player attempts to understand the maelstrom of experiences that they’ve undergone during a night of playing Bacchus. The context of the event matters immensely. Someone who sees Bacchus as just a game will have very different memories of the event than someone who goes into the evening expecting a holy experience.

In order for the designer to affect the critical step of synthesizing desired emotions, we need systems that ensure the player has access to the correct cognitive labels. By influencing the language players use to comprehend an experience, you can control how they end up remembering the experiences. One of the more powerful techniques for ensure people use the language you desire is taken from the propagandist / change agent’s cookbook: the small group discussion.

Theory: In the 1940s, Edward Schein, one of the founding fathers of organization psychology, was brought in to help the government market its rationing plans to the public. In particular, they were interested in convincing people to eat ‘sweetmeats’, the indescribable innards of animals that were typically tossed into the rubbish heap.

Schein conducted two experiments. The first was a traditional lecture that described all the benefits of sweetmeats in terms of nutrition, patriotism, etc. The audience listened attentively and then filled out a survey asking if they would change their consumption habits. Only a small percentage agreed to try sweetmeats in order to help the war effort.

In the second experiment, the format changed. Schein gathered together small groups of people and sat them down with a facilitator to discuss the option of eating sweetmeats. The facilitator presented the topic and periodically interjected facts that might help the discussion or clarify misconceptions. Most of the conversation, however involved people talking about their fears, their questions and the group weighing together the benefits of sweetmeats. When this same group filled out the survey, the vast majority said that they would change their eating habits and start cooking with sweetmeats.

Schein’s critical observation of this process is that that new groups go through a process in which they set group norms, expectations of social behavior and common beliefs. In the second experiment there was ample time for the group to negotiate the new set of norms. In the first, due to the one way communication, there was no opportunity, so the audience left with their existing beliefs intact.

This process of creating new social norms happens remarkably quickly and can result in people doing things at the end of the process that they would never have contemplated before the process begins. We like to think that our behavior and beliefs are fixed, immovable and formed by great deliberation and moral character. In reality, they are heavily influenced by the normative behavior of the group we participate in. Given the right group circumstances, the behaviorial code of most individuals can be rewritten to an amazing degree. You can witness this daily in most corporate environments, but more extreme examples are the tales of brainwashed prisoners of war or even the brutal behavior of U.S. troops when placed in an environment like Abu Ghraib.

 


Small group discussions can be used to seed a group with positive new concepts. You can build up a vocabulary of new cognitive labels that are then triggered at a later point during gameplay.

Technology: By creating controlled social environments, we can encourage and influence the norm-setting process. Online games with their absolute control over social connections, language filtering and feedback mechanisms offer an ideal voluntary environment for norm resetting. Concepts, terminology and even desired behavior can be seeded in a group by putting them together, present those new ideas and discussing them in a positive, constructive light.

Imagine that our Bacchus players have an online hangout. They meet up after each bout of group dancing. They discuss what they’ve felt, discuss concerns and how they can improve. A higher level character acts as a moderator and feeds the group more refined descriptions about what they have experienced. It is, in many ways, no different than a Bible study group. The setting provides an effective manner of setting up the context for the next encounter and seeding the players with language to describe the physical experience in emotional or spiritual terms.

Benefits: If you look at the social trends in the United States, a growing percentage of the population prefers to live alone. This naturally limits the amount of ambient socialization by that individual. They have no spouse to share the events of the day with, nor do they know their neighbors. Outside of the workplace, community and culture has less and less practical meaning to the modern man. If these isolated individuals turn to online communities in order to fill out their social network, there is an enormous opportunity to create a new set of designer virtual norms that are rarely if ever challenged by outside forces. The individuals that buy into the game will behave according to the standards of their dominant social group, fellow gamers dancing through life in an artificial, designer-manipulated culture.

Limitations: One of the biggest limitations with using small group discussions is that norms set in the group tend to deteriorate outside of the context of a group. You can see this phenomenon occur with the ubiquitous company offsite. A team comes together in a new environment and agrees to change the world and their behavior. As soon as they get back to the office, the hothouse atmosphere of the offsite dissipates and the regular rhythms and expectation of families, bosses and existing processes take hold again.

It takes repeated indoctrination, especially at a young age, to embed social norms deep enough that they can be relied upon to the produce the desired results. Isolating the group from external normative forces is also highly effective. There is a good reason why cultists pragmatically isolate their believers in walled-off compounds.

The other issue that comes up with setting up a new collection of social norms is that existing groups react defensively against those who step out of line. Protection of group boundaries is an impressively powerful social force that must be tampered with using great care. A title like Bacchus, with its overt religious theme and focus on resetting social norms, would likely raise at least an eyebrow or two.

The near future

The four techniques demonstrated in the Bacchus design will hopefully provide some food for thought. The question that occurs to me is “how realistic is any of this?” Let’s put aside the navel-gazing for a moment and look into the future with our crystal ball.

Technology: Some of the technologies I’ve discussed are already beginning to make their way out onto the market. We’ve seen a couple of generations of video cameras built into consoles. Voice recognition software is readily available to PC users. The Wii and the Wii Fit balance board are continuing the trend towards more physical game play. It is a natural evolution to add heart rate and skin conductance monitors. As the years unfold, there is immense opportunity for hardware designers to differentiate their platforms by increasing the accuracy with which games can track the player’s conscious and unconscious actions.

Game design: Though the availability of the appropriate technology does not worry me much, the lack of proven, field-tested game design techniques for inducing emotion does. Bacchus is a thought experiment and though it may stimulate discussion, I hold no illusions about its practicality as a blueprint for a working title. For emotional game design to truly blossom, there are several obvious areas of investment.

The techniques I’ve mentioned in Bacchus are really just the tip of the iceberg in terms of possible design tools. There is a wealth of research on using intense sounds or visuals, sleep and food deprivation and of course, various drugs to alter the player’s physical state. Every year, novel and effective biometric techniques continue to improve in accuracy, cost and usability in the field. Setting appropriate cognitive labels is perhaps less well studied, but we can draw heavily upon the realms of advertising, propaganda and organizational psychology.

At first blush, the success of a game design like Bacchus seems bizarre and unlikely. Yet, when I look out into the market and see the success of emotionally rich games as diverse as World of Warcraft, Spin the Bottle or Survivor, it seems that such games are inevitable.


Conclusion

Let’s review what we’ve covered so far.

Each of these techniques attempts to use applied psychology to evoke artificial emotions. This is a fundamentally different tactic than you find used by most novelists, scriptwriters or musicians. It is worth exploring further. Instead of looking at emotion in media as a reflection of the artist’s internal muse, we can treat the player’s emotion as a system that we can model, interact with, and through the use of strong feedback systems, push toward desired states.

To simplify the situation immensely, most media, be it music, movies or books taps into emotion by rehashing pre-existing experiences. Games, though they may fall back on rehashed experiences occasionally, are uniquely capable of creating new emotionally powerful experiences. In a novel, you can read about someone falling in love. In an MMOG, you can actually fall in love. Real experiences generate vivid, new emotions.

Here is a thought. When trying to create emotion in your players, tone down with the fixation on Hollywood, camera techniques and in-game narrative. It isn’t our unique strength as a medium. Instead, explore what would happen if we, as designers, actively attempted to create and manipulate the social, psychological and physical environments of our players in order to induce artificial emotions. Toss the storyboards and scripts. Game design becomes an exercise not so dissimilar from the movie The Truman Show. You provide the carefully balanced system that sets up the appropriate physiological states and cognitive labels. The players react with predictable, measurable human drama.

In this brave new world of emotional experiences, you design interactive systems that play the player like an instrument. Except instead of tunes, they are belting out tears.

Take care,

Danc.


References And Wry Comments

Two factor theory of emotion

The fascinating topic of Psychophysiology

What is the effect of certain cognitive stimuli on the human body? This is a topic fundamental to game design and happily for us, there is an existing body of work in the academic world. As with most such areas, existing research is focused on medical issues as opposed to entertainment.

Techniques for measuring emotional arousal

How the mere memory of electric shocks makes hearts race

Triggering increase salivation through detailed descriptions

Here is an example exercise that demonstrates how a detailed description can trigger a physical reaction due to relevant stimuli. It is somewhat more controlled than your typical movie-goers experience, but the same principles apply.

Instruct another person to close their eyes and then read her the following passage. Ask her to focus on visualizing each detail as clearly as possible. Some subjects experience increased salivation and report being able to ‘taste’ the lemon.

Emotional memory

Can games make you cry?

Everyone’s favorite clichéd design topic. The Wired article demonstrates our strong reliance on relevant stimuli.

Reality television is an emotional game

Reality television shows are masters of experiential games that evoke emotions in their players. If you are deeply curious about how this works, I might recommend watching any episode of any season of The Bachelor. There will be tears.

Sympathetic nervous system

Experimental recreation of out of body experiences

Relationship of relevant stimuli to red herrings

In my previous essays I’ve discussed skill atoms and skill chains. Relevant stimuli are represented in skill chains as a ‘red herring atoms’, a set of player triggered stimuli that evoke experiences outside of the game.

http://www.gamasutra.com/view/feature/1524/the_chemistry_of_game_design.php

Return to the full version of this article
Copyright © UBM Tech, All rights reserved