Another important factor players consider in choosing between alternatives is risk. Game theory says that players should weigh the options such that they'll maximize overall reward in the long term. For each alternative, they should multiply the possible reward by the odds of receiving that reward and choose the best option.
However, this article is concerned with what players actually do, not what they mathematically should do. Psychologists generally use two terms to describe how subjects react to risky situations. Subjects are risk-prone when they prefer the more uncertain alternative and risk-averse when they tend towards safer options. In one experiment, pigeons were offered a choice between two keys to peck. The left provided 8 pieces of food every time, the right provided 16 half the time and no food half the time. The pigeons consistently preferred the more reliable schedule, and were therefore risk-averse. In a later study, the left key produced 3 bits of food every time while the right key produced 15 one-third of the time. In this study, the pigeons preferred the riskier alternative.
So far, this is perfectly in accord with game theory, with subjects taking risks when those risks offer an overall greater chance of reward. But what about the example mentioned earlier in this article, where subjects preferred a variable interval schedule to a fixed interval schedule? Even when the two options provided equal rates of overall reward, subjects preferred the probabilistic option. The difference lies in the expected outcome of each individual response. In the pigeon experiment we just described, each choice was discreet. A peck, an outcome, and the subject was presented with a fresh choice. Each choice contained the totality of possible outcomes, so the subjects' behavior reflected the total contingency.
In the fixed-interval / variable-interval experiment, one could respond any number of times on the fixed interval option but would not receive the reward until the interval had elapsed. On the variable interval schedule, every single response had a small chance of being rewarded. Therefore, there was always a reason to try the variable schedule, but only occasionally a reason to respond on the fixed schedule. The subjects were responding to the proximate outcomes, rather than the overall outcomes. This is an example of how subtle changes in the schedule can cause drastic changes in behavior. Whenever we provide players with rewards, we're creating a schedule of reinforcement that will influence them to behave in particular ways. Because we can't avoid these effects, we have to understand them so that they can be made to work for us, rather than against us.
One factor we haven't addressed yet is when the decisions are made. Many of the choices we make in games don't have immediate effects, only helping or harming the player minutes or hours down the line. A character might have to choose whether to take a potion that gives them extra strength now or save it for later play. A player in a tank combat game might choose a fast, lightly armored tank rather than a slower, better protected one. Not all choices are followed by immediate consequences, and this delay often distorts the player's perception of their options.
Take the situation where a person has two possible options, each with a different level of reward. For example, a person might choose between receiving one piece of candy or two pieces of candy. If the delays are equal, the person would naturally choose the one with the larger reward. However, as the delay to the lesser reward decreases, the relative value of that reward starts to rise. If someone is offered one piece of candy right now compared to two pieces next year, most people would probably choose the more immediate reward.
Because he wanted to hear the Sirens but also make it home alive, Odysseus ordered his crew to tie him to the mast and to plug their ears.
This kind of decision making is often studied in children, who tend to be more strongly affected by these delays. However, its effects can be seen throughout life, from decisions about saving money to the relative addictive qualities of recreational drugs. A drug which takes effect faster will generally be more addictive than a slower one of equivalent strength.
A practical question arising from this research is under what circumstances do people tend to make more accurate decisions. One of the answers that psychologists have discovered has a parallel in an ancient Greek myth, Odysseus and the Sirens. Odysseus knew his boat was about to sail near the place where the Sirens were singing and that anyone who heard them would throw themselves into the sea in a vain attempt to reach them. Because he wanted to hear the Sirens but also make it home alive, he ordered his crew to tie him to the mast and to plug their ears with beeswax so they would not hear the call. In this way, his ship sailed safely past, his crew unhearing of both the Sirens and his pleas to be untied.
Because he made the decisions at a long delay from both outcomes, his choice was a good one. If he'd waited until the Sirens were right there and had to choose, his decision would have maximized the short term happiness of listening to their song over the longer term reward of making it home alive.
More generally, the more distant all of the outcomes are, the more people's choices tend to maximize long-term success. Of course, you may not want players doing deep long-term thinking. It's up to the designer what's best for his or her game, whether to skew the players towards one option or another, towards one strategy or another. Delays between action and outcome are just one of the tools available to influence how players choose.
every choice a real human being makes would take a model as complex as
the human mind. Psychology cannot offer use that yet, but it can give
us rules of thumb and general patterns of choice that can describe a generous
portion of what we do when presented with multiple options. Every game
offers its players a sequence of choices, each with attendant consequences
for choosing wisely or poorly. By understanding some portion of the rules
that govern how human beings react to those choices, we can design games
that elicit the kinds of choices that make the game a more enjoyable experience
for the player.