Applying Risk Analysis To Play-Balance RPGs

How often do players and game magazines compliment a recently released game by saying that it's "well balanced"? How often do they say this about products that have been out for years? In the real-time strategy (RTS) and role-playing game (RPG) genres, the answer to the first question is seldom, if ever. In the case of massively multiplayer online role-playing games (MMORPGs), the answer to the second question is not as often as they should.

Good game balance is, in many ways, the holy grail of game design. Companies invest significant time and resources in attempts to balance their games so that they're neither too hard nor too easy for players. These efforts can be a significant drain on a company's resources, since designers and engineers working to solve balancing problems aren't adding functionality. Instead, they're spending valuable time reviewing the same equations or lines of code in an attempt to determine why, for instance, one character class consistently outperforms another.

Unbalanced games hurt development companies in other ways, too. An unbalanced game frustrates players and generates negative publicity and reviews. Both of these situations negatively impact game sales and/or subscriber retention levels, which in turn decreases the cash flow to the publisher and developer, and restricts the capital and resources available to those firms. Without those resources, the ability to improve the functionality and features of existing games is limited. Additionally, the funding available for the development of new games is decreased.

Developing AAA games titles is a multi-million dollar expense and the costs continue to rise. On average it costs three million dollars to release a title, and MMORPG development costs easily exceed ten million dollars. This is all upfront cost and does not include the post-release expenses incurred as designers and engineers attempt to remove bugs and balance their games. In the case of some games, the post-release expenses can be quite significant. Much of the post-release costs stem from ineffective balancing during the initial design, alpha and beta phases. Fortunately, the proper application of risk analysis techniques can help you in two ways. It can reduce development time, and therefore costs, and also increase sales and subscriber levels due to positive word of mouth.

Probabilistic and Risk Analysis

At the most basic level, an RPG is simply a large collection of numbers and equations. As such, an entire game could be developed using only a spreadsheet program, such as Microsoft Excel, and a probability plugin, like Palisade's @Risk. @Risk is a commercial add-in designed to work with Microsoft Excel. The program allows a designer to exchange uncertain and variable values for @Risk functions. By doing so, an entire spectrum of results can be observed instead of a simple average value. The use of @Risk and Microsoft Excel allows one to model and analyze critical game systems.

Of course, a game created with only a spreadsheet and probability package would lack the meaning, context, and emotion that an RPG brings to players. However, such a game contains the same fundamental systems that allow players to advance and evolve within online games, with the benefit that meaning, context and emotion are stripped from the game. By focusing just on the underlying combat algorithms, systems can be developed and modeled rapidly and simulated thousands of times in order to determine the likely outcome of a situation. By analyzing these outcomes, truly balanced games can be created.

Risk analysis techniques work the same irrespective of the industry they're applied to, and the desired results are even quite similar. Just as the petroleum industry might try to predict future utilization of fixed assets, a game developer might attempt to predict future results of a given game situation. In the petroleum industry, data models supply executives with information so they may make cost/benefit decisions regarding the assets. Similarly, a combat model for a game helps a designer predict the outcome and see if adjustments need to be made to races, classes, and groups such that balance is achieved. In both cases, risk analysis techniques decrease uncertainties and allow the maximum potential to be reached.

While this article focuses primarily on RPG gaming systems, the application of risk analysis techniques can be used to play-balance games within most other genres. Real-time strategy games are excellent candidates in particular, due to the variety of combat scenarios that need to be tested. Where risk analysis modeling is not applicable are relatively simple games such as platformers and side-scrollers, since the systems that players interact with in these games do not generally contain systems that are worth modeling - they can be balanced by the designer through simple trial and error.

Origins Of This Article

(Editor's Note: This section amended at author's request on 6/16/03.)

The origins of this article began after making observations of Asheron's Call 2 (AC2) during its beta and retail stages. While reviewing the game's pseudo-class-based system and its associated skills, inconsistencies were observed in the balance of certain game systems. Through correspondence with AC2's program manager, Matthew Ford, it was discovered that while AC2's developers tuned game balance by using numerical analysis techniques common to MMORPG development, neither AC2 or any other MMORPG in Ford's knowledge used the deep risk analysis methods described in this article. I found this interesting; it explained the degree of specialization, class, and realm/faction imbalance found in many retail products.

In my correspondence with Ford, we discussed the concepts behind risk analysis and its practical application for MMOPRPGs. We also examined the predictive benefits gained by modeling and testing MMORPG systems. Ford agreed that any MMORPG would find this risk analysis a "valuable companion to traditional mathematical tuning methods"; he particularly saw it as a well-founded way to catch unexpected consequences created by seemingly unrelated changes to game balance values. Ford also agreed that this method could be a good way to rapidly iterate experimental game dynamics in the early stages of a game's design, as well as refine game balance in the later stages of development.

Significant Game Systems

When designing an RPG, numerous systems need to be developed and balanced. Many times what appears to be ideal on paper is found unacceptable when actually applied in game, as small differences in skills, spells, or other abilities become pronounced in the course of using them thousands of times. Some of the systems that need to be balanced in RPGs include:

• Player Character Races
• Player Character Classes
• Player vs. Environment Conflict (PvE)
• Player vs. Player Conflict (PvP)
• Player Advancement and/or Skill Improvement Rates
• Crafting Cost and the Time Required to Increase Skills
• Resource Limitations

This list is just a fraction of the systems that need to be designed and balanced in a successful RPG. And to make things even more complicated, someone trying to balance one game system may unwittingly unbalance another, since game systems are not always discrete elements. Most systems interrelate, and adjustments made to basic areas like player races and classes have the potential to affect every facet of a game.

Developing a Model

In order to apply risk-analysis techniques to game balancing, a base-case scenario is required. In the case of a combat system, a base case represents a typical player character of predetermined level/class/race/etc. For an existing game, care should be taken to gather data so that an average player character can be simulated. If balancing is attempted prior to the stage where the game is playable, then designers should use both personal experience and future expectations to develop a base case that accurately reflects what they want to see in the completed game.

Qualification of Input Parameters

Once the base case is determined, all of the constants, variables, and equations contained by the system need to be quantified:

• Constants. In an RPG, constants represent unchanging values in the models. Examples of constants might be character level, NPC spawn timers, and initial racial statistics. While constants will be adjusted as games are being balanced, no probability distributions will be applied to them.
• Variables. These represent in-game values that can differ between players. They include the allocation of skill points, duplicate instances of the same NPC monster, and craft quality distributions. Variables are accounted for by using probability distribution functions. Normal, log normal, Chi squared, and many others distributions can be used to represent random elements in a game.
• Equations. These are the calculations that determine the success or failure rate of an action, the mathematical relationship between shield skill and blocking percentage, or the effect of primary statistics on a to hit or damage roll.

When designing models, a caveat exists: a model is only as good as the data fed into it. When inaccurate data or equations are placed into a model, the results generated by that model will be inaccurate and irrelevant. "Garbage in, garbage out," is a frequent adage. It is therefore important to understand both the range of conditions available in the game system as well as a practical understanding of player tendencies and minimum/maximum potential.

If a model was being developed to tweak the success rate between a basic melee player character and a basic melee monster, it might require the following for each:

Model Requirements:

• Primary Statistics
• Health Totals
• Mana Totals
• Armor Type
• Armor Level
• Armor Absorption
• Weapon Type
• Weapon Damage
• Offensive/Defensive Bonuses
• Offensive/Defensive Penalties
• Evade/Parry/Block/To Hit percentages

The above represent only a partial listing. If systems within a game are affected by features such as the time of day, or other outside factors, then the model will require inputs for those elements. Simple games require fewer inputs in order to generate meaningful results. The converse is also true - complex games require more effort and control when developing a model.

In order to create more accurate models, most of the above requirements should be inserted into the model in the form of variables. The use of variables, as opposed to constants, allows for a range of conditions to be observed. If, for example, players can vary their primary statistics between values X and Y, then the model should account for that. By factoring variations into a model, designers will not be limited to results provided by constant average values.

Once all the constants, variables and equations have been quantified and input into the spreadsheet, it is time to run the simulation. The number of trials performed is arbitrary and up to the modeler. The more trials conducted, the more even distribution you'll get, and therefore you'll get more accurate results. However, especially in the case of exceptionally complex models, conducting trials takes time. In general, a simulation containing five thousand iterations is adequate. This limits the effect of any given iteration to only 0.0002%. If unaccountable outliers are observed, the number of simulations performed can be increased in order to smooth the curves.

Comparing Results to Design Objectives and "What If" Scenarios

Once a simulation is complete, the results can be analyzed. One of the primary benefits of risk analysis software is the ability to observe cumulative probability graphs for specified outputs. Cumulative probability graphs illustrate the percentage chance that an outcome Y will occur at least X percent of the time. In the case of combat between a basic player character and a basic NPC monster, designers can observe the hit point variations each round for the duration of combat. Figure 1 shows a sample cumulative probability distribution for melee hit points, and illustrates the benefits.

	Figure 1: Sample Cumulative Probability Distribution for Melee Hit Points. Click on Image to Enlarge.

Tabulating the chance of death each round allows designers to observe the likelihood of a combatant dying on a round-by-round basis (see Figure 2). Naturally, subtracting the chance of dying from the value 1.0 produces the chance of survival (e.g., 1.0-.6=.4, or 40% chance of survival). In a fight to the death (which most game battles are), survival of the player character indicates that the enemy was dispatched first. In the case of PC vs. NPC combat, plotting the chance of NPC death and one (1) minus the chance of PC death displays the likelihood of victory or loss each round. The point where these two lines intersect is the expected outcome of the fight. If, on a plot containing the chance of NPC death and one minus the chance of player character death, the lines intersect at sixty percent (60%) in round twenty-five (25), then the player character will have a sixty percent (60%) chance of victory in any single combat against that NPC. This intersection point can be compared to the desired outcome of the combat simulation. Desired outcomes might be:

· A 75% chance of beating an equal level monster using no special skills or styles
· A 95% chance of beating an equal level monster using optimum level skills and abilities
· An average of 60 hours spent crafting before a trade skill reaches its maximum
· A 40% chance that one class will beat another class using optimum skills/styles.

If the desired outcome and the simulation results are equal, the system is balanced and a designer can move on the next simulation. If the simulation result and the desired outcome are unequal, then adjustments need to be made to some part of the system.

	Figure 2: PC Melee vs. NPC Monster Expected Combat Outcome. Click on Image to Enlarge.

When attempting to balance a combat system in this fashion, the process is reminiscent of supply-and-demand graphs used by economists. On a supply-and-demand graph, certain factors shift the curves in various directions. The same holds true for the victory and defeat curves. By varying constants or equations, such as weapon damage and "to hit" rolls, it is possible to horizontally translate one or both of the curves. By adjusting inputs until the point of intersection occurs at the desired outcome, the game system can be balanced.

The benefit of an effective model is that it not only lets you balance specific classes and encounters, it also lets you explore limitless "what if" scenarios. For example, suppose that player dexterity affects the "to hit" roll, damage roll, and blocking chance. If a designer wants to increase player damage by increasing a class's dexterity, he has to worry about the effects on the "to hit" roll and blocking percent. If a designer wants to increase a race's base dexterity, he needs to consider the effects of this change on every other class in the game. However, if a model was already developed, it would be a simple endeavor to increase the dexterity of the class or race in question, re-run the simulation, and determine whether the dexterity increase threw off the balance in relation to other classes, races or NPCs.