     |
 |
 |
 | | | | |
 |
|||
 |
||||||
          |
 |
 |
||||
|
Means to an End
Consider this tiny section an intermission embedded within an otherwise tedious article. This tiny, self-referential section serves only one purpose in life: to remind you of what a “mean” is. This tiny, self-referential, and pedantic section would like to passively remind you that a mean is the mathematical average of a set of data. This tiny, self-referential, pedantic, passive, and well-meaning section hopes that you take something meaningful away from reading it; for it is now that this tiny, self-referential, pedantic, passive, and pun-throwing paragraph must end. Variance and Standard DeviationVariance and standard deviation are very important to understand, and have a lot of tangible value. Aside from helping us draw valuable statistical conclusions, these terms enable us to speak a lot more intelligently about distributions. Instead of saying “a great many data points cluster about the middle”, we can say “68.2% of the sample falls within one standard deviation of the mean.” Chicks dig that speak; guys dig that speak; heck, who doesn’t dig that speak?
Variance and standard deviation are related to each other, and they both measure the same thing: data scatter. Intuitively, a high variance or standard deviation means your data is all over the place. When I play darts, I get a high variance in my throws. Variance and standard deviation can be easily calculated from any set of data that you have. I’d put the equations in here, but that would break my “don’t sound like a textbook” rule. So instead of an equation, here’s a description: Standard Deviation: the average amount by which data points in the sample or population differ from the mean. Standard deviation is represented by the Greek letter σ (sigma) In other words, say you test 100 people on how long it takes them to complete Level 1 in your newest game. Let’s assume the average (mean) of all the data is 2 minutes 30 seconds. Now assume the standard deviation calculates out to be 15 seconds. This standard deviation indicates that the grouping or “clumping” of the play sessions. In this case, it’s saying that on average, play sessions are within ±.25 minutes of 2.5 minutes. That’s pretty consistent. What does this mean and why do you care? Easy. Pretend that instead of the above results, you got these results: Mean = 2.5 minutes (same as above) So here we have the same mean but a vastly different standard deviation. This set of numbers means that you have much more scatter in the play times. On average, play times are about 90 seconds off of the mean play time. Given that the mean play time is only 2.5 minutes, that’s huge! And it’s probably not good to have that much scatter, for various game design reasons. It would be much different if you were talking about a standard deviation of 90 seconds (1.5 minutes) on play times of 15 minutes. Consistency is measured by a small standard deviation. Ratio your standard deviation against your mean to get a good warm-fuzzy number. In the first example, 15 sec / 150 sec = 10%. In the second, 90 sec / 150 sec = 60%. A standard deviation of 60% is bigggggg with indulgently repeated g’s. In the third, 90 sec / 900 sec = 10% again…respectable. This is not to say that a large standard deviation is *always* bad. Sometimes as designers we want a large standard deviation in whatever we’re measuring. But a lot of times it’s bad, because it represents a lot of scatter and variability. The important thing is that calculating standard deviation will tell you a lot about your game/mechanic/level/etc. Examples of useful things to measure standard deviation for:
Margins of ErrorMargins of Error go hand in hand with statistical conclusions. Think of every Gallup Poll you’ve ever seen; there is always a margin of error expressed, such as ±2.0%. Because polls are using samples to estimate a population, there can never be 100% confidence (see later in the article). Margin of Err.0or indicates how accurate the results are. It is absolutely vital to know Margin of Error whenever you are talking about a population bigger than your sample. If you take data on your entire population, then theoretically you don’t need a Margin of Error – you already know all the data! For example, if I ask everyone on my street whether they prefer Chess or Go, then I don’t need a Margin of Error as long as I am just reporting about people on my street. But if I want to draw a conclusion about everyone in my town based upon the data points from my street, then I have to calculate Margin of Error. The bigger your sample size is, the smaller your Margin of Error. Mo data is bettuh. (Self-)Confidence IntervalsYou can use inferential statistics to draw conclusions about future data. One useful trick is the calculation of confidence intervals. Conceptually, confidence intervals are closely related to standard deviation, and are basically a mathematical way of saying how certain we are that a given piece of data will fall in a specified range. Confidence interval: a mathematical way of saying “we can guarantee with A% confidence that B% of the data will be between values C and D.” That’s a mouthful. But it’s useful to know, with a specified amount of confidence, what a value is likely to be. For a good example, I’m going to step back into my previous career for a blissful yet ultimately unsatisfying moment:
I used to do stress analysis and design of aircraft bits and bobs. If you know, or need to know, anything about aircraft - and commercial aircraft in particular - it’s that it is the most regulated form of transportation that exists. People don’t like it when wings fall off of planes. ‘nuff said. One of the methods we engineers use to keep said wings on said planes is designing to a very high confidence interval of material strength properties. A typical confidence interval used for aircraft design is the “A-basis allowable”, which means we are 95% confident that 99% of the values in any given shipment of a specified material fall above a certain value. Then, we design to that value against the worst possible air conditions, and then finally apply a big factor of safety on it. Gotta be sure. Confidence intervals are very informative and useful whenever you *really want to know* what kind of data values to expect. Fortunately, games are not typically a matter of life and death, but if you are trying to balance an (unpatchable) console game, you probably want to have more than gut feel and intuition to go on. Calculating confidence intervals could be used to give you hard facts about how your game plays, and whether there are obvious exploits. Whenever you want to calculate good confidence intervals, the ol’ standby rule of statistics still holds true: mo is bettuh. The more data points you have in your sample, the better your confidence interval calculation will be. |
|||||||||||||||||
Previous
|
|
|