Welcome Back. I’ve Been Waiting…

If you’re reading this, then chances are you also read Part 1, “Probability for Game Designers.”

If you haven’t read it, you really should, and that’s not to say it is full of good stuff (the article is tripe, actually). I just recommend reading it because if you don’t, you might be unprepared for the silliness that may ensue during this serious *ahem* and erudite *cough* discussion of statistics.

This article focuses on a few select statistical topics that I believe should be understood by game designers. In particular, statistics really is useful and important for system designers, mechanicians, balancers, and other subclasses of designer that are usually relegated to steerage.

Disclaimer taken care of, let’s move on to the fizzy stuff!

Statistics: A Two-Drink Minimum Science

Although heavily grounded in mathematics, statistics is...well...weird! Seriously - if you ever have to start dealing heavily in two-sided confidence intervals and Student’s T-tests and chi-squared tests (or anything else squared, for that matter), it can get a little hard to digest at times.

The Secret Badge of Statisticians Everywhere

You see, people like me really prefer physical metaphors. I’ve always liked physics and mechanics, because a lot of the time you can give yourself a reality check simply by analyzing reality. When you’re calculating the rate and direction at which an apple falls from a tree, you can reality check it in your head if your result says the apple should shoot off straight upward at 1,224 MPH.

At its best, statistics is understandable and rational; at its worst, it’s a little strange. Hence, I recommend libations and togas for any involved statistics discussion. I have asked the fine editors at Gamasutra to provide such togas and an open digital bar. What, didn’t you get your passcode? Hmmm, weird.

In any case, the topics in this article aren’t weird at all. For the most part, they are tangible, crunchy bits of statistics that you can develop gut feels for.

Statistics: The Dark Science

Statistics is, of all the sciences, the one that is very prone to misuse by the Forces of Evil. That is, if you had to attribute one science to the villain you are creating for your new book (you are writing a book, aren’t you?), you could do much worse than pick statistics. You could also give him a cape, dress him in black, and refer to him as “The Spider” or “Mr. Jones”, but I digress.

The reason that statistics can be loosely compared to villainy is that, used improperly, this branch of science can be called upon to infer all sorts of relationships that aren’t actually meaningful or even true (see the end of this article for an example of what I mean). When in the hands of politicians and other ne’er do wells, this can guide big decisions. Big decisions based upon inaccurate conclusions are never good.

All this is to say, statistics is incredibly useful and helpful when used properly. But like any stuperpower, it can be applied in nefarious ways, or even just plain dumb ways.

Statistics – What’s All The Fuss About?

I was going to crack my knuckles and write a tight summary, but then noticed that Wikipedia already had something that was darn near poetry. Here it is:

Statistics is a mathematical science pertaining to the collection, analysis, interpretation, and presentation of data. It is applicable to a wide variety of academic disciplines, from the physical and social sciences to the humanities; it is also used for making informed decisions in all areas of business and government. (Courtesy Wikipedia.org)

That’s actually a very moving passage. In particular, the last bit is the tour de force of the paragraph:

...it is also used for making informed decisions...

Of course, the writer forgot to add “in game design,” but we can forgive him his condescension towards our burgeoning industry.

Here’s my own try:

Statistics is a mathematical science that deals with collecting and analyzing data in order to determine past trends, forecast future results, and gain a level of confidence about stuff that we want to know more about. (Courtesy Tylerpedia)

And if I were to modify it for Game Design, I would say (and am, in fact, saying):

Statistics can help you shine a flashlight upon your broken mechanics and shattered design dreams. It does this by giving you actual hard, scientific data to support meaningful design decisions.

What Do We Need to Know?

Statistics, like any hard science, is deep and complex. Like the tour of Probability in Part 1, this article only touches on a few selected topics that I, in my unlimited hubris, have deemed Important Enough to Know®. (Yep – unlike the many TMs I throw around, this one is so potent it’s registered!)

Pop Quiz Again

I’m sad to say that I have resorted to another test. Don’t hate the Quizza, hate the Quiz.

A Taxed Quizzee

Q1a) Focus testers have just finished playing through a level in your new snail racing game “S-car GO!” Twenty testers played, and your are informed that the lap times came back in a range from 1 min 24 seconds at the low end to 2 min 32 seconds at the high end. You were expecting an average time of 2 minutes or so. Was the test a success?

Q1b) You collect more data for the same level, do some analysis, and find that the stats are: mean = 2 min 5 sec, standard deviation = 45 sec. Should you be satisfied?

Q2) You design a casual game that will surely soon be the talk of soccer moms everywhere (an admirable goal). In final QA, you release a beta build and then take data on a whole bunch of trial sessions. Over 1,000 play sessions are recorded, with over 100 unique players (some players were allowed to play repeated sessions). Crunching the data shows a mean score of 52,000 pts with a standard deviation of 500 pts. Is the game tuned up enough to release?

Q3) You design an RPG, and then collect data on how fast it takes new players to progress from level 1 to level 5. The data comes in as follows: 4.6 hrs, 3.9 hrs, 5.6 hrs, 0.2 hrs, 5.5 hrs, 4.4 hrs. 4.2 hrs, 5.3 hrs. Should you calculate the mean and standard deviation?