Gamasutra: The Art & Business of Making Gamesspacer
arrowPress Releases
September 16, 2014
PR Newswire
View All

If you enjoy reading this site, you might also want to check out these UBM Tech sites:

Adding Telemetry to Infinite Mario
by Ben Weber on 12/28/13 02:17:00 pm   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.


I originally posted this on my personal blog in September 2010, but it is no longer available since Posterous shut down.

BioWare announced findings for Mass Effect 2 from a massive telemetry project in which millions of game sessions were analyzed. Some of the interesting findings include popularities of the different classes in the game, such as the fact that the soldier class was played more than all others combined. One of the possible applications of collecting large amounts of data is to incorporate results from real-world player data into future games. This post provides a tutorial for adding telemetry to your own games. While the presented approach is a Java application, it should be applicable to any language capable of an http post.

Since the announcement of the IEEE Mario level generation competition, I've been exploring procedural content generation for Mario levels. The goal of the competition is to maximize player enjoyment by adapting to specific player styles. The competition uses a heavily modified version of the Infinite Mario engine developed by Notch. My variation of the game with dynamic difficulty adjustment is available to play here (arrows to move, 's' to jump, 'a' to run). After releasing this game mod I asked:

  • How good is the average player?
  • What causes players to lose?
  • How long do players continue to play the game before quitting? 

To answer these questions, I've added telemetry hooks into the game. This process can be simplistic for games capable of posting to http and as long as you have a server capable of running PHP scripts. On the server end, you can use a PHP script to append telemetry data to a file, as shown here:

 < ?php
    $message = $_GET['message'];
    if ($message != '') {
        $dataFile = fopen("telemetry.log", "a");
        fwrite($dataFile, "$message\n");

This code checks for a message parameter, and then appends it to a file. In a real-world deployment, additional security measures are needed, but this tutorial is focusing on simplicity. Next, we need a way of posting telemetry data from the game engine. In Java, we can use the following code to post to our PHP script:

URL url = new URL("telemetry.php?message=" + message);
URLConnection con = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
while (in.readLine() != null) {}

This code passes the content in the message variable to the PHP script using a HTTP post. The result is a method for posting game data to our server for analysis. I used this minimal approach to collect gameplay data from over 100k gameplay sessions. The figure below shows a histogram of frequencies of difficulty levels generated by my level generator. Each player starts out at level 50, which I have excluded from this figure. Then as a player progresses through the game, the difficulty adapts to the player's skill. The results of my initial data mining experiment suggest that the initial difficulty level is too high and should be set to a value in the 35-40 range.

Difficulty of Generated Levels 

Now that we see that the game starts out too difficult I investigated the second question, why are players dying so frequently? Since we now have telemetry data, we can determine the cause:

Cause of Death

It appears that most players are not completing levels due to deaths caused by enemies. If my design intent was to create levels that are difficult to traverse due to gap challenges, then I have failed. At this point in the process, I can take results from my data analysis and apply them to future game design experiences.

I hope this post helps out any game designers interested in studying their users' behaviors. This has only been an intro, and there are many more issues to address, such as scaling up and extracting more complex patterns of player behavior. Further reading on this topic includes my workshop paper on player modeling and the recent Game Analytics book.

Related Jobs

Amazon — Irvine, California, United States

Software Development Manager
Cortina Productions
Cortina Productions — McLean, Virginia, United States

Software Application Developer / Programmer
Avalanche Studios
Avalanche Studios — New York, New York, United States

Technical Artist
Shiver Entertainment
Shiver Entertainment — South Miami, Florida, United States

Senior Technical Director/Artist


Javier Degirolmo
profile image
Funny, in my experience environmental hazards tend to be a lot more of trouble than enemies in platformers. Depends on the game mechanics though.

This probably isn't detailed enough anyway. As I said, generally enemies aren't much trouble, but there's always that one enemy that makes players go insane (the crab enemy in Metropolis zone in Sonic 2 is a good example of this). Stuff like that can skew statistics quite a lot.

Michael Stevens
profile image
Similarly, a death *to* an enemy is not necessarily a death *because* of an enemy (and vice versa).

Michael Stevens
profile image
I have a couple of issues with how your generator behaves in relation to your interpretation of the data.

1) You have max values for gap length and height variation, but it doesn't sound like you have anything governing enemy density, variety, or positioning. That could account for the high enemy-death to pit-death ratio. Along the same lines, slow completions are most directly compensated for, so it makes sense that time ups represent the fewest deaths. It would be fine the way it is if you were looking at something static or to a lesser extent just the generator at LV50, but by also scaling difficulty you introduce uncertainty over whether the generator is equally able to scale all elements. You need to be showing control data.

2) You have it set to recalibrate difficulty after three deaths in the same level. This is a pretty subjective/artificial boundary to impose while also considering session length. I'm sure there's a subset of players who were more frustrated by having a level shuffled before their attention waned than they were by dying three times. Newer Marios start the player with 5 lives, and while I think that's probably closer to the average attention span, it's going to be a subjective pressure no matter what number you pop in there due to variance in play style.

(I read a couple of your linked blogs and your workshop paper. They go into different levels of detail on how the generator works and what you learned, so I apologize if I've misunderstood, crossed a wire, or missed an important detail. Also, I appreciate that this specific blog is more along the lines of "here's a cool thing you can do" and not so much presenting the experiment itself, so sorry if this criticism is totally unwanted. I enjoyed the article.)

Ben Weber
profile image
The cause of death is definitely tied to the properties of the generator and playing a few levels will show how overwhelming enemy placement is in the system.

The system has a single value for representing difficulty that influences how frequently enemies are placed, the size of gaps, and the frequency of ground height changes. It's much easier to increase difficulty by adding enemies than by making more challenging jumps, because the level needs to be passable and gaps too large may be impossible to complete.

Ideally I'd like to see a generator that incorporates many of the design patterns shown in this series: