GAME JOBS
Contents
Intro to User Analytics
 
 
Printer-Friendly VersionPrinter-Friendly Version
 
Latest Jobs
spacer View All     Post a Job     RSS spacer
 
June 6, 2013
 
Wargaming.net
Build Engineer
 
Gameloft - New York
Programmer
 
Wargaming.net
Build Engineer
 
Virdyne Technologies
Unity Programmer
 
Wargaming.net
Quality Assurance Analyst
 
Wargaming.net
Python Developer
spacer
Latest Blogs
spacer View All     Post     RSS spacer
 
June 6, 2013
 
Free to Play: A Call for Games Lacking Challenge
 
Cracking the Touchscreen Code [1]
 
10 Business Law and Tax Law Steps to Improve the Chance of Crowdfunding Success
 
Deep Plaid Games, one year later
 
The Competition of Sportsmanship in Online Games
spacer
About
spacer Editor-In-Chief:
Kris Graft
Blog Director:
Christian Nutt
Senior Contributing Editor:
Brandon Sheffield
News Editors:
Mike Rose, Kris Ligman
Editors-At-Large:
Leigh Alexander, Chris Morris
Advertising:
Jennifer Sulik
Recruitment:
Gina Gross
Education:
Gillian Crowley
 
Contact Gamasutra
 
Report a Problem
 
Submit News
 
Comment Guidelines
 
Blogging Guidelines
Sponsor
Features
  Intro to User Analytics
by Anders Drachen, Alessandro Canossa, Magy Seif El-Nasr [Business/Marketing, Design, Game Developer Magazine, Console/PC, Social/Online, Smartphone/Tablet, GD Mag, GD Mag Exclusive]
3 comments Share on Twitter Share on Facebook RSS
 
 
May 30, 2013 Article Start Previous Page 4 of 6 Next
 

3) Core business attributes: The essential attributes related to the core of the business model of the company, for example, logging every time a user purchases a virtual item (and what that item is), establishes a friend connection in-game, or recommends the game to a Facebook friend -- or any other attributes related to revenue, retention, virality, and churn. For a mobile game, geolocation data can be very interesting to assist target marketing. In a traditional retail situation, none of these are of interest, of course.

4) Stakeholder requirements: In addition, there can be an assortment of stakeholder requirements that need to be considered. For example, management or marketing may place a high value on knowing the number of Daily Active Users (DAU). Such requirements may or may not align with the categories mentioned above.



5) QA and user research: Finally, if there is any interest in using telemetry data for user research/user testing and quality assurance (recording crashes and crash causes, hardware configuration of client systems, and notable game settings), it may be necessary to augment to attributes on the list of features accordingly.

When building the initial attribute set and planning the metrics that can be derived from them, you need to make sure that the selection process is as well informed as possible, and includes all the involved stakeholders. This minimizes the need to go back to the code and embed additional hooks at a later time -- which is a waste that can be eliminated with careful planning.

That being said, as the game evolves during production as well as following launch (whether a persistent game or through DLCs/patches), it will typically be necessary to some degree to embed new hooks in the code in order to track new attributes and thus sustain an evolving analytics practice. Sampling is another key consideration. It may not be necessary to track every time someone fires a gun, but only 1 percent of these. Sampling is a big issue in its own right, and we will therefore not delve further on this subject here, apart from noting that sampling can be an efficient way to cut resource requirements for game analytics. 

Figure 2: The drivers of attribute selection for user behavior attributes. Given the broad scope of application of game analytics, a number of sources of requirements exist.

Preselecting Features

One important factor to consider during the feature selection process is the extent to which your attribute set selection can be driven by pre-planning, by defining the game metrics and analysis results (and thereby the actionable insights) we wish to obtain from user telemetry and select attributes accordingly.

Reducing complexity is necessary, but as you restrict the scope of the data-gathering process, you run the risk of missing important patterns in user behavior that cannot be detected using the preselected attributes. This problem is exasperated in situations where the game metrics and analyses are also predefined -- for example, relying on a set of Key Performance Indicators (such as DAU, MAU, ARPU, LTV, etc.) can eliminate your chance of finding any patterns in the behavioral data not detectable via the predefined metrics and analyses. In general, striking a balance between the two situations is the best solution, depending on available analytics resource. For example, focusing exclusively on KPIs will not tell you about in-game behavior, e.g., why 35 percent of the players drop out on level 8 -- for that we need to look at metrics related to design and performance.

It is worth noting that when it comes to user analytics, we are working with human behavior, which is notoriously unpredictable. This means that predicting user analytics requirements can be challenging. This emphasizes the need for the use of both explorative (we look at the user data to see what patterns they contain) and hypothesis-driven methods (we know what we want to measure and know the possible results, not just which one is correct). 

 
Article Start Previous Page 4 of 6 Next
 
Top Stories

image
Keeping the simulation dream alive
image
A 15-year-old critique of the game industry that's still relevant today
image
Here's the first list of Unreal Engine 4 integrated middleware
image
The demo is dead, revisited
Comments

Taylor Stallman
profile image
Great article. After just graduating from college last December with a focus on database marketing. This article is spot on. It even taught me a number of things! Thanks a lot for the article, I'll be sure to use this the next time I need to analyze data.

Henrik Strandberg
profile image
Excellent article! I'd also be very curious to learn about your approach to and experience from QAing the data sets; after all, if you haven't verified (via QA) that the data is correctly generated, aggregated, transformed and exposed, how can you trust the analysis?

In my experience that's one of the main constraints in game play related feature selection; QAing thousands of data points is simply unrealistic.

Lukasz Twardowski
profile image
Henrik, that's a really good question. Most of initial instrumentations of analytics generate false numbers. One, because people who instrument analytics are not the same people who use analytics. They rarely understand how to collect data, what for and how to check its integrity. Two, because games are complex and usually have plenty of small glitches or shortcuts that might not impact players experience at all but can totally corrupt your data sets.

Developers who are aware that their analytics may produce false numbers trust data only if results are in line with their assumptions. That makes any analytics kind of pointless. Those who are not aware of that and trust their data usually end up making expensive mistakes. It's really way better to go blind without data and trust your team experience in both cases.

Of course there are ways to deal with that problem and get reliable results from analytics:

1. make sure that engineer instrumenting analytics service is working directly with someone experienced with that specific service - either someone in-house or a support guy from analytics vendor who will explain the process and audit the integration.

2. having a real time analytics during integration really helps as you can record your session and check results instantly. It's also important that you have ability to clean up database (or filter out your most recent activity) to make sure that you are checking the data from the last session only. If your analytics doesn't give you that comfort, log every outgoing data point on your side and do the math by yourself.

3. Even if you are very diligent about the integration, chances that you will get it right from the beginning are low. If you don't want to get into troubles due to data misinterpretation double check it using qualitative approach. Some of analytics services allows you to export data points by session id to excel and some others* have full set of features to analyze individual sessions or users. This will help you identify mistakes in data collection but also better understand correctly collected data before you jump into conclusions.

*I know only UseItBetter Analytics (disclosure: I'm co-founder) that does that for games but I might be totally wrong about it. Maybe Mixpanel or Playnomics?


none
 
Comment:
 




UBM Tech