|
Customer metrics: As a customer, users can download and install a game, purchase any number of virtual items from in-game or out-of-game stores and shops, spending real or virtual currency, over shorter or longer timespans. At the same time, customers interact with customer service, submitting bug reports, requests for help, complaints, and so on. Users can also interact with forums, official or not, or other social-interaction platforms, from which information about these users, their play behavior, and their satisfaction with the game can be mined and analyzed. We can also collect information on customers' countries, IP addresses, and sometimes even age, gender, and email addresses. Combining this kind of demographic information with behavioral data can provide powerful insights into a game's customer base.
Community metrics: Users interact with each other if they have the opportunity. This interaction can be related to gameplay (combat or collaboration through game mechanics) or social (in-game chat). Player-player interaction can occur in-game or out-of-game, or some combination thereof -- for example, sending messages bragging about a new piece of equipment using a post-to-Facebook function. In-game, users can interact with each other via chat functions, out-of-game via live conversation (TeamSpeak or Skype), or via game forums.
These kinds of interactions between players form an important source of information, applicable in an array of contexts. For example, a social-network analysis of the user community in a F2P game can reveal players with strong social networks -- who are the players likely to help retain a big number of other players in the game by creating a good social environment (think guild leaders in MMORPGs). Likewise, mining chat logs and forum posts can provide information about problems in a game's design. For example, data-mining datasets derived from chat logs in an online game can reveal bugs or other problems. Monitoring and analyzing player-player interaction is important in all situations where there are multiple players, but especially in games that attempt to create and support a persistent player community, and which have adopted an online business model, which includes many social online games and F2P games. These examples are just the tip of a very deep iceberg, and the collection, analysis, and reporting on game metrics derived from player-player interaction is a topic that could easily take up several volumes.
Gameplay metrics: This subcategory of the user metrics is perhaps the most widely logged and utilized type of game telemetry currently in use. Gameplay metrics are measures of player behavior: navigation, item and ability use, jumping, trading, running, and whatever else players actually do inside the virtual environment of a game (whether 2D or 3D). Four types of information can be logged whenever a player does something or something happens to a player in a game: What is happening? Where is it happening? At what time is it happening? And: Who is involved?
Gameplay metrics are particularly useful for informing game design. They provide the opportunity to address key questions, including whether any game world areas are over- or underused, if players utilize game features as intended, and whether there are any barriers hindering player progression. These kind of game metrics can be recorded during all phases of game development, as well as following launch.
Players can generate thousands of behavioral measures over the course of a single game session -- every time a player inputs something to the game system, it has to react and respond. Accurate measures of player activity can include dozens of actions being measured per second. Consider, for example, players in a typical fantasy MMORPG like World of Warcraft: Measuring user behavior could involve logging the position of the player's character, its current health, mana, stamina, the time of any buffs affecting it, the active action (running, swinging an axe), the mode (in combat, trading, traveling), the attitude of any NPC enemies toward the player, the player character name, race, level, equipment, currency, and so on -- all these bits of information simply flow from the installed game client to the collection servers.
From a practical perspective, you may want to further subdivide gameplay metrics into the following three categories (in order to make your metrics more searchable, for instance):
- In-game: Covers all in-game actions and behaviors of players, including navigation, economic behavior, as well as interaction with game assets such as objects and entities. This category will in most cases form the bulk of collected user telemetry.
- Interface: Includes all interactions the player performs with the game interface and menus. This includes setting game variables, such as mouse sensitivity and monitor brightness.
- System: System metrics cover the actions game engines and their subsystems (AI system, automated events, MOB/NPC actions, and so on) initiate to respond to player actions. For example, a MOB attacking a player character if it moves within aggro range, or progressing the player to the next level upon satisfaction of a predefined set of conditions.
To sum up, the array of potential measures from the users of a game (or game service) can be staggering, and generally we should aim for logging and analyzing the most essential information. This selection process imposes a bias, but is often necessary to avoid data overload and to ensure a functional workflow in analytics.
|
In my experience that's one of the main constraints in game play related feature selection; QAing thousands of data points is simply unrealistic.
Developers who are aware that their analytics may produce false numbers trust data only if results are in line with their assumptions. That makes any analytics kind of pointless. Those who are not aware of that and trust their data usually end up making expensive mistakes. It's really way better to go blind without data and trust your team experience in both cases.
Of course there are ways to deal with that problem and get reliable results from analytics:
1. make sure that engineer instrumenting analytics service is working directly with someone experienced with that specific service - either someone in-house or a support guy from analytics vendor who will explain the process and audit the integration.
2. having a real time analytics during integration really helps as you can record your session and check results instantly. It's also important that you have ability to clean up database (or filter out your most recent activity) to make sure that you are checking the data from the last session only. If your analytics doesn't give you that comfort, log every outgoing data point on your side and do the math by yourself.
3. Even if you are very diligent about the integration, chances that you will get it right from the beginning are low. If you don't want to get into troubles due to data misinterpretation double check it using qualitative approach. Some of analytics services allows you to export data points by session id to excel and some others* have full set of features to analyze individual sessions or users. This will help you identify mistakes in data collection but also better understand correctly collected data before you jump into conclusions.
*I know only UseItBetter Analytics (disclosure: I'm co-founder) that does that for games but I might be totally wrong about it. Maybe Mixpanel or Playnomics?