What kind of in-game metrics and analytics tools do you use to measure your game's health?
TH: I think what's most interesting, from a developer standpoint versus a business view, is all of the game design metrics that we collect and that any F2P developer can collect. So yes, we look at retention rates, and monetization rates, and what's selling, but our designers have access to really, really detailed data on the strength of every weapon. They can look at, for instance, the kill-to-death ratio of the nine different classes in the game, and whether those ratios are actually bearing out in reality as we would expect from the game design.
With a single day's worth of data, our design team can see enough statistically relevant data to see if any design changes are working as intended or not. So that's really what's most exciting to the game design team.
Every match you play in Tribes, that data, in terms of how many kills, what weapon you used, how effective those weapons were, how effective your team was -- all of that is being captured in a persistent database, and our designers can use that data to improve the game.
MH: We have very extensive monitoring and metrics tools in PlanetSide 2 for us to figure out stuff like how many people logged in today, logged in yesterday, and percentages of falloff of people. Also every kill that happens, every death that happens, we track and we can filter that through a variety of tools to figure out balance -- figuring out which areas of the game people play and stick around with.
So yeah, data gathering and metrics for a game like this, where we're planning on making changes for years to come, being able to track all our metrics and what people are doing and what we can do to make people keep playing is really, really important.
BE: We designed our own proprietary telemetry system that logs pretty much every user action in the game, from where they click, to how well they do in each match, to how much they spend and when, to their average FPS. Our community of players also gives us regular feedback and has been a huge asset in the Closed and Open Beta phase.
How do you decide what to charge for and how much to charge? Is there a coherent philosophy behind your monetization design?
TH: The philosophy that we started with in Tribes was that we wanted it to be relatively less expensive in terms of time or money to unlock different classes, so that various roles on the battlefield would be filled up pretty quickly. Then, in terms of weapons, we wanted there to be more progression involved in terms of player time or money. So classes first, weapons second, just so there would be diversity on the battlefield. Beyond that, it's fairly metrics-driven, and we do a lot of experimentation in terms of price points.
MH: I think it's a feel thing. I don't think there's really a formula that you can plug stuff into to figure it out exactly. It has to do with how many items you're going to allow people to unlock. What sort of progression is involved in unlocking items? What's the gameplay associated with unlocking?
For us, as you're playing PlanetSide and getting kills and capturing bases and all that stuff, you're earning stuff that you need. You don't really need to go out of your way to do stuff that's not fun, or not part of the core game, to get the points that you use to unlock new items. So the very core of the entertainment experience of the game is also helping you progress your character and unlock new stuff.
But we set kind of a wide range of prices from things that are like 50 cents to, I think our highest-priced items right now are bundles that give you multiple items for around 10 bucks. At the end of the day, with a free-to-play game, the best possible thing you can do is make people feel good about the purchases that they're making. Make them feel they got a good value for what they're spending, and that they're supporting a game they enjoy. If you can accomplish those two things I think you can be successful in the free-to-play space.
BE: We're still working out how elastic our economy is, testing a variety of price points, value propositions, and rarity. We generally start with a theory on value, and the player confirms (through a purchase or not) the value of an item, and how much would they be willing to pay for it. Then we test the theory and analyze the results. Based on early Closed Beta data, we tuned our prices and content toward the results of these tests. Now that we're in Open Beta and seeing true user-buying habits, we've tweaked a few of our original theories, most notably by adding both temporary and permanent buying options.