Metrics can rule you -- but should they? The Workshop Entertainment's new design director and Free Realms veteran Laralyn McWilliams explains how a pivotal moment in her life showed her that overreliance on analytics and friction in social games isn't the answer.
This article isn't about whether free-to-play games are bad, or social games are evil. For the record, I don't believe either case is true.
This, like life and like metrics, is about evolution. It's about change. It's about picking new behaviors based on the results of previous behaviors. It's about how an understanding of the end game can change the way you play.
It started back in 2006, when I began working on Free Realms. There really weren't many free-to-play games outside of a few upstarts from Asia, so the whole space felt brand new. I was lucky enough to work with a great group of developers at Sony Online Entertainment with an unprecedented amount of experience in online games. As a team, we understood how to make a great MMORPG, but free-to-play was the Wild West.
After launch, we started gathering data and looking at metrics. Although tracking and logging player choices had been an important part of online games for years, we quickly realized we needed better information and we needed it more quickly. This kicked off a frantic but exciting year of post-launch changes, and metrics played the key role.
Metrics gave me information, but they also let players talk to me directly and honestly. This wasn't the "guess, ship, and pray" design process of console games. I wasn't making decisions in a void anymore. With a combination of understanding the game, tracking changes, watching metrics, and listening to players, we made significant improvements to the player experience. We also significantly improved the amount of money we made.
This was free-to-play at its best: happy players, happy development team, happy businesspeople and execs.
I started speaking about metrics at conferences. I talked about what an important design tool they provide, and how they can guide your decisions on a live game (or on the game before launch via usability testing). I shared some metrics about play patterns among casual players, including some findings that were genuinely surprising to the development team.
Then I entered the world of social games.
Development of social games revolves around three core concepts:
A common way to look at metrics for online games is using a funnel. The typical funnel for user retention and monetization looks like this:
The funnel would have to be scaled larger than your monitor for you to actually see the whales down there at the bottom. That’s the size of a group they represent when viewed in context with the rest of the player base.
For many companies, modern free-to-play design -- especially in social and social/mobile -- focuses on the whales. The game is tuned to please the whales, even though the personality that lends itself to the highest spend in a game is certainly an edge case.
The games aren't just tuned to please the whales, though: they're tuned to squeeze the maximum amount of revenue out of them. Since the monetization is based on friction -- on players paying to bypass elements that stop them from playing, completing, or enjoying aspects of the game -- squeezing the most money out of the whales means continually turning up the friction until you hit the "sweet spot" where they pay regularly.
With the increase in friction comes the increase of players who hit the wall where the session gets so short or the grind gets so tedious that they quit playing. Online games have always had a certain amount of players quitting (called churn). The churn in social games is tremendous because the friction curve quickly gets so steep it curves most players right out of the game.
I'd entered the world of social game design with strong experience in free-to-play design and as an outspoken advocate for metrics as a part of the design process. As I tried to understand social game "best practices," I watched as people made decisions based purely on metrics with no interpretation. I watched as they made changes that increased short-term revenue without regard to the churn consequences of those changes on the larger player base.
I continued to speak at conferences about the importance of metrics as an information source for designers, but when I met someone at a dinner and he said, "Oh, you're the metrics lady," I felt unsettled. I believed in metrics... as a source of information. I'd always said that metrics aren't the answer -- they're one step in discovering the answer. Yet I was seeing metrics used without any attempt to understand the long-term effects of change, or the emotional side of the play experience.