Do you guys pull out of games if they're not working? What is your strategy there? Do you want to try and bolster them or redirect them?

PH: Well, I think the one big difference is that you can actually test your game theories and economics and theme and content with data, with an audience. You can limit that audience size. The funnels that you can control in terms of overall launch size is under our control, so we can take a relatively small amount of audience and tune and tweak the gameplay in terms of progression and otherwise.

So, you want to really make sure the game is engaging with people first. That's what CrowdStar does really well. You make sure that the game is engaging with people first rather than just throwing something out and hope it will monetize on day one.

Engagement is critical. We measure that with data literally every day. We can tweak and change features. We can watch player progression or speed at which players are progressing through content or otherwise. And we can adjust.

And so we work through that quite diligently on a new launch pretty much by the hour on some features and by the day on others, we're watching those feature changes and adjusting constantly.

And then when we're feeling comfortable and happy with what we would consider launch metrics, then you open up the funnel of real users.

The game is changing so rapidly. How do you have a sense of what's good metrics? Facebook is growing its audience -- though not as fast as it used to be. The landscape is changing, so how do you have a sense of what you even want to be at?

PH: Yeah, I think that's a good question. Obviously, two years of pretty solid data and metrics on existing games is a pretty amazing starting point. If you have 25 to 70 million gamer DAU [daily active users] a day, it's a pretty significant amount of data to understand what works and what doesn't.

There's so many different factors in that. It's like data: who's coming back the next day, engagement over longer periods of time, like three, five, seven days. There's a number of things we track very specifically. So, you can imagine there'd be a dashboard at launch, you'd have the top 12 things that are just consistently should be measured because they make sense for any game.

And then beneath that layer, there's a whole set of subcategories of very specific content, [which] measures for that individual game. So, yeah, we watch all our games consistently with data.

It's obvious why data is useful, but I think the skill is in interpreting and knowing actually what it means.

PH: Absolutely.

I've talked to developers who've said it can lead you down the wrong path if you don't understand exactly what the data represents.

PH: No doubt. Yeah. So, that's the great dynamic about working here. When you're in a meeting in the morning or studying the data, we're also looking at the game. If you think any like successful business in terms of a partnership, it's like left and right-hand side of the brain, there's this constant sort of art, science discussion going on.

So, yeah, you're right. I mean, if you just follow digital logic, you're kind of doomed because if you're not thinking about how we should be sort of amazing and delighting people, no matter what game business you're in, whatever platform you're in, if you're not amazing or delighting people, then you've missed. It doesn't matter.

What logic gives you... It can stop arguments dead in some cases. If, for example, you launch a game and everyone is kind of getting stuck and bored at level 9, there's not much of an artistic conversation around that. It's pretty clear.

But in terms of what CrowdStar is really motivated by and driven by is product. So, it's never an engineering task or an engineering feat. We always think about player experience first, always, everyone. So, yeah, it's a combination.

There's also the potentiality to misinterpret data. You can see "this is where people are getting stuck." But that won't necessarily tell you why. Or maybe people are getting through the tutorial too fast. It is too easy, or is it because you can just hold down the button?

PH: Yeah. Exactly. There's a number of ways with tackling that. The first port of call is community. So, our forums are just wild, as you can imagine, lots of activity managed really well by a community team.

And we listen to those guys first and foremost because they're really the people that are getting stuck. And the awesome thing about that is you're getting the reasons for that explained in pretty much plain English -- sometimes in language you don't really want -- but generally you get that in very plain speak.

Even the audience for Happy Aquarium is speaking in blue language? [laughs]

PH: Yeah. That was the amazing thing for me. They're very eloquent when they're describing what they feel is wrong and right in the game. And the community is so passionate. Once the game has been on the market for a year and you've still got two million people playing a game, they're pretty vocal still, and it's awesome. That's really our first port of call.

When I say data, it's not just 1s and 0s in Excel. It's data coming in right from community, from game health reports, from player feedback, from all these different places. So, my role is really for all those game teams and game leads is to help them prioritize that over a week and two-week process, so we're sort of always adjusting.