Show Me the Money: Why the Measurement of PENS Can Surpass the Measurement of “Fun”

We’ve outlined the core elements of the PENS approach and highlighted some of the strong relationships with important outcomes, both with regard to the player experience and commercial success. Next we would like to present some more of our data in detail, taking the full PENS model into account and looking at the power of our complete motivational lightbox when competence, autonomy, and relatedness are working together to create the player experience.

Remember that one of our hypotheses is that the PENS variables are the power behind things such as enjoyment and perceived value. Hence our data needs to show that PENS is where the deeper action is. We accomplish this objectively through a variety of analyses, but will here focus on correlations and regressions.

A quick note on regression analyses for those who are unfamiliar with them. Regressions allow you to determine the relative contribution of different factors to your outcomes. As an example, let’s say you want to know what factors contribute to good grades in the classroom. You find two variables predict good grades: Student motivation and sitting towards the front of the class. To raise grades, do you focus on student motivation or reconstructing classrooms so more students sit near the front? Or both? It is important to know the relative contribution of variables to outcomes, and how one (motivation) may be behind the other (sitting at the front). Regression analysis allows us to do just that: We can look at groups of variables and see their relative contribution to outcomes. We can give credit where credit is due.

So let’s now illustrate this using two factors in our data: game enjoyment and the PENS measures. We recently finished an eight month longitudinal study of MMO gamers in which we asked them in March about their experience of fun/enjoyment (i.e. the usual kinds of questions developers ask during playtesting), and also asked them about their experience of need satisfaction in areas of autonomy, competence, and relatedness (i.e. the PENS model). We then talked to them again eight months later in November to see if they were still playing the game they reported playing in March, and to get their impressions of the game and its value.

For those still playing the game after eight months, our model showed better predictive value than enjoyment in the correlations, even though both related positively to enthusiasm (“This game rocks!”), and perceived value (see Table 5)

You might think this means enjoyment measures are a reasonable alternative to PENS and can equally predict some outcomes. But our data shows they don’t. When we ran our regression analyses to see who is really pulling their weight (remember the motivation v. front of the class issue?), the PENS measure related strongly to all of the outcomes, while enjoyment questions did not. In other words, only our need satisfaction components predicted these important outcomes, including sustained subscriptions, as seen in Table 6.

This directly supports our hypothesis that it is the deeper need satisfaction that is experienced in games that leads to their success, both in the direct emotional experience of the player (i.e. “This game rocks!”) and in commercial outcomes (such as perceived value and future intent to purchase). Apparently what “rocks!” about games is that they satisfy motivational needs, supporting our contention that our motivational lightbox lies behind the experience of enjoyment, and more directly measures what is important to the gaming experience in order to catalyze sustained player enthusiasm and commercial success.

How does this contribute value directly to a game’s bottom line? As the tables above show, in this MMO study only our model was able predict future play eight months later. Fun/enjoyment measures did not relate to continued play. Based on the data, we estimate that an MMO developer could increase retention (decrease churn) by 15-20% if they tested gameplay ideas using a PENS approach, which based on today’s subscription rates (i.e. about $15/month) roughly equates to about $25,000 in additional revenue every month for every 10,000 players. For a game with 100,000 subscribers, that could mean $3MM in revenue each year (mostly dropped right to the bottom line).

Let’s look at another study that shifts the focus from MMO’s to two adventure console games…