Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
June 24, 2019
arrowPress Releases
If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 

Analyzing and Interpreting Feedback From Your Community

by Brian Peterson on 11/18/14 01:51:00 pm   Featured Blogs

6 comments Share on Twitter    RSS

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

Monument Valley is currently being discussed as a cautionary example of the dangers of charging for paid content updates on mobile. The game received a “barrage” of one-star reviews when it updated last week with an option to buy paid content. Bad reviews are bad… right?

I don’t know whether putting paid content in your mobile game is a good idea – that’s a question for someone else, or perhaps a higher power. I’m much more interested in how players reacted to the introduction of paid content in Monument Valley, how developers reacted to those players’ reactions, and how developers can learn to interpret feedback from their games’ communities more effectively.

Any update to a game is going to upset some of that game's players. This is especially true when a real or imagined currency is involved. It's natural for developers to focus on the negative reviews: game designers and programmers usually enjoy solving problems, and one-star reviews are like being handed a list of problems waiting to be solved.

That’s not how people work though. No game will ever make everyone happy, and no game will ever achieve 100% positive feedback and unlock the “Perfect Game” achievement. Since it’s impossible to please everyone, there will be unhappy players sharing negative opinions about every game, regardless of its quality. If the mere presence of negative feedback is not enough to go on, how do you take a wider look at player feedback and decide whether a change made to a game has been received well or not?

It’s important to first look at historical trends. According to AppAnnie, Monument Valley's most-frequently rated iOS version before the introduction of paid content was 1.0.3. This version has almost 13,000 ratings, with an average rating of 4.4 stars.The version with the most ratings after the introduction of paid content received a greater percentage of one-star reviews than other versions. But the perceived “barrage” was more like a brief yelp of discontent, which ustwogames was able to drown out by leveraging their dedicated fan community.

 

 

 

 

 

 

 

The version that was current when this tweet was posted, 2.0, now has almost 5,000 ratings, with an average rating of 4.5. This is higher than the 1.0.3 version’s average. If most players were really unhappy with the game, there would have been enough one-star ratings to have had a negative effect on the review average, regardless of ustwogames getting sympathy on Twitter.

Timing is also very important, and is tied closely to players’ motivations for leaving reviews. One-star reviews began appearing very quickly when players realized that the new levels would cost additional money. The initial reaction was perceived as negative simply because of the timing. Players who were unhappy with the change reacted by getting upset and leaving a bad review to let the developer know they were upset. This takes only a minute or two at most, and after the initial backlash these reviews died down in frequency.

For players who were happy to pay for more levels, their reaction to the update was to buy the new content and play the new levels. Leaving a positive review does not come immediately or naturally, which is why ustwogames was smart to provide their fans with a call to action on Twitter.

It’s also important to identify who is doing the complaining. In the case of Monument Valley, the players who made fan art, dressed up as Ida for Halloween, created papercraft totems, and bought $50 prints from iam8bit were not complaining on Twitter and posting one-star reviews. That would have been a very clear indication of the existing community’s rejection of the new paid content. 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

There is probably a large overlap between players who are complaining about the game’s new paid content and players who complained that the original game was too short and not worth the money. These are players who may not have purchased or played the game if they had full knowledge of what they would eventually dislike about it. They became “accidental players” in a way: they may enjoy some aspects of the game and may be invested in it due to sunk costs of time or money, but they were never part of the game’s actual target audience. It can be very confusing when players like this post feedback about a game they probably didn’t enjoy in the first place.

If a game is very good, like Monument Valley, this group is very small. Changing your game to win over a small group of accidental players can be tempting, but it may be at the expense of the games’ more-dedicated fans, the developers’ revenue, or both.

Highly-engaged fans appreciate Monument Valley for its level design, art, and story. One half-baked suggestion I’ve heard is to sell the additional levels as a new game, Monument Valley 2. But the developers have made it clear that these levels were designed to fit into the original game, and were never planned to stand on their own. Fans who appreciated the work that went into the story and pacing of the original would likely be disappointed by a new game that was just a glorified level pack. It’s much more important to keep your existing fans happy than to attempt to appeal to people who already don’t like your game.

Now let’s talk money: Monument Valley is already a huge success, selling well over a million copies despite having a price of $3.99. Would adding free levels to the game have expanded the audience of new players significantly? It’s doubtful that the few people still on the fence about purchasing the game will be convinced by 8 more levels. The game can still be beaten in an hour or two, so players who judged its value by measuring the length of time spent playing it are still going to be unsatisfied, free levels or not.

Based on this situation and many others I’ve witnessed, here are a few things to remember when looking at player reviews and ratings:

  1. Familiarize yourself with the usual levels of positivity or negativity for similar games. If you’re consistently updating a game, look at the feedback trends that occurred when your game was changed in the past. You’ve probably had some updates that went better than others. What did feedback from your players look like each of those times?
  2. While you’re looking at old feedback, pay careful attention to the timing. Negative feedback gets posted much faster than positive feedback, and it always takes some time for your fans to warm up to something new. Sometimes the best thing you can hear immediately after an update is silence – it could mean that everyone is busy enjoying your game!
  3. Reach out to your community for feedback. Happy players are quieter than unhappy players, so they may need a nudge or a clear opportunity to share their thoughts with you.
  4. Pay attention to where the feedback is coming from. If your game specifically targets and appeals to middle-aged women, feedback from a teenage boy should be taken with a grain of salt. Complaints from outside your target demographic are not nearly as important as complaints from your current fans.
  5. Identify players in your community who you can always depend on for an opinion one way or the other. Some of these players may always be negative, with criticism about everything in your game. They are still important to listen to – if they compliment something in your game, you know it must be really great! Most importantly, stay in touch with the fans that really love your game and are always happy to talk about it. If something makes these fans upset, that’s when you should really get worried.
  6. Delegate the task of reading reviews to a community manager or customer support rep, most of which have developed a layer of very thick skin and can summarize trends rather than focusing on specifics. Do not read every single review and comment for a game that you worked on! Working on a project for months or years will naturally cause negative reviews to feel very personal. This can cause minor or irrelevant feedback to be magnified in importance and significance. It’s natural to focus on the negative and only read the bad reviews, but it’s much more important to know how you could improve your game for the players who are already fans.

Thanks for reading, and feel free to post a comment and share your thoughts or stories about community feedback!


Related Jobs

Gear Inc.
Gear Inc. — Hanoi, Vietnam
[06.22.19]

Technical Director
Legends of Learning
Legends of Learning — Washington, DC, District of Columbia, United States
[06.21.19]

Senior Unity Engineer - $140k - Remote OK
Wargaming.net
Wargaming.net — Chicago, Illinois, United States
[06.21.19]

Server Engineer
Wargaming.net
Wargaming.net — Bellevue , Washington, United States
[06.21.19]

UI Engineer





Loading Comments

loader image