Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
March 9, 2021
arrowPress Releases







If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 

Five tips to make better games through feedback testing

by Kim Kohatsu on 01/22/21 10:58:00 am

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

Developers are data nerds. They love crunching numbers from App Store metrics or A/B tests on a finished mobile game. But what I wish more developers realized is how powerful qualitative feedback testing can be. Listening to players in the early stages of production and continuing to solicit their opinions is key to keep them coming back and spending money. 

Having worked with developers big and small, I’ve seen how effective surveys can be in driving successful mobile games. For the maximum return on investment, follow these five tips to introduce surveys and feedback testing as a critical step in your iterative process.

1. Include feedback testing in each stage of game design 

You probably already do plenty of testing: usability tests, QA, split tests, etc. But for such a data-led industry when it comes to monetization and retention, games studios too often leave the creative side of development to chance. 

Don’t miss this huge opportunity. Gather player opinions at every stage of the development process, and as you iterate, confirm your creative choices. 

  • Ideation - In the concept stage, solicit feedback on your game idea, the game’s name, and storyboards. You don’t need a working build; a simple game design document is enough to ensure you’re headed in a viable direction.

  • Design - Test your target audience’s reaction to art styles, character art, and UI. Use the feedback to improve the look and feel of your game.

  • Development  - Show character animations or game mechanics in action to gauge player reactions. Choose the best movements, music, voiceover, and sound effects. 

  • Pre-launch - Ensure that your App Store marketing assets, including the app icon, screenshots, trailer, and description, resonate with players. Test different ad creatives for Facebook and other networks if you’re planning user acquisition campaigns. Rather than testing creatives in the wild (and paying the ad spend), have confidence that your ads will communicate effectively before you start running them. 

Screenshot of PickFu test of three app icons

 

 

2. Test with a manageable sample size 

If you survey thousands of people right away, you will likely have too much data to digest in any meaningful way. 

Instead, start with around 200 survey respondents. It’s a small enough pool to be manageable while still being statistically valid. Plus, as most feedback testing is priced per response, you can start with a small testing budget and expand if you think the results justify it.

3. Build the right test audience

All feedback is not created equal. Your test audience must accurately reflect your end-users. If a role-playing game is aimed at women, for instance, men’s feedback isn’t as valuable. Think about which demographic and behavioral traits apply to your prospective audience, such as: 

Demographic traits

  • Gender

  • Age range

  • Education level

  • Marital status

  • and more…

Behavioral traits

If you’re not entirely sure of your audience yet, start with a broad audience and see which groups are most responsive to your ideas. Hone in on specific demographics and behaviors to build a picture of your target player. For example, if you’re ideating a new puzzle game, show the designs to a broad mix of gamer types and demographics. If you find that active puzzle gamers are the most enthusiastic about the designs, then as you move on to testing game mechanics, conduct your surveys with this targeted group. 

Pre-supposing your core demographic at the outset could add assumptions too early in the process. Continue to refine your game’s player persona. Once the game is launched, cross-correlate with any demographic insight you have from users. 

4. Don’t forget about competitors 

This nifty research trick can help you stay ahead of the pack: test your game assets against the competition.

Any aspect of your game, from concept through launch, can be put to the test. Collect outside feedback to understand why players prefer one game over another. If your game comes out ahead, you can proceed with confidence. If the other game beats yours, you still have time to change course.
Screenshot of PickFu competitive test of two mobile games

If your game is already launched but underperforming, competitive testing can help identify areas for improvement so that you can course-correct. 

5. Ask the right questions

Good survey answers derive from good survey questions. As you build your feedback tests, avoid these common mistakes: 

  • Don’t ask leading questions that may create bias. If you ask a question like, “What do you love about this character?,” your survey will collect only positive answers. Use objective language such as “Would you want to play as this character?” to get an honest picture.

  • Don’t present assets in different formats. If you show four icon designs but they aren’t the same resolution and size, survey respondents might react to the relative size or clarity of an icon, rather than its design. Keep aspect ratios and image sizes consistent. 

  • Don’t test more than one thing at a time. If you’re trying to decide on a good name for your game but you present each name in a different logo design, then survey respondents may react to the logo they like best rather than the name they like best, jumbling your results. Keep everything consistent except for the parameter you’re testing.  

  • Don’t overwhelm respondents. If you submit an entire screenshot of your App Store listing, then respondents will react to different parts of it: screenshots, description, name, and potentially even fields largely out of your control, such as star rating, price, or reviews. Instead, focus their feedback on one asset or question, so that you can methodically improve each component.

In summary  

Appropriately done, feedback testing is an invaluable way to perfect your game long before it gets as far as the App Store. Even a small early testing budget can generate considerable savings in resources and ad spend down the line. By confirming design choices and benchmarking your game with its intended audience, you can make data-driven decisions all along the way. And, as I said at the outset, developers love data. 

 


Related Jobs

Gear Inc.
Gear Inc. — Hanoi, Vietnam
[03.08.21]

[Vietnam] Senior Game Designer
Brass Lion Entertainment
Brass Lion Entertainment — New York, New York, United States
[03.08.21]

Principal Software Programmer
Bit Fry Game Studios
Bit Fry Game Studios — Portsmouth, New Hampshire, United States
[03.08.21]

Technical Animator
Bit Fry Game Studios
Bit Fry Game Studios — Portsmouth, New Hampshire, United States
[03.08.21]

Senior Technical Artist





Loading Comments

loader image