Gamasutra: The Art & Business of Making Gamesspacer
Finding Out What They Think: A Rough Primer To User Research, Part 1
View All     RSS
January 17, 2019
arrowPress Releases
January 17, 2019
Games Press
View All     RSS






If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 

Finding Out What They Think: A Rough Primer To User Research, Part 1


April 24, 2012 Article Start Previous Page 2 of 4 Next
 

Time for the Crunchy Stuff

Are you still with me? Great. Here, then, are the research methods that I will cover, in varying degrees of depth, in the rest of this document: focus groups, heuristic evaluation, and questionnaires/surveys.

Focus Groups

This method can be something of a dirty word (and is definitely part of that whole "design by committee" problem). So let us deal with it first and get it out of the way.

You are probably familiar with focus groups, even if you've never seen them used in person. Basically this is where you get a bunch of people, have them play your game, and then put them in a room to talk about it. They can be free to talk about what they like and what they didn't like, but you also have a facilitator in the room who can ask specific questions of interest.

This process can also be used quite early on in development, where instead of getting people to play the game, you give a presentation or talk about ideas for the game and get feedback on that.

The advantages of focus groups are that they involve a lot of people, so you can get more feedback. They can also be somewhat efficient, as everyone is together in one place, and the facilitator can ask follow-up questions. So if someone mentions they like or dislike something in particular, you can gain a bit more detail on why.

Although at the same time if you are not careful, focus groups can get away from you, and end up wasting far too much time. To avoid this last problem you really need a good facilitator to run a focus group. The facilitator has to be strong enough to guide the conversation to areas that are useful, but not dominate the discussion.

Probably the biggest risk with a focus group, and one of the reasons why focus groups are not often used, is that just one or two members of the focus group can dominate the discussion. Due to group pressure, you may not hear from other people who have valuable insight into the game. Focus groups also have a tendency to get more into discussing solutions for issues rather than just the issues themselves. This is not what you want.

Finally, focus groups are a subjective method, and all you have to go on is what people say -- and as much as we like to judge people on the "attitudes" they hold, and think that they predict behaviour (they usually don't), we all know that what people say is not always what they actually do.

Pros

  • More people can mean more feedback (although see cons...)
  • Gets everyone together in one place
  • Allows for follow-up questions
  • Can be useful when discussing concepts

Cons

  • A good facilitator is required
  • Strong voices may take over and reduce feedback overall
  • Too many solutions, not enough issues
  • What people say is not always (or even often) what they do

Heuristic Evaluation

Heuristic evaluation is where you get an expert (or experts) in games user research, get them to play your game, and then they evaluate it on a set of criteria (heuristics). Kind of like a scientific game review... Kind of.

Basically, to do this, the expert(s) will use a list of heuristics, which are basically rules or mental models, and give you feedback on whether your game fits these heuristics, and where problems might come up. These heuristics can vary, but here is a selection of some possible heuristics listed in a 2009 article [PDF link] by Christina Koeffel and colleagues:

  • Are clear goals provided?
  • Are the player rewards meaningful?
  • Does the player feel in control?
  • Is the game balanced?
  • Is the first playthrough and first impression good?
  • Is there a good story?
  • Does the game continue to progress well?
  • Is the game consistent and responsive?
  • Is it clear why a player failed?
  • Are there variable difficulty levels?
  • Are the game and the outcome fair?
  • Is the game replayable?
  • Is the AI visible, consistent, yet somewhat unpredictable?
  • Is the game too frustrating?
  • Is the learning curve too steep or too long?
  • Emotional impact?
  • Not too much boring repetition?
  • Can players recognize important elements on screen?

The article itself lists over 29 of these heuristics, and goes into much more detail than I have provided here, so I recommend reading it if you have the time.

The good thing about heuristics, in my opinion, is that even if you aren't an expert, they can provide a list of things for you to think of when you look at your game. For instance, does it provide enough feedback to players that their actions are affecting the world? Does it force players to hold the controller in an awkward fashion? And so on. Again, this may seem like common sense stuff, but it really is amazing how often so-called "common sense" is not common at all.

Now, one obvious advantage to heuristic evaluation is that you only need to use a small number of experts (just one in some cases), and being experts they know what they are talking about. This also leads to the problem that you do need experts, and where do you find those? Plus have you found the right one(s)? The types of heuristics used can vary by expert, and of course should fit your type of game -- so this is obviously important.

Also sometimes experts can be a bit too expert and miss stuff that might be a problem for novices. This is because as we become more experienced at something we no longer need to consciously consider everything that we perceive and do, whereas someone who is still learning is still thinking about what they are doing all the time. This is why generally if you want to see something done well you watch an expert, but if you want to learn how to do something it is often best to ask a novice.

Pros

  • Smaller number of people needed
  • Relatively fast turn around
  • Experts are expert

Cons

  • Where do you find experts?
  • Did you find the right one?
  • Experts can be too expert

Article Start Previous Page 2 of 4 Next

Related Jobs

Crystal Dynamics
Crystal Dynamics — Redwood City, California, United States
[01.16.19]

Camera Designer
Deep Silver Volition
Deep Silver Volition — Champaign, Illinois, United States
[01.16.19]

Mid/Senior Multiplayer Programmer
Robot Entertainment
Robot Entertainment — Plano, Texas, United States
[01.16.19]

Technical Artist -- Rigging/Pipeline/Tools
Game Circus LLC
Game Circus LLC — Addison, Texas, United States
[01.16.19]

Technical Artist





Loading Comments

loader image