Focus Group Technique
Methodology. Team members gather together a small group of potential game players to discuss their opinions on the design of the interface, as well as game mechanics and story. Team members develop a flexible script of a dozen or so important questions that need to be answered. A moderator then guides a discussion in which participants are asked to discuss features or aspects of the game in turn or as part of an open discussion. Focus groups are often used to review game concepts or prototypes before they are put into a full development phase.
Example. A group of six to 10 users can, over the course of a couple of hours of discussion, often identify most of the aspects of an interface that will actively annoy or upset most players. This not only improves the game's quality and reception, but may also provide valuable insight into how (and to whom) to market the game.
Positive aspects of the Focus Group Technique
- The basic principle here is that several minds are better than one. Focus groups allow for a sharing of ideas amongst participants. New ideas can also be generated from this sharing of ideas and opinions, as can design strategies leading to more efficient and targeted gameplay, mechanics, and story.
- All the feedback from participants is collected all at once over a couple of hours. In an iterative design or decision making process this means you can collect a lot of data very fast, allowing for follow-on focus groups the next day or next week to assess new concepts or even new prototypes.
- Designers, developers and producers can observe the focus group directly or through recorded video and audio feeds. Often the feedback gained by the team is very powerful in nature -- "I can't believe they hated that one feature -- we thought that was really cool!" Moreover, when designers, developers, and producers can see the reaction of their audience directly this tends to eliminate the need for time consuming analysis and report-writing.
Problems with the Focus Group Technique
- This technique works best with an experienced moderator. In a pinch, a team member with good listening skills will work (producers are often good at this) but they must be very careful not to push the discussion toward ideas or answers that they prefer, or the focus group is useless. Leading questions from the researcher can manipulate the participants into answering questions a certain way, which reduces the usefulness of your findings.
- Several minds can easily be lead by one ego. If one participant has strong opinions and expresses them loudly and confidently, other participants in the focus group may feel pressured to agree with the dominant member. When this happens, much of the value of the group is lost. Remember that the loudest or most talkative participant does not always have the best ideas.
- It's important that all the participants feel that their opinions are equally valued. Very unequal levels of skill or experience can lead to participants deferring to the member who as logged the most hours in similar games. Often it is helpful to have groups made up entirely of beginners or experts. You may want to screen participants ahead of time to help with this.
Naturalistic Observation Technique
Methodology. Team members observe users playing similar games in the environments where they are generally used. Game tournaments, game LAN centers, and internet cafés allow for this type of observation, as do some specialized user experience play center labs. Like biologists observing animals in the wild, team members try to observe gamers interacting with the game as unobtrusively as possible.
This technique is used in the scientific fields of Anthropology, Sociology, and Social Psychology, but it works very well in development settings as well, assuming you have the time and resources to do this yourself.
Example. While engaged in the assessment of the front-end menu for a new game, team members observed that users spent a lot of time clicking on a particularly detailed and cool-looking piece of game art that was used as background decoration. Team members quickly realized that players thought the decoration was a control for the menu system, and were confused when clicking on it did nothing. When the position and format of the decoration were changed, the problem went away.
Positive aspects of the Naturalistic Observation Technique
- During traditional playtesting, gamers often react to the sterile lab and testing environment and behave differently as a result, answering questions and playing the game differently from how they would in a more comfortable setting. The use of naturalistic observation techniques can help to overcome this effect.
- With this technique, large amounts of information can be collected in a very short period of time.
- By carefully observing how users interact with the game, team members can often identify unexpected and critical information about how users perceive a game, as in the example above.
Problems with the Naturalistic Observation Technique
- Team members may not understand why a user does something because they cannot get inside their head to know what the user is thinking.
- Sometimes, the environment where observation is occurring may have an impact on how users behave. If one of the goals of a game tournament is to win as fast as possible and rack up a large number of victories without worrying about losses this could cause team members to observe play behaviors that would not occur during normal gameplay.
- It can be difficult to see and document exactly what users are doing, especially detail-oriented behaviors like choosing an icon or manipulating a slider bar.