[Continuing his series on playtesting, ex-Ubisoft veteran Pascal Luban (Splinter Cell series) examines the practicalities of getting consumer feedback on your game.]

Proximity, responsiveness, relevance... these are the watchwords of efficient playtests.

In the previous installment of this article, I had explored the reasons for the rising importance of playtests in game development.

In an industry where games represent increasingly high financial risks for publishers, playtests have come to function as a strong guarantee for quality gameplay. I will share with you today my experience regarding the methodology employed in preparing and conducting them.

Heeding the Clients: The Design Teams

Foremost, one must be aware of a fundamental say: the role of playtests is not to redo the design in place of the design teams -- for either game or level design. They are instead conducted to help them. This observation is crucial, because it drives the entire approach to playtests.

Firstly, we must respect the hard work of the design teams. Having had my own responsibilities in game and level design, I know how difficult it is to make "a good game". We must respect those who put their whole hearts into building the best game possible; we must not scorn or undervalue their work.

Secondly, playtests must adapt to the needs of the design teams. Good tuning for maps or gameplay mechanics is often the result of trial and error. Knowing this, designers should require experimentation; playtests can afford them the opportunity to test out their hypotheses regarding design issues, and must therefore adapt to particular needs as they arise.

Lastly, playtest results must be made available to the concerned parties as soon as possible, as time allotted for game development is always short.

Preparing a Playtest Campaign

A playtest campaign generally requires around one month of preparation. We must first define its objectives, because they will determine what types of playtesters we shall have to recruit, the scale of the sessions (1, 2, 4, 8, 12 players), and their duration (from half a day to a full week).

We will also have to attend to the logistics as well as the legal framework (non-disclosure agreement, eventual monetary compensation for playtesters when sessions last over a half-day, etc.) And we must, of course, prepare the design teams to effectively utilize the playtests.

One does not grow the best crops in dry land; a playtest's effectiveness is rooted in the playtesters themselves. Half the battle in running an effective playtest campaign lies in wisely choosing playtesters, which requires investment of time, energy, and perhaps a bit of money and patience.

Recruiting takes time: we must not only hire as many candidates as possible (in order to have a solid pool of playtesters). We must also evaluate them. The purpose of evaluation is obviously to judge the candidate's gaming competence, but also his ability for analysis and self-expression.

Evaluation may take several forms. An initial selection can be done through a more or less thorough questionnaire, to be completed by the candidate. The true evaluation, however, must be done during the sessions themselves, where we can observe the candidates at play.

We must establish a protocol for obtaining the most consistent results possible. There is no "all-purpose" evaluation protocol; we must also be able to adapt to specific circumstances as the situation mandates.

When I built a playtest structure at the Bucarest Ubisoft office, I encountered an interesting problem: we needed playtests for console games, but all the players we could find locally were exclusively PC gamers. I had to set up a specific protocol to evaluate the ease with which our Romanian candidates could adapt to console gaming.

Ubisoft's Splinter Cell: Chaos Theory

The protocol consisted of briefly explaining the gameplay controls of a complex game (the multi-player mode in Splinter Cell: Chaos Theory), and then setting them loose in the game in order to gauge the speed at which they adapted to the gameplay. This selection method proved to be quite efficient.

Candidate selection must therefore be done according to a given playtest campaign's objectives. We may have need of only extremely skilled players who have already mastered the genre, or we may require novices, if the objective is to playtest the accessibility of the game.

Communication regarding playtests also takes time. Before candidates can turn up on your doorstep, they must first be made aware of your need. In my experience, while recruiting through generic classified ads will yield a high number of candidates, many will be too young (careful of those labor laws!), and most will be only casual gamers.

A good way to recruit experienced players is to make use of forums, gaming clans or specialized stores. It takes much more time but I always got great playtesters this way. In playtesting, quality matters more than quantity!