Gamasutra: The Art & Business of Making Gamesspacer
arrowPress Releases
May 31, 2016
PR Newswire
View All
View All     Submit Event






If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
Do we really need game testers?
by Johan Hoberg on 03/07/16 02:37:00 pm   Expert Blogs   Featured Blogs

7 comments Share on Twitter Share on Facebook    RSS

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

Do we really need game testers when we move to a more Agile way of developing games? If the development teams own the quality of their product and do their own testing, is there really a place for dedicated game testers? Play tests with real users/untrained testers is one thing, but here I am talking about professional testers performing testing throughout the game life cycle. To answer this question we need to talk about competence instead of roles. In a cross-functional development team, is there a specific competence that a game tester can bring to the table, or can they just be replaced with more coders that perform the testing themselves? I will try to answer this question based on my own experience as a software tester, and introspection into how I approach testing problems.

To find an answer, we need to break down test competence into its components, and see if this competence is unique in any way, and if it adds value to the development team.

Let us start this breakdown with trying to think about some general competencies we expect game testers to have, but which are not unique to game testers, and is something we expect from any developer or engineer. This will obviously not be a comprehensive list, but let us try to cover many competencies we usually associate with game testers, so that we can exclude them from the elusive test competence, to get down to the core of what a game tester actually brings to the table in a development team.

Competencies we (might) expect game testers (and most other people) to have, which have nothing to do with test:

  • Communication skills
  • Collaboration skills
  • Agile mindset
  • Understanding business value
  • Quick learner
  • Critical thinker
  • System modeling skills
  • Coaching skills
  • Understanding/writing code
  • Reporting skills
  • Risk analysis skills
  • Review skills
  • Being an active listener
  • Being great at giving feedback
  • Usability skills
  • Domain knowledge
  • Fine motor skills
  • Reading comprehension

Even if you had all these skills (and you would be a great person if you did), there is nothing in this list that specifically requires a tester. A great developer probably has many of or all of these skills, and many additional development-related skills.

So if test competence is not covered by the list above, what is it then? To understand this, it is my belief that we must look in the Cynefin Framework [1]. There are many different types of problems, and according to this framework they can be divided into four categories; Obvious, Complicated, Complex and Chaotic.

 

Sidebar: How does complex behavior arise?

“The behavior of a complex system is often said to be due to emergence.” [3]

“Emergence is a process whereby larger entities, patterns, and regularities arise through interactions among smaller or simpler entities that themselves do not exhibit such properties.” [4]


 

It is my belief that higher levels of test competence are related to the ability to handle complex test problems. So what does this mean? First, let us define a complex test problem.

Obvious Test Problems: Tests in which the relationship between cause and effect is obvious to all

Complicated Test Problems: Tests in which the relationship between cause and effect requires analysis or some other form of investigation and/or the application of expert knowledge

Complex Test Problems: Tests in which the relationship between cause and effect can only be perceived in retrospect, but not in advance

So a game producer should have no problems solving an obvious test problem, and a game developer should have no problems solving a complicated test problem, but it is my belief that a game tester is better suited for solving complex test problems due to their specific core competence.

Let’s take a step back and look at the implications of what I just wrote. This means that I see no reason to have dedicated game testers to handle obvious and complicated testing problems. This can be done by developers, and in some cases even producers, designers and other roles. Obvious and complicated testing problems do not require high enough test competence to justify a dedicated game tester on the team. Sometimes you might however need a tester to coach the members of the team to raise their test competence to a level that allows them to handle these obvious and complicated problems.

The next step is to define this test competence I keep mentioning. I believe it consists of (at least) the following components:

  • Exploring a complex system through tests
  • Modeling unpredictability in a complex system
  • Provoking a system to reach an unpredictable outcome
  • Having a toolbox of unpredictable behavior

So what does this actually mean? What is it that I expect an experienced tester to achieve with the four bullets above? How does this add value to the development team?

If an experienced tester is confronted with a complex test problem, then this is how I expect them to approach this problem:

  • They start with a software system under test and some information about this system (could be requirements, risks, historical data, etc.)
  • Based on previous experience (either domain specific or general) and/or based on a toolbox of testing heuristics [2], and the system information, I expect them to come up with a number of test ideas
  • Based on these ideas, they then model a faulty system (in their mind or documented in the form of high-level test cases or test missions) which could show an unpredictable behavior
  • They then provoke the system to try to reach this faulty state (which may or may not require specific tools, and/or understanding of the system)
    • Documenting how to provoke the system to reach the faulty state would probably result in a step-by-step detailed test case
  • If they succeed they may have found a potential bug, and if the system does not show unpredictable behavior then the system works according to specification
  • They then continue to explore the system using more test ideas, but also equipped with the new information the previous test provided

By approaching a complex test problem like this, I believe that the game tester will provide value to the development team by providing them with information about how the system actually works, and by helping them understand and disperse the complexity of the system.

So a game tester needs a wide array of skills to be effective (the first list of competencies), but they also need a strong test competence, which is what makes them unique in the development team, and allows them to provide value in the form of a better understanding of the complexity of the system.

Everyone in a development team can test, but not everyone is equally equipped to handle complex test problems.

So do we really need game testers? The answer is yes, and no.  What we need is test competence. If we can get this competence into the development team in some other way, then no, we don’t need dedicated game testers. If we cannot, then having game testers on the team may be the only solution.

 

References

[1] Cynefin Framework

https://en.wikipedia.org/wiki/Cynefin_Framework

[2] Heuristics Cheat Sheet

http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf

[3] Complexity

https://en.wikipedia.org/wiki/Complexity

[4] Emergence

https://en.wikipedia.org/wiki/Emergence

 


Related Jobs

Respawn Entertainment
Respawn Entertainment — CHATSWORTH, California, United States
[05.31.16]

AI Software Engineer
Respawn Entertainment
Respawn Entertainment — CHATSWORTH, California, United States
[05.31.16]

Gameplay Software Engineer (mid-senior level)
Respawn Entertainment
Respawn Entertainment — CHATSWORTH, California, United States
[05.31.16]

Graphics Software Engineer (mid-senior level)
Turbine Inc.
Turbine Inc. — Needham, Massachusetts, United States
[05.31.16]

Senior Android Engineer





Loading Comments

loader image