Tools of the trade

White-Box Testing vs. Black-Box Testing

These two methods form the second axis of the testing grid. The amount of information available to testers about the underlying workings of the software is largely dependent upon which testing method they use.

In black-box testing, there is little or no information about the internal workings of the software; all a tester sees is what the end-user would see. Playing a beta build of a game on a test console is a good example of this.

White-box testing, on the other hand, uses internal functions or a second software suite (usually a debugger) to track what's going on as the software runs, relaying information back to the tester or to a test log as necessary.

Good examples of this testing method would be testing a software build in a programming platform's development environment or using automation software to test minor variances of an action repeatedly with a debugger running in the background.

While it seems like everyone would want to use white-box testing, the fact that an extra piece of software is running in the background and intercepting data as it flows through the software being tested is an important consideration.

This interception can interfere with the normal working of the software, which sometimes causes problems that normally wouldn't occur, or may even prevent problems that normally would occur. As such, it's important to keep a balance between white-box and black-box testing to ensure that the software in question receives thorough testing.

Testers

These people specialize in breaking software. They each have their own preferred ways of doing this, but most of them are competent with a variety of testing methods. The amount of access they have to the resources necessary to perform adequate testing, however, is often determined by their level of association with a developer.

Third-Party Testers

The lowest rung, so to speak. These testers are often the backbone of the testing process. Third-party testers are often contracted on a project-to-project basis. They usually have limited access to special testing equipment, though testers who show special talent with certain methods will frequently be given access to extra tools. Third-party testers are often used for black-box testing.

Hiring contract testers in large groups can often result in a mixed bag, but even testers who have little or no technical knowledge can usually perform both ad-hoc testing and test cases with positive results with the right training.

Second-Party Testers

Testers who work in the testing group of a subsidiary or secondary company under a larger company, second-party testers can be either contract or fully employed. Due to their closer relationship to the developers, they often gain access to more advanced tools. This often results in a stronger focus on test cases and white-box testing. Most second-party testers are at least moderately experienced in the testing process.

First-Party Testers

Testers that often work or communicate directly with the developers, first-party testers are usually full-time employees of the company they work for, though skilled contractors may occasionally be used in this capacity. They have the most access to testing tools, and they will often manage groups of testers in their tasks. Most first-party testers are also very familiar with the testing process and various development cycles.

The Tester-Developer Dynamic

This dynamic is why the levels of association described above are important. All too often the developer and the testing group they work with will find themselves at odds with each other on various issues, which often creates friction between the teams.

The further away one group is from the other, the greater the likelihood of this problem arising becomes.

Lack of tools or resources is often the greatest complaint with third-party testers, but disagreements on how to handle bugs are very common with all levels. Cost-effectiveness, time constraints, and feasibility are all objects of contention when bugs are accepted or written off. It should go without saying, but all parties involved should remember that they're all working toward the same goal: a polished, functional product.

To this end, developers should understand that many testers fall within the core demographic of their end-users, and sometimes the opinions of testers would be well heeded.

Likewise, testers should understand that developers have a greater understanding of where the project stands and will make decisions based on information that the testing team may not have access to. These situations can also be mitigated by sharing information between the groups, as well as sharing useful tools with each other.

Managing teams

The Need for the Separation of Tasks

Nobody likes to have their toes stepped on. When you've assigned a task to someone, be it a single person or a group, it's important to ensure that they have the chance to finish that task.

The reasons for this are twofold: first, it's good for the morale of the person or group that had the task assigned to them, as it shows a level of trust in them; second, it helps to reduce the chance of redundant effort.

Having more than one set of eyes on a test is always good, but it's usually best to wait for all the other tests to be completed first. The goal of separating tasks is to get a full battery of tests completed by the combined efforts of the entire team.

To that end, the tasks need to be appropriately separated and assigned to the teams that will be able to complete them most effectively. For instance, if a game requires gameplay testing, a text check, and certification testing, then there should be three teams assigned groups of the tasks necessary for those tests.

Once a team completes the tasks they have been assigned, then they should double-check the work already completed by other teams (with the exception of certification testing, which often has to be performed by people with special qualifications; those tests should be double-checked by another member of the certification team). This is one of the best ways to ensure that at least two people have seen each part of the software being tested.