Introduction

Many student games suffer from a severe lack of play-testing. Between classes, homework, and projects, most teams are not willing to work testing into their schedule. It seems too time consuming and not nearly as beneficial as using that same time to code. My teammates and I had all made this mistake in the past, but resolved to change for the better this year. Our play-test schedule ended up working out really well for us, and the game has exceeded our expectations in every way.

In this article I will be discussing some of the ideas and strategies our team adhered to while play-testing Toblo, a game developed at DigiPen Institute of Technology for our junior year project. We began with a core gameplay which revolved around building towers. The game was to be relatively slow-paced, with thought and care going into each block placement. What we now have is a very fast-paced game of ‘capture the flag’ in which the player throws blocks and destroys everything in sight. This design overhaul occurred shortly after our third play-test, during which it became abundantly clear that people had more fun knocking the towers down than building them. The most visible benefit of our play-tests was an entirely new core gameplay, but the tests also gave us plenty of other insights as well.

Toblo Kaboom

Testing Toblo

Like first attempts in anything else, our initial play-tests were great learning experiences. If there was a way to screw something up the first time through, we made sure to find it. Luckily, we were also able to learn from those mistakes and eventually turned our tests into an invaluable tool for improving the game. As Toblo’s testing director, I hope my advice can help other students to avoid falling into the same traps my team did.

When approaching a scheduled test date there are a couple things to keep in mind. Most importantly, avoid adding new features less than twenty-four hours before the play-test. Spend that last day ironing out any show-stopping bugs or crashes that have popped up, not introducing new ones. We knew to avoid such things going into our tests, but frequently went against our better judgment. Keep in mind that a new feature which seems absolutely necessary to implement at the last minute has high potential to ruin the entire test. We made this mistake quite a few times, the most notable being when our new flag capturing logic prevented testers from even capturing the flags!

Another pre-test precaution to consider is disabling any features which are not yet ready. Like moths to a light bulb, testers will almost always focus on the things that are obviously not working. Rather than give feedback about the things being tested, they are going to give bug reports. This is rarely the type of thing testers should be worried about. Taking a few extra minutes to get those features out before the play-test will save the frustration of sifting through useless feedback later on.

Watching the players closely during the test can lead to some valuable insights. Their reactions and general mood during gameplay can be a fantastic measure of the game’s best features. Toblo's design shift actually came as a direct result of this method of observation. Our team noticed people becoming vocal and happy any time they were throwing blocks and destroying things. When this was contrasted with a lack of visible reactions while towers were being built, we realized that something had to change. It was at this point that our focus shifted toward the destruction that our players were already enjoying so much.

Keeping a close eye on the game itself is another useful testing technique. People will break the game in new and creative ways which the developers never could have imagined. One of our more memorable testers had so much trouble figuring out the timing of his jumps that he would repeatedly jump into block walls until he got stuck. Bugs like this are rarely seen while playing the game normally, so make note of when and how they are occurring. Nothing is more frustrating than seeing a bug during the test and forgetting how to recreate it.

Toblo Towers

It is a good idea to prepare a series of questions beforehand which will be presented to all testers. Make them as specific and focused as possible. Open ended queries may seem like a good idea at first, but should be avoided for the most part. It is generally much better to use focused questions which generate consistent feedback, even if it is negative. In fact, consistent negative feedback is one of the most valuable tools a team has when tweaking the gameplay. One or two people who complain about something may or may not be right, but 20 or 30 people all requesting a change cannot be ignored.

One thing learned along the way was to ask questions in person, as opposed to handing out sheets of paper. Confronted with a questionnaire, many people will just look to finish quickly and move on. When engaged in a conversation, they may feel that their feedback is more valued and give insightful answers. It should also be easier to get a feel for how much they enjoyed playing. A written response rarely conveys the same emotion that is possible through speech. After asking the prepared questions, always get the tester’s own thoughts about what could possibly improve the game. Almost all of these suggestions will be thrown away, but some of them turn out to be excellent ideas. Assuming that the team always knows what’s best is an easy way to make a game nobody wants to play.

We have taken two different approaches to play-testing Toblo. The most frequently used method has been a standard organized test in which the players are fully aware they are testing the game. This should be used extensively during the first stages of playability, and well into the final stages of development. The techniques discussed thus far were employed during this type of test.  We conducted these tests in DigiPen's main computer lab, where we grabbed anyone who happened to be nearby. We found that free candy helped lure a steady stream of fresh players. After a few hours of testing, the team usually had plenty of useful feedback to dig into.

Northwest Games Festival

The second form of testing, which we have started to utilize closer to the end of our development cycle, is what we call a blind test. As far as the players know, they are just trying the game out. Unaware of the test, these players will simply look to enjoy the game. On the other hand, people who know they are play-testing tend to look for bugs and dig for things to critique. Our first attempt at a blind test was on June 3 at the Northwest Games Festival in Portland, Oregon. We were able to get a massive sampling of new players with all types of play styles. As the festival progressed we witnessed which gameplay ideas were working out and which still needed tweaking. We also unearthed a multitude of bugs which hadn’t been seen before.

We learned to waste no time before compiling feedback after a test. All those notes that made complete sense while jotting them down tend to get progressively more cryptic as time passes. It is helpful to organize all the feedback into broad categories at first. This will make it easier to prioritize changes based on any areas which received overwhelmingly negative responses. When mostly negative feedback is received about any one thing, it absolutely must be addressed before the next test. Trying to test again without the necessary changes in place will usually produce completely identical results. If testing isn’t revealing any new information, the same time could be better spent elsewhere.

After a few testing iterations, there should be a noticeable improvement in how easy the game is to pick up and play. Once unleashed on the public, it will be fine-tuned and ready to go. For Toblo, the Northwest Games Festival represented the first time anyone outside of our school had seen the game. We were a little nervous, but also confident that we had a fun, playable product. We knew people were going to figure out how to play the game, and have a good time doing it. The team felt this confidence because we had already gone through many valuable play-testing sessions, and had worked hard to get our game's "first five minutes" to be as fun as possible.

Northwest Games Festival

Conclusion

While play-testing itself won’t automatically make a good game, it can give your project the extra push needed to go beyond just-another-student-game. We became game developers because we want to see other people enjoy the things we create. There is a much better chance of this happening when play-testing is used effectively.

_____________________________________________________