Noel Llopis

High Moon Studio developers Noel Llopis and Sean Houghton have been working with a unique build method called test-driven development (TDD). In an intermediate level lecture at GDC 2006, the programming duo discussed what TDD is, how others can use it in their development cycle, and the benefits of testing often--a primary focus of the methodology.

TDD is a cyclical, three-point procedure that moves as such: write test, write code, refactor, and back to write test. Llopis and Houghton are using TDD with C++ as well as Unreal code base.

TDD is simple in both concept and implementation. “It comes down to one very simple cycle. You think of a very, very small feature and ... you write a very small test for that feature. We're talking about something very small,” said Llopis. Then, the programmer tests the piece of code. If it fails, he says, “you write some code to make that test pass. So now you have some passing tests. The next step is the crucial one for TDD: you refactor things,” changing the code in whatever necessary ways without changing its functionality.

The whole cycle should take less than a minute, according to Llopis, stressing the importance of quick, small improvements. Although the process could take as much as three or four minutes, “if it takes ten minutes, it's not a good sign,” according to Llopis.

Typical problems programmers face, such as bugs, unstable builds, risk (when implementing changes), and early freezes, are all addressed in TDD.

Benefits of the TDD system include simplicity, modularity, and instant feedback.

“The main root of the problem [is that] code gets out of hand too quickly,” says Llopis. With TDD, the code is self-contained and is tested in isolation. “What is the first thing that you do with TDD? You write a test. You're writing your code from the point of view of the user of your code. You are your own first user,” says Llopis. “And you end up with really much more usable code.”

TDD also provides a safety net, freeing programmers to make the kinds of changes they actually want to make, rather than working around a complex in-place codebase. “Otherwise, how do you do major changes? You run the game, you run a couple of levels?”

TDD also improves feedback. Programmers not using TDD only receive feedback at often wide-spread intervals, such as milestones, which might come once every two months, or possibly at interation points (every two to four weeks). And though nightly builds or automated builds, which can give feedback once a day or once an hour, TDD, by its nature, provides even more frequent feedback: “every 30 seconds to three or four minutes,” according to Llopis.

“In the worst case, if something horrible happens, you only lost three minutes. It also tells you that every three minutes, you're making progress,” he says, which can boost a coder's confidence and perceived value. “You're not just more confident, but it changes the feeling you get as a programmer working on your code,” he adds, commenting on the “joy of programming again with instant feedback.”

	Sean Houghton

Houghton provided examples of how TDD can be used in a game. Most of the TDD sample code was less than 15 lines total, reiterating the importance of keeping the unit tests small.

“It's the most simple test you can run using TDD: testing return values,” says Houghton, whose example involved a character with a shield that's sustaining damage. He used CHECK_EQUAL and GetLevel to test his code.

The tests, he says, can be put in four places: testGame.exe (links with Game.lib); #ifdef UNIT_TESTS (used on Unreal); GameTests.DLL; or GameTests.upk (for Unreal script tests). “You can put them anywhere you want depending on how your build system is set up. Every time you build, you want to run your unit tests,” he adds.

“You want your unit test to run as part of the build. A failed unit test is considered a failed compilation.”

Llopis showed more examples of running a couple of tests on the shield component, interacting with the code on the screen to give the programmer audience a strong understanding of exactly how the scenario would play out for the coder. Llopis repeated the TDD cycle a number of times until his code compiled properly.

“We compile. The build fails. [The program] highlights the test that failed for you. This is as important as a syntax error. Let's fix it by implementing the code that failed.

We'll use the simplest possible update. Clearly, this is not how we're going to ship the game to the customer, but right now, just make the test pass.”

With each iteration, Llopis created a new variable, which isolates the problem when the tests fail. “Only try to get one part to pass,” he recommends.

Houghton furthered the examples by showing how a mock object can be used to keep up the speed and cyclical approach that TDD promotes.

Houghton and Llopis shared a few of the best practices they've established using TDD:

1. Test only the code at hand, isolating tests or objects (using mock objects to refrain from having too many objects).
2. Keep tests simple. Houghton and Llopis shared some examples of physics code tested at High Moon Studios to show how precisely how simple they mean. The code shown contained about 15 lines.
3. Keep tests fast. With the TDD methodology, tests should not take 20 or 30 seconds to run. Programmers should aim to run 500 or 600 tests in about half a second.
4. Keep tests independent. “If a previous test messes up the state, then the next test is going to fail. .. Restore to a cleanroom state before you're done with your test,” says Houghton.

“Not only are programmers willing to do the TDD work, they like it so much, that they go home and they use TDD on their home projects. They really see the benefits, and their work becomes much more enjoyable,” says Llopis.

A paper version of this talk, including more detail about testing API, middleware, and running tests on consoles, is available on Noel Llopis' web site: www.gamesfromwithin.com.