What Went Wrong
I’d like to say that we had fewer problems developing AoK than we did for AoE, but it didn’t turn out that way. Some problems listed in the AoE postmortem were addressed in AoK and others weren’t. And like that band going back into the studio to record a follow up to a hit album, we encountered a whole slew of brand new problems, many of which we found we were just as unprepared for as we were the first time around. I’ve tried to include some of the issues that became more important due to the fact that we were making a sequel to a successful game.
1.We still don’t have a patch process
This was a problem area from the AoE Postmortem, and as of this writing it still has not been addressed. I outlined the reasons we needed a process to issue patches for our game in a timely manner in the AoE Postmortem. Additionally, a new reason reared its ugly head: cheating in multiplayer games. At first people found bugs in AoE and exploited them to win unfairly. Then it got even worse. Programs called “trainers” were developed that would actually modify the game’s code while it was running to allow players to cheat.
The wireframe model for the trebuchet unit (right) and the
fully textured and skinned version (left). The trebuchet
proved to be one of the most popular units in the game.
Being the developer — not the publisher — of AoE, we don’t have the final decision if or when a patch is to be released. As a result, all during 1999 our reputation as developers was assaulted by fans who saw us as uncaring about the problems that were driving people away from online play of our games. The topic of cheating in multiplayer games is so extensive I hope to do an article on it in the near future. We addressed this problem with our publisher and were promised a patch process. Unfortunately, AoK shipped with a couple bugs that seriously needed addressing in the short term. They’re not show stoppers, but if not addressed soon, the game’s (and our) reputation may suffer another black eye. If a patch for AoK is out by the time you read this, then you can conclude that we finally established our process.
2. Unfinished versions of the game got out
This is a problem that is born of success. Prerelease versions of nearly all games wind up circulating in pirate channels known as “warez.” This happened with AoE. Imagine our surprise at reading an entire review of the game (an alpha version) eight months before it was released. Fortunately, almost no one bothered with it until the game was properly released because nobody knew much about it. AoK was a completely different story. It was a highly anticipated sequel to a very successful game, and the various warez sites were tripping all over themselves to get a copy of the latest build. And get a copy they did. They were usually only one or two weeks behind our latest build. It seemed as though copies were leaking out from every imaginable source — play-testers at Microsoft, previews sent to magazines, even internal sources. Unfortunately, positively identifying and fingering the culprits was almost impossible. There were hacking attempts on our FTP server and network, though the real rub came from the pirates in Hong Kong and Singapore. They took the warez versions of AoK, burned them onto CDs, added some cover art, and sold the game throughout the Pacific Rim. In Korea, the CD vendors operating in front of Microsoft’s headquarters had a warez version of AoK for sale. Warez versions were even turning up on eBay. Though we doubt bottom-line sales were hurt much, our pride certainly suffered. Any of our future games will probably require connecting to a secure server of our own design to operate, even for single-player games.
3. Play-testing had a lot of problems
This item is a catchall for several problems that we encountered. On the good news front, our new offices have a dedicated play-test area equipped with identically configured machines. The bad news is that we didn’t make the best of it.
A Turkish mosque shows off the greater detail and improved skills of our art staff.
Many of our play-tests were not organized and focused enough, seriously reducing the amount of new and meaningful feedback obtained. It wasn’t always clear when we were testing for specific bugs and issues, and when we were testing for “fun.” We had a schedule of participants which drew upon the whole company, but schedule conflicts and lax enforcement resulted in the same people playing most of the games. We played too much multiplayer and not enough attention was given to the single-player game. And some people took it much too seriously, trash-talking other players, celebrating wins at the loser’s expense and storming off when they were losing. Play-test problems weren’t confined to Ensemble, though. At Microsoft, it was discovered that a play-tester had turned cheats on, playing to win not to test, in almost every game for over a month, which invalidated all the feedback from that group for the prior two months.
4. Art asset management was nonexistent
The number of individual frames of graphics in AoK is in the tens of thousands, and we didn’t do a good job managing it. The programmers had a source-control system to help coordinate their primary output of code and the designers had the game’s database system, but no such equivalent existed for the game’s art assets. Artists could be working on something with no idea that anyone else was also working on it. There was no way to get a momentary snapshot of who was working on what, other than going around from office to office. Plus, there was no way to tell which files were actually live and being used and which ones were just taking up space. Also missing was a way to go back and find prior versions of art, or to guarantee that new versions wouldn’t be overwritten. As we have grown as a company, this problem has grown even faster. To address this problem in the future, a source-control similar to the art asset management system is being developed for use in all future projects.
5. Problems with third-party APIs and software
Another one of the items from the AoE postmortem returns again. Microsoft’s DirectPlay API still has a number of issues that make it less than perfect. One of its biggest problems is documentation and testing of the lesser-used portions of the API, or rather the lack thereof. Late in the development of AoK, our communications programmer, Paul Bettner, was able to communicate with the DirectPlay developers and an interesting scenario played out several times: Paul would attempt to solve some problem and the developers would indicate that it wouldn’t work because of bugs in DirectPlay that they knew about but that were not documented.
A scene from one of the game scenarios. The updated graphics engine and building scale allowed us to create scenes much more impressive than in AoE.
DirectPlay wasn’t the only problem. We decided to use DirectShow to handle our cinematics. The short version of this story is that it just didn’t work. And then there was the Zone software for Microsoft’s online Gaming Zone. The Zone software was developed too late in the process and had a number of problems, due to a lack of time to test and correct. Unfortunately, this means that direct TCP/IP games are more reliable than those played over the Zone, which is disappointing. This was not all the Zone’s fault because we did not get our requirements to them soon enough.