At the Montreal Games Summit, Realtime Worlds producer Phil Wilson gave an in-depth postmortem of Crackdown, discussing obstacles encountered and mistakes made during the development. Wilson noted that while he did a postmortem for Game Developer, he wanted to add some other aspects of the game from a different direction, looking at things from a chronological standpoint.
Wilson said the team wanted to go “beyond GTA” – offering free-form play, the feeling of empowerment, responsive and intuitive controls, and “capturing the moment." They wanted to allow the player to create moments such as jumping out of their speeding car, then spinning around and shooting the gas tank, so they aimed to offer offer lots of "toys" to play with, and also provide “shared experiences,” whereby players could talk about the things they did in the game, not that they watched in a cut scene.
Wilson highlighted the goal of creating a game that is deep, but in which the entirety of the world is completely available from the beginning. They still wanted to offer progression through play, and allow fluidity of action (to allow showboating) and reward experimentation.
They didn’t consider story to be a pillar of the product, as Wilson explains, “We’re not about story, and initially we were proud of that, but now I’m not sure that was the right decision. We should have found a way to motivate the player through the story, too.”
He presented an image of Crackdown's cover, and said, “Nobody on the development team liked the cover. I’m sure I’m not alone among developers in feeling that about our game’s cover, though.”
“There was a four month proof of concept in 2002, and Microsoft wanted the team to shoot for finishing Spring ’05. No set deadline, but roughly that," Wilson recalled. So they set out a 12 month pre-production stage, with 9 coders and a handful of asset creators, with a planned ramp-up to 35 for production.
The Pre-Production Environment
“Obviously in pre-production you want to find out what you want to know but don’t yet," Wilson said. "What you don’t want is a publisher who wants to see a head start on production."
He continued, "So obviously after a year, Microsoft asked us ‘where’s the vertical slice?’ I hate that expression, but they wanted to see a demo with everything integrated and to demonstrate artificial progress."
It wasn’t until February of 2004 that they were able to provide that first playable demo -- however, doing so saved the project and allowed them to clarify the design, unifying the team. But in the process, they found a massive amount of game design challenges with the sandbox play.
Early takes at skill development, for example, allowed the players simply to drive around to raise their driving skill, and to just jump up and down on the spot to raise their jumping skill. It was ripe for abuse, and that’s where “skills for kills came from,” Wilson explained.
The open structure also allowed players to run straight up to bosses that were far too hard for them and become frustrated at their inability to beat them, despite not being developed enough for the battle. That’s where the “probabilities of survival” came from – although, as Wilson noted, “many gamers would initially take a probability of survival of about 0.01% as a challenge. They’d be like 'cool, bring it on.' They soon learned, though.”
He also discussed the effect the open structure of missions had on the experience. Although they had designed each boss encounter to be approached from multiple angles, most players found the one way they liked to take on a boss and used that way to finish off all of the bosses (as every method was effective for every boss.) “By trying to put in variation we actually created a way for players to limit variation,” Wilson described.
Meeting The Xbox 360
When Microsoft came to them with the Xbox 360 hardware, they decided it was best to go to the new platform, and Microsoft said, “another year should do it, get it to us for Spring 2006.” But a chicken and egg situation came up, as they knew they needed more people to make the required progress -- but of course, without showing good progress, they would never get more people.
By September 2004 they had begun to achieve good, yet incremental progress – at this point they realized that they needed to create a “scope document,” which Wilson would refer to as a Product Backlog – since working on Crackdown, he has been completely sold on SCRUM/Agile project development.
The team initially hated that, as they loved making incremental developments, and they don’t want anyone to stop their work. “But at least it was a top down approach,” Wilson conceded.
In creating the scope document, they captured better information on implementation, requirements, how well-defined and understood the features were, the work required to complete them, their priority and their risk level. Wilson explained, “We should have done that at the start of pre-production, not the end.”
In fact, he continued, in doing so, they realized that they still weren’t at the end of pre-production. “We had been way too ambitious, and serious cuts were required, and we would still have major bottlenecks with tens of new coders required,” recalled Wilson.
What Happened Next
”There was no pragmatism from stakeholders. They actually kind of went mad," Said Wilson. "They actually used our scope document to add requirements! And they had a very set definition of minimum content. They knew that if they didn’t get their boats they wouldn’t have a game -- we didn’t ship boats.”
However, continued Wilson, thanks to the changes, by November 2004, they had managed to create the whole city and make it navigable in co-operative mode. "It may not look like much” Wilson said, demonstrating images of the low polygon, textureless city, “but it’s very nostalgic for me.”
Unfortunately, in January 2005 the team made what Wilson calls a “gross mistake” by porting to Renderware 4, which was at that point in alpha. “I know that Microsoft wanted us to do it and be right at the forefront of the technology, but it made no sense, as the alpha was way below the level we needed," Wilson said.
He continued, "We never should have done it. I notice the Gears of Wars guys didn’t, and good on them. Why did we choose Renderware 4? Because we were idiots. There’s a common misconception that when EA took over Criterion that’s when things went terrible for Renderware, but to be honest, we were having loads of problems even before that.”
Wilson talked extensively about how many difficulties they had with Renderware. “Our pipeline sucked," he stated. "It could have taken upwards of an hour to test assets. Which just means that people didn’t test assets, leading to some quality control issues. But we couldn’t justify starting again so we limped on.”
He went on to talk about taking the title to the X05 Microsoft press event in Amsterdam -- “and again Mark Rein was right. You don’t show it 'till your ready. And we weren’t ready.”
Thankfully, enough progress had been made for Microsoft to send help, says Wilson. “We managed to get some excellent contractors over the summer, who were really talented guys, but if you ever have your publishers give you extra help, try and make sure they don’t retain management over them. Having part of your team managed by people 6000 miles away is not pleasant.”
As a result, by May of 2006 they made miraculous progress before E3 creating something that people could see was something special, but “we shouldn’t have been on the show floor. It wasn’t ready for just anyone to pick up the controller.”
Too Much Purple
Phil took an aside to discuss the stylized visuals of Crackdown: “We had defined the style from the outset: a deep rich color palette, crisp real-time shadows, exaggerated assets, bold outlines on all geometry. The marketing PR guys hated our visual style – not aesthetically, but because it was another aspect of the game that they felt they had to explain.”
In fact, it was a major problem, Wilson continued, as the “visual bar” was something that Microsoft didn’t feel they were matching. “The visual bar? I like to consider it a crowbar. They’d come back from seeing something like Gears of War, and all they would say was “phoar, Gears!"
"They saw our game in an equation," Wilson said: GTA + Spiderman x purple = Crackdown. "We knew we couldn’t have any more purple jokes and we hired some really talented guys to finish off our graphics," he added.
Microsoft did a really deep consumer survey and it paid off -- graphics were the number one thing gamers enjoyed from Crackdown, Wilson pointed out.
By June 2006 Wilson admitted it was “enough’s enough” -- but they still had some issues, as he describes: "Microsoft was pushing the crowd system, the long distance traffic, and this was some really difficult, high-risk stuff. It worked out in the end but it was an incredible risk."
"We were the worst though," he admitted. "We still wanted water reflections, clutter, final crimes, complex AI, flaming characters and special vehicle action."
As to the clutter -- basically floating trash –- Wilson said, "We coded that late, and didn’t get the renderer right, so when it goes into the distance it turns black instead of fogging out. We ended up shipping with little black dots visible in every vista. Unfortunately, to patch that would require the entire environment to be rebuilt, and no one wants to download a gig of environment in a patch.”
Wilson added, “We never adequately scheduled anyway.” He noted that when it came to play testing, they were very lucky, as they had usability play testing at a dedicated Microsoft facility, with Live user test streams, and data harvesting and data processing provided.
Unfortunately, in October 2006, the feedback was not good. For fun factor, it'd fallen into the bottom 30 percent of all Microsoft titles in testing, and it ranked in the bottom 50 percent for replay interest. However, by November 2006, things had turned around -- it received the top score for fun, and its interest level was second only to Halo 2 -- "Which I'll take," quipped Wilson.
A Drastic Turnaround
So what made such a huge interest for the user experience? A lot more hand holding in the early stages. And Wilson did note that although Microsoft wanted to take control of all of the QA for this game, "When the developer is so far away from the publisher, you really need your own team. So we hired a few guys but we should have spent more, it was a fool’s economy.”
And at times, they did have poor communication with Microsoft, who had a “we will test this – you check the bugs” attitude. "That type of relationship is not healthy," Wilson said.
Wilson also said Microsoft was using an archaic defect tracking system – "when it went down, no one even knew where it was in Redmond to reboot it.”
By the end, they found they had 37,000 bugs in total. And as a result: 3,000 low severity bugs were waived-- "Which is horrendous. I see them when I play the game, but people were very forgiving," said Wilson.
For release, they were bundled with the Halo 3 beta – "We expected it, but we would take any help we could get," said Wilson. "I know how hard it is to get a new IP off the shelf. After all, when GTA hit it didn’t sell very much at all, and it was very frightening. It wasn’t really until the violence furor hit that it started to sell.”
He discussed the importance of the demo. They worried that they were “giving away the farm,” but they wanted to offer the condensed experience. But the feedback said, "the Crackdown demo is like crack!" And it turned out to be the most downloaded demo. Moreover, with a purchase ratio of 3:1, it was "the highest that Microsoft that had ever seen, which was great,” according to Wilson.
“The title launched in February this year to critical acclaim -- what you really want to hear when you’ve worked so hard and crunched for so long -- and we were really pleased with that, and the sales of 1.5 million," said Wilson.
Wilson added, "I wanted to clarify that there were some tense times, but Microsoft are a unique publisher in that they funded us and worked so hard with us to complete the project. They’re a really good publisher to work with.”
In summary, Wilson felt they produced a high quality experience, but a low quality product, that a it was surprise hit, but it was not able to get a sequel – from RTW, anyway. “That broke my heart, and I don’t want to talk about it any longer in case I tear up,” he added.
Concluded Wilson, "We’re glad we shipped it, though, and we still play it, and it’s in Edge Magazine’s top 100 -- number 100, but, still, we’re in there. That’s page one! And we won two Develop awards, and two BAFTAs. I can’t tell you what that means to a studio starting out.”