3. Input from outside.

The first external BioShock focus test was meant to be a sanity check: to get a better sense of what was working well but needed polish and what wasn't working at all.

At this point we had already done one small round of internal focus testing with friends of friends, which had turned out mostly positive feedback. So, just after the first beta, the entire design team plus a contingent of 2K producers headed off to see how a group that knew nothing about our company or BioShock would react to the first level.

It was brutal.

The first level, they said, was overly dense, confusing, and not particularly engaging. Players would acquire new powers but not know how to use them, so they stuck to using more traditional weapons and became frustrated. They didn't interact with the Big Daddies, and they didn't understand (or care) how to modify their characters. They were so overwhelmed by dialogue and backstory that they missed key information. A few of the players did start to see the possible depth of the game, but even they were frustrated by the difficulty of actually using the systems we had created.

Based on this humbling feedback, we came to the realization that our own instincts were not serving us well. We were making a game that wasn't taking the initial user experience into account, and we weren't thinking enough about how to make it accessible to a wide variety of players.

After the focus test, we went back to the drawing board for the entire learning sequence of the game. We scrapped the gameplay in the first two levels entirely and re-architected them to be a much slower paced experience that walked the player through the more complicated gameplay verbs, such as "one-two punch"-combining weapons and plasmids. We changed the medical pavilion from having sandbox-style gameplay to using a series of locks and keys that were set up to ensure that the player knew how to use at least a few key plasmids. And we made a development rule that future changes would be data-driven, not based solely on our own instincts.

After the first round of changes, we had two rounds of internal 2K play testing to gather more data about the user experience, releasing builds of the game to several 2K studios and soliciting feedback about how far people got and which weapons or systems they enjoyed. We received feedback from 2K game analysts, Microsoft, and a few other advisors.

When we brought the demo back to focus testing, which was barely a month before we were (then) scheduled to complete the game, the experience was very different. Although players still got stuck and frustrated at various spots, they understood the game systems and saw the potential inherent in them. While we still had work to do to make the game more accessible, at least now the problems were much more easily solvable.

4. Small empowered teams.

While developing our first internal demo, we realized just days before completing it that it was on the wrong track. By that point it was too late to take on all the problems in the demo, but we decided to try to improve the core interactions. We used a small, focused strike team approach to target and solve AI problems, choosing one problem at a time, analyzing and tackling it, then moving on. Although this approach wasn't enough to salvage the original demo, it was recognized in our internal postmortem of the demo as an effective process that we should do more often.

One of the most visible successes of the strike team system is the tuning of the weapons of the game. All the weapons had been in and working for several months, but as the game got closer to content lock, they still weren't feeling as good as they should.

To tune each weapon, a team consisting of one designer, an animator, a modeler, a programmer, the effects specialist, and an audio designer held a kickoff meeting where they analyzed and brainstormed about each aspect of a single weapon. They came up with a task list for each team member, went off to work for a day or two on their tasks, then came reviewed all the results. When they were satisfied, they moved on to the next weapon.

Over the course of development, we created multidisciplinary strike teams to work on a wide variety of problems, including AI, animation, visual effects, and cinematics. The results of those teams were universally better than the previous non-iterative process.

The 2K Boston BioShock team.

5. Talented people, flexible staffing.

BioShock was initially scoped to be developed in about two years with a small team of 30 people-25 in Boston focused on gameplay and five in Australia working on the core engine. As the team completed successful milestones and demos, and made strong cases for more development resources it became clear that we needed to tap into the Australian office.

Initially, Australia was intended to supply a small core technology team that worked on the renderer, engine, and core tools and processes for console development. The Australians had a tremendous impact on development because by taking care of the core engine and pipeline tasks, the Boston programming team was free to focus on gameplay systems and production.

One of the fastest and easiest ways to staff up any newly-opened position on the BioShock project was to pull from the Australian team. By the time BioShock went gold, almost everyone in the Australian office had worked on the game in one way or another.

The huge advantage to using the Australian team resources was that they already knew the engine and the game, and had easy access to the core technology team. They came up to speed incredibly quickly, and could be productive almost immediately upon getting project tasks. And although the time difference made communication a challenge, it also meant that critical bugs could be worked on literally day and night.