Multiplayer was also totally broken at the time, as we continued to rework the architecture. Because we were so close to the finish line, Sierra and Saber together made a tough decision to focus on the Xbox 360 and PC SKUs and make sure they didn't slip, and give PS3 version a lower priority.
It turned out to be a great decision because it provided the team with focus, which is vital on a project where the stakes are so high. However, we were not happy at all with the prospects of missing 2007 on PS3.
At an internal meeting with Andrey Grigoriev, who leads our engineering team, we brainstormed ideas on how to keep the PS3 SKU on track without putting Xbox 360 and PC at risk. We understood that we couldn't put a lot of senior resources on PS3 full-time, but it was vitally important to keep the PS3 build "alive" -- the code needed to be compiling and running, stability issues had to be in check, tons of small kinks with the build processes needed to be identified and worked out, and TRCs needed to be dealt with.
So we designated a small team of people (actually, two programmers and one QA specialist) to work full time on this SKU. When they stumbled upon issues they could not resolve themselves, other members of the team were asked for help with such specific areas as audio, multiplayer or UI.
Miraculously, this approach worked out perfectly. By the time Xbox 360 version was close to completion in early September, and we had more resources becoming available, we still had some pretty major technical problems unaddressed on the PS3 SKU. We were severely out of memory (both main and RSX), loading times were unacceptable, performance was sub-par, and a large number of TRCs were still open.
However, we had a pretty stable build which we could make and test on a daily basis just like on two other SKUs. This was the key. Our entire technical team was now ready to address all of these issues. We were at a complete asset and feature freeze. And we still had a full month before we had to make our final submission to Sony.
In addition, many of the technical hurdles on PS3 are similar to those on Xbox 360 -- and we had a lot of people who just handled that SKU, so they had the experience and the tools. Things were working out. For example, the very complex task of optimizing the loading times was handled by Roman Lebedev in under a week, reducing loading times from whopping four minutes to a TRC-compliant 25 seconds.
In addition to this strategy which was put in place and successfully executed, a few other factors played a critical role. Mark James, Sierra's Tech Director assigned to our project, helped us to forge direct relationships with the first party hardware vendors and our middleware providers.
Whenever we hit a wall (which oftentimes was happening at 3 AM Russia time -- which is an afternoon on the West Coast of the U.S.) we could call the technical experts at Sony or Microsoft directly and ask specific questions and sometimes creatively negotiate a solution which would satisfy the technical requirements and was feasible within our timeframe. This allowed us to keep the ball rolling the following day and not waste any time. Using email would certainly have slowed things down tremendously.
Finally, developing a robust set of tools paid off in a big way, especially during the final sprint towards GMC on the PS3. A great example of one of these tools is our Performance Analyzer. This relatively simple tool allows us to put performance brackets in the code and outputs performance counters into a file, which can be viewed and analyzed later on.
Entire play-throughs of the levels were recorded and dissected later on by performance experts, identifying code bottlenecks, threading issues or art problems. (Frankly, if the format of this article would allow for an extra "what went right" section I would put tools there, because having a robust toolset is second only to having a great team of people. Now that TimeShift is finished and is on the shelves, we are putting nearly our entire engineering team on development of the tools for our future projects).