If you're reading this paper, it's probably because you work on the audio side, the design side, or perhaps you would like to work on some side of the game industry. Maybe you're just a fan of Halo. Whatever your reason is, I'd like there to be something interesting for all concerned, so I'm going to throw in some history as well as technical information.
Halo evolved over a number of years, and the audio design for it evolved as well. Some of this evolution had to do with a desire to reach for higher production values, but much of it had to do with how the game itself changed based on the platform and hardware that became available over the course of Halo's development.
Bit of History
In 1998 we all still lived in Chicago, and I worked as an independent contractor on Bungie's two Myth titles. As we prepared to finish Myth II, Jason Jones, the lead programmer and creator of Halo, showed me some early work he and a few others at Bungie were doing. I saw some vast outdoor/indoor environments, with cool inverse kinematic animations, sci-fi weapons, a cyborg, and a jeep. There were lots of drawings of aliens, other vehicles and even some storyboards for a movie that might be shown at E3. The ability to seamlessly transition from outdoor to indoor was there, as well as a jeep that responded to physics in a way that was realistic and fun at the same time. I began working on vehicle and weapon sounds that could be triggered in the technical demo. At that time we were supporting the current Creative and Aureal sound cards and so we had some basic 3D-surround effects working with two speakers or stereo headphones.
At E3 in 1999, Bungie had a closed-door press event that highlighted the tech demo of Halo. The press people who saw the demo understood that if they wanted to see it, they couldn't talk about it yet. The interesting thing about that strategy is that it probably created more buzz than if they were allowed to talk about it.
In July of 1999, Bungie's cinematics director, Joseph Staten, came to me with a request to make a soundtrack for a live demo of the game during Steve Jobs keynote address in New York for Macworld. The Mac version of the Halo engine was running pretty well on Open GL. By this point, the game was supporting a robust scripting language and had more vehicles, structures, and creatures, as well as aliens and humans. The plan was for a two to three minute scripted demo of the game that would run in real-time on the Mac. The only problem was that we had no sound code written for the Mac version, and so it would be played in complete silence. We decided to create a music track that would run for the duration of the demo, loosely synced by simply hitting the "play" button on a CD player at the same time the "enter" key was hit on the Mac. We talked through the story of the script and came up with some general timings (which of course were very loose) and I went back to my studio to write and produce a piece of music that would score the drama of the scene as well as establish a mood and feel for this ancient, mysterious ring artifact found by humans 500 years hence in some unexplored corner of the galaxy.
I felt that I could evoke an ancient and mysterious feeling by starting with some Gregorian monk style chanting over a bed of strange ambient sounds, and then give the action sequences that followed an epic and important feel by getting orchestral strings from the Chicago Symphony to play over a somewhat rock&roll rhythm section. I added an improvised Qawwali chant voice over the top to help reinforce the "alien" nature of the environment. Whether these decisions were the right ones or not doesn't matter. I had two days to write and produce this piece and there simply was no time to ponder or experiment, which is sometimes a good thing. Since this was also a venue that would feature a big screen, a large auditorium, and a gigantic stereo sound system, I wanted to not only capture the mood but also hook the audience. Anything that sounded like "game music" was going to be a disappointment. Plus the track needed to be interesting enough in it's own right so that the audience wouldn't notice that they weren't hearing any sound effects. It seemed to work out pretty well.
For E3 2000, Bungie decided to show a hands-on demo of Halo on the PC. At the same time we wanted to recreate the theatrical atmosphere of the Macworld demo. We needed a longer and more involved story that would present the kind of game-play we were hoping for, and a cinematic feel for the audio. There would be a live demonstration of the game on a PC using the still somewhat primitive sound engine triggering one-off sounds of weapons and vehicles. This would be followed by scripted game sequences that had been captured, edited and played back off a DVD. The script was written, the storyboard created and then we recorded voice actors. The game characters were animated to the edited voice files using the engine. We received captured video of segments of the story that was now steadily growing from its original projected length of six or seven minutes to almost ten minutes. We set up a surround sound Pro-Tools session, and kept adding to it as the video got longer. The music and sound design recorded for the DVD was mixed in 5.1, with the goal of giving the E3 audience as close to a movie theater experience as possible. A small self-contained movie theater with surround speakers was built for the show. We knew at the time that the game could play the way the movie looked, but I knew that we had set the bar pretty high in terms of the audio. There was no such thing as real-time 5.1 surround sound for computer games.
We knew since GDC 2000, that the Xbox would be a pretty cool platform for a game like Halo. However, it came as somewhat of a surprise to find out after the first showing of the DVD at E3 that Microsoft was making an offer to buy Bungie Software, move us all to Redmond and have us build Halo as an Xbox release title. Since very little of the actual game had been finalized yet, it seemed like a great opportunity, and after many discussions, the purchase and move became reality. Just ten days prior to the Microsoft offer, I had joined Bungie as a full-time employee. My decision to move to Redmond with the rest of the Halo team was made easier in part because of my personal desire to see Halo's audio meet or exceed the expectations we had set in the marketing demos produced thus far. With my own understanding that the sound capabilities of the Xbox might possibly include more channels of sound than any sound card currently available, and some sort of real-time surround implementation, my choice was clear. In addition, designing game audio for a single platform with a known set of specifications was something that I had never done before, and being right next to the audio hardware and software folks might also be an advantage. Plus my family was glad that some of our best friends had moved out to Seattle the previous year.
All previous work on Halo was basically broomed once we got the specs on the Xbox and we started over with new code and content development in the fall of 2000. Work on the audio didn't start until February of 2001, since we first had to finish audio production for another Bungie title, Oni, and also build new audio and video studios within the confines of Bungie Studios at Microsoft.
One of our first tasks was to define the goals as well as terminology that we all could agree upon. This lead to a document that we often referred to during the development process. Our main goal for the audio in Halo was that it would set the mood, give the player information about what is happening (especially things that can't be seen) and make the world seem more alive and real. Music should provide a dramatic component to game play, like combat and exploration, in addition to underscoring story and cinematic sequences. Dialog should unfold the story, provide characterization and draw the player deeper into the experience. Sound makes it real, music makes you feel.
The Halo tech demos and marketing movies that we had made up to this point had all been examples of linear audio production; pre-mixed, pre-determined duration, and traditional audio post-production techniques. It was time to figure out how to get the same results using the game engine and the Xbox to make dynamic music, sound effects and dialog.
Our understanding is that any sound that responds to a game-play condition or event is considered to be dynamic. Dynamic sounds can be affected by the structures in the environment (like reverb or occlusion), they are spatially located, and they can respond to physics (such as Doppler shift). For example, the sound of an engine will change with the rpm's or gear shifting, and the volume of the sound of a shell casing bouncing on a surface will change with both it's velocity as well as it's change in distance from the player.
Dynamic dialog occurs when different recorded lines are triggered based on changing conditions. Working closely with our AI programmer, Chris Butcher, we came up with a system that would allow our friendly marine AI's to say something in response to what they see, hear, or feel in several differing game states.
Dynamic music can vary in length or change in volume and intensity based on conditions that occur in the game. I never want the player to be aware that they have the ability to change the way music is being played. That would call attention to something that should be more subliminal, and remain on an emotional rather than cerebral level. For example, if music goes up the scale when ascending stairs and down when descending, the player might stop playing Halo and start playing the "making the music go up and down" game.
Every piece of raw audio data is called a "soundfile" and the set of instructions that organizes and determines how the soundfile is to be played is called a "soundtag". The Halo audio engine only recognizes the soundtag. The most important feature of a soundtag is that it contains enough permutations and the proper randomization so that players don't feel like they're hearing the same thing repeated over and over. Even the greatest and most satisfying sound, dialog or music will be diminished with too much repetition. It's also important to have the ability to randomize the interval of any repetition. It might be difficult to get the sound of one crow caw to be vastly different from another, but the biggest tip off to the listener that something is artificial is when the crow always caws just after the leaf rustle and before the frog croak every thirty seconds or so. The exception to that rule are specific game play sounds that need to give the player immediate and unequivocal information, such as a low health alarm.