Producing Audio for Halo

Producing Content

I believe that music is best used in a game is to quicken the emotional state of the player and it works best when used least. If music is constantly playing it tends to become sonic wallpaper and loses its impact when it is needed to truly enhance some dramatic component of game play. In Halo, there are more than 80 minutes of original music, but these minutes are spread over the course of a single player's experience, which could extend from 20 to 70 hours of play. Therefore, much of the time no music is present.

I wrote several pieces of music, some linear and some designed to be dynamic, that express many dramatic or emotional states; combative, spooky, tense, sad, calm, defeated, or victorious. Some musical themes underscore linear cut scenes and became associated with a character, mission, or location. I wrote and produced each piece first in a linear fashion, and then re-mixed, edited, or re-arranged it to fit into the music playback engine we had designed, tailoring it to the specific context.

The music engine is relatively simple in construction. It consists of three basic file types within a single soundtag. The "in", which starts the piece, the "loop" which is the middle section and plays for an indeterminate amount time, and the "out", which is how the piece ends. In addition, there is the "alt_loop", which plays instead of the loop if called by the game, and the "alt_out", which plays if the end is called during the alt_loop. The looping sections are made up of as many looping soundfiles as desired, and each looping soundfile can be of any length and weighted to determine the likelihood of being played. The level designer only needs to insert the commands: start, start_alt, and stop, in order to call any given music soundtag. This made it relatively easy for me to sit with each level designer and "spot" a level in the same way I might sit with a film director and "spot" a movie.

Within each music soundtag, I could also set the preferred type of transition (immediate, cross-fade, or wait until the end of the current loop) between the alt and out soundfiles. We can give real-time volume commands to the currently playing soundtag and also give it an overall duration limit. This kind of soundtag was also used for ambient background sounds.

We also took advantage of multiple tracks in the ambient soundtags. An individual soundtag might be made up of more than one track of looping soundfiles. For example, a beach ambient soundtag could contain looping wave soundfiles and looping wind soundfiles, each with as many variable-length loops as desired. In addition, this soundtag could also contain "detail" soundfiles; sounds that add interest and color but don't loop in themselves. In the beach soundtag, detail sounds could be elements like seabirds, splashes, and gravel movement. The detail soundfiles can be weighted, given random 3D positioning random period, volume, and pitch ranges. The soundfiles for both music and ambient soundtags were 16bit 44.1k ADPCM compressed stereo, and read into RAM in 128k chunks in order to allow us to use longer pieces.

Most of the rest of the sound effects in Halo were made up of non-looping or "impulse" soundtags. These sounds play one permutation at a time, either in their entirety or interrupted by another sound that takes precedence. For example the automatic rifle soundtag is a collection of gunfire soundfile permutations that are called in sync with the weapon's rate of fire. Any single permutation is long enough to include the authentic ring out of an individual gunshot. Each call is interrupted by the subsequent call until the trigger is released and then that permutation is allowed to finish playing. Explosion permutations are not interruptible and thus can overlap. Mono looping soundfiles are the permutations for sustained sound effects like engines, fire, and the like. All of these soundtags can be called in scripts, or attached to objects, animations, particle systems, characters, or locations. The pitch and amplitude can be controlled in real-time by different parameters contained in other tags. For example, the pitch of an engine looping soundtag will vary depending on numbers coming from velocity and RPM's from the engine tag. The engine will sound like it's revving high if it's forward velocity is at top speed or all four wheels come off the ground and the accelerator is still pushed down. We were also able to cross-fade switch between different samples based on real-time events. The most important thing for these types of sound is perfect synchronization: what the players are hearing "feels" right with what they are seeing.

The volume and pitch of impacts and scrapes need to respond to the velocity with which these events occur. Another technique we developed for implementing soundtags was something I call "cascading". This is used when attaching sounds to huge numbers of like events like particle systems, where the number of potential calls could overwhelm the number of voices available. In addition, hundreds of individual glass particle impact sounds occurring at the same time doesn't sound like a big sheet of glass breaking. The cascading system let's me create, for example, three soundtags that are related to each other and will call a subsequent soundtag when a certain number of calls for the initial soundtag have been reached. In the glass-breaking example I will have three soundtags, small_glass, medium_glass, and large_glass. The game's particle system will be sending out a request to play the glass impact sound for each glass particle that impacts. The engine will play 6 small_glass permutations and then on the 7th request cascade up to 3 medium_glass permutations and on the 10th request (all within a short period of time) play one large_glass permutation, after which the cycle starts over again. This cascading system will give the effect of sounds that are synchronized to specific events and at the same time be able to build a cacophony of sound with minimum voice usage.

Dialog
The last big area of sound design for Halo is the dialog. We had quite a complex and involving story to tell with some interesting characters, but telling a linear story dramatically was not our only goal. During combat especially, we wanted the characters fighting with and against the player to seem fully alive and truly participating in the chaos of battle. We developed a matrix to help us script out all the possible situations in which the different characters might speak. This matrix contains 12 main categories, each containing multiple (the largest 28) soundtags, with each soundtag containing multiple permutations. For example, in the category "hurting people" one of 4 main soundtags, "damaged enemy" contains between 15 to 20 permutations depending on the character. For the friendly AI marines, we knew it would be impossible to record a unique actor for each marine that might appear in the game, so we used 6 actors to record 4 PFC's and 2 Sergeants and had the resulting soundtags attached to the corresponding marine models. We were also careful to keep track of those who lived and those who died in order to maintain the balance of the voice actors. If you start an encounter with 8 marines, two of the characters would be doubled up, let's say two of Pvt. Bisenti and two of Pvt. Mendoza. If at the end of the encounter only 4 marines remain, even if Bisenti or Mendoza never got killed, we would swap one of the duplicates for a missing Jenkins or Johnson, in order to keep the characters balanced.

All the actors were SAG and AFTRA talent, and during the casting auditions, their ability to improvise was tested along with their ability to bring the proper emotion to the script. There are more than six thousand lines of dialog in the game and one of the elements I still find enjoyable is listening to new and unique combinations of real-time interactions between the marine and alien AI's. I literally never know what they're going to say next.