I Heard You Calling
Synaesthesia has become a buzz word for
some high-minded abstractionists in the games industry, but 3D games can co-opt
the phenomenon for their own immersive purposes. Video games are particularly
adept at connecting sound and visual motion into a way that can produce a real
physical sensation in the player.
"We did a lot of real-time
post processing with DSP filtering on sounds to convey distances and room-space
perception. Each space the player is in has its own reverb type, to keep audiovisual context," said Guerrilla's
Mario Lavin, sound director on Killzone
2.
"We also wanted to make the
player feel the weight of their equipment, so all the character movements
trigger randomized equipment rattles, belt clicks, clothing rustles, etc."
Sound can play a major part in building
atmosphere and suggesting mood to the environment and story. In Halo 3: ODST, for instance, your player
will wheeze in pain after just a couple shots, emphasizing the new
vulnerability of the troopers in comparison to Master Chief.
In Darkside
Chronicles, sound effects are used to insinuate short burst of narrative
without forcing players to watch cutscenes or read text. In the opening chapter,
set during the events from Resident Evil
2, sound effects are used to establish a mood of imminent danger and imply
eerily recent disappearances.
"The police siren follows, and the players become aware
that they are under desperate circumstances," said Noguchi. "A song
being played at a nearby shop confirms that there was normal human life
happening just moments before the zombie invasion occurred."
To conventional thinking, this is wasted
effort and geometry; players aren't doing anything here, they're just moving
through space. In reality, players create associations with, and assumptions
about, the gameworld at all times. Even if you're not killing zombies or
collecting ammo, you're still taking part in the experience, picking out
audiovisual cues at every turn.
Sometimes a combination of visual and
sound cues can be used to completely replace the HUD, letting players absorb
all the critical information they need from the game world itself. "We spent a lot of
time working on Faith's breathing, the sound of her clothing and footsteps, for
instance. That can tell you so much about what speed you are travelling at,
over what surface, or if your wall run is about to end," said O'Brien.
"In most games,
footsteps are a pretty simple thing to add, but running and moving was so
integral to Mirror's Edge that we had
to create a huge library of footsteps and a system to manage them. We had them
for different speeds, different surfaces, different landings; the list goes on
and on. The breathing system was also key. During playtests we actually saw
players starting to sync their breathing with Faith!"

Mirror's Edge
With different
sounds competing for audio bandwidth, proper layering can be vital. "Our
audio team spends a huge amount of time and iteration to make sure that not
only are the individual sounds unique and reinforce the game atmosphere, but
that they complement one another, so the player does not lose the individual
sounds in a cacophony of sources," Retro's Kelbaugh told me.
In the movie
industry, there's a saying that if you're having a problem with the third act
of your screenplay, the real problem is in the first act. The analog for game
development is in planning. "It might seem kind of obvious that the
audio always comes last when you're building a game, and you tend to not have
the time," said Visceral's Bagwell.
"It was really important to us to
lock the game early enough to give the audio guys enough time to make the game
really sing and make it sound amazing. If the level designers are still moving
things around, or they're still moving the cameras around, you can't really set
that up."
Having big budgets and long development
cycles can definitely make it easier to focus on atmosphere and immersive
flourishes, but immersion can still be accomplished with some rigorous upfront
planning that identifies what qualities are most important to your game. Would
you rather spend an extra two months prototyping or add an extra two levels to
the end-game gauntlet?