Surround sound, as a concept and a format, has been around for nearly as long as sound itself. Indeed, there seems to have been an early desire to immerse the viewer within the picture. After almost a hundred years, the surround formats have settled down into recognizable consumer formats and brands. Game audio comes to the scene ready to inherit all the technologies and expectations of motion picture surround. In the second part of our look at Hollywood integration into game development, we examine how theatrical and consumer surround sound is used, and how it borrows from, and looks beyond, its filmic antecedents.

Games have adopted a slew of branding from cinema in terms of surround sound: Dolby, THX, DTS. This makes complete sense as it represents a standard to which video games content can adhere, provides consumers with recognizable logos in terms of quality cinematic sound and provides compatibility and support from systems already entrenched in the home cinema market that games consoles can make immediate use of. There will probably never be the development of a grassroots surround format exclusively for games, although we may see some of the more cutting-edge developments for cinema surround getting more attention from gamers than from a film audience.

Broadly speaking, video games are about physical immersion in a universe of action and reaction. Films, however, are about emotional immersion within a narrative (1). New technologies that allow further realism and higher resolution in terms of physical immersion will be better received by video gamers than cinema-goers. On the whole, surround sound does not facilitate a huge leap in the immersion of an audience into an emotional narrative; if one looks at early sound film, such as Fritz Lang's M, the use of one mono sound channel creates a very complex and layered world of narrative associations and emotional cues which are as compelling as any modern day surround sound picture. However, in films such as Apocalypse Now, much of the subliminal, poetic narration would be impossible on a single channel.

First Wave Consumer Integration into Surround

In the last generation of consoles, a significant proportion of the game playing demographic did not own surround sound systems, even though it was still considered as an area rich for exploitation on the part of content creators. An initial wave of purchasing occurred when the new consoles came onto the market, predominantly by a middle class who had considerable disposable income. This initial wave likely purchased a home movie surround set-up into which they could integrate their consoles and DVD players. This initial wave of buyers potentially deemed whether a particular console, and console titles, gained commercial and indeed critical success in order to enable the machine to be marketed to the second, third and forth wave of consumers who, in all likelihood, did not own surround sound systems, but conventional, often mono, TV sets.

The success of the DVD format, for both games and movies, can be viewed as an encouraging and pre-emptive mass consumer move into the surround sound market, as certainly prices of DVD systems and the surround speakers and decoders themselves, are something that are now being offered at affordable prices. It can therefore be safely assumed that the initial wave of surround ready games bolstered a critically successful environment which has supported demand for more affordable versions of this technology. This first wave of games (Medal of Honor, Halo) has also been able to set the critical and aesthetic benchmarks. This is how the integration model worked for the first generation of surround sound consoles. With next gen consoles, surround is a given; it is expected. Anyone with a home entertainment system will expect their games to sound as cinematic as the movies they watch, if not more so.

Immersion and Emotion

In terms of aesthetics, looking towards the Hollywood film industry for precedents and parallels can only paint part of the picture. As has already been discussed, games represent physical immersion in a universe of action and reaction. Given these terms, there are certainly closer and more recent analogies for surround sound that bear closer artistic sinews to game-surround. These parallels are the ‘IMAX' and ‘ride film' presentations, which take immersion through surround sound as fundamental to the overall experience they are selling.

IMAX

	Surround sound is integral to the immersive effect of an IMAX theater.

The theatrical desire for giant-scale cinema started in the early 1950s in response to the threat of television. Cinerama, unveiled in 1952, used three projectors and three screens. In 1953 came the ‘wide screen' formats CinemaScope and VistaVision, giving birth to a whole slew of formats such as Technirama, Panavision, Superscope and many more besides. IMAX grew from a team of Canadian film-makers creating large format films for Montreal 's Expo 1967. By the time of Expo 1970, in Osaka , the IMAX formula had been perfected. Since it's inception in the early 1970s the 15/70 large format film medium has been at the forefront of a gigantic immersive entertainment experience. The sound is mixed, usually in situation, in gigantic cine-sphere auditoriums.

Tim Archer, has created sound design for, as well as mixed large-format film for over 20 years, creative director of Master's Digital, based in Victoria, Canada. (2)

“Technically speaking, IMAX theatres have 6 full range speakers. There are 3 across the front screen (LCR), 2 point-source surround speakers and 1 speaker on top of the screen for height. A typical theatrical 5.1 set-up will use smaller speakers in an array as surrounds. Bass management also differs from conventional cinema surround. The subwoofer information in a 5.1 system is a physical track containing all the low-end, while in an IMAX theatre, the subwoofer information is derived from the 6 main speakers. So all low-end is filtered from the 6 main speakers and routed to several bass cabinets in front of the theatre.”

The complete immersion of the senses is the primary goal for these large format experiences. The space defined by the screen is intended to reach into and beyond our peripheral vision, something which the normal movie screen doesn't do due to its clearly composed rectangle. Many IMAX films and cinematographers will compose their shots with this periphery in mind, allowing the viewer's gaze to explore this vast space, rather than absorbing the image in its entirety. Film sound has always enabled the experience of ‘periphery' events, this is evident even in the films as early as The Jazz Singer (1927), in which we can hear the audience's reactions to Al Jolsen's performance, off screen, therefore creating the ‘imagined space' beyond the rectangular screen. Even in pre-sound film, the imagined space outside the frame was referenced through use of on-screen mirrors, reflecting a space behind the camera's position.

The philosophy of ‘large format' sound, in particular surround, is also carried through right from location recording, to the final edit and mix.

Tim Archer:

“I find the Large Format film genre more challenging and interesting to work with than a standard ‘dialogue driven' feature film. On a feature film, the location recordist is mostly concerned with recording clean dialogue, whereas my job on a Large Format film is to capture multi-channel atmospheres. These recordings are the initial fundamental layer of the sound design. I also try to capture as many indigenous sound effects as possible, to give the track a real ‘sense of being there'. This same philosophy carries through the sound edit and final mix. I will edit and premix in a large mix room then, for final mix, I gather up my gear and set up and mix right in the IMAX theatre. It makes a huge difference watching the soundtrack against the enormous 6 story screen. Obviously, each museum, theme park and science centre has different speaker configurations and acoustics and can only be mixed at that exact location.”

Surround sound is so much an integral part of the IMAX experience, utilizing all the speakers equally, not to mention the height speaker, in order to create the feeling of depth, height, immersion and spectacle.

Tim Archer:

“The IMAX picture is so large and crystal clear that the soundtrack has to equal this. The smallest detail can make the difference between sounding real or artificial. You are also attempting to immerse the audience in a realistic environment with layers of atmosphere tracks and well placed specifics, by using all 6 speakers in conjunction with each other. The surrounds and the height speaker are just as important as the front 3. This approach is quite similar to creating “soundscapes” for location-based venues like museums and science center exhibits. You are looking to simulate reality, by strategically hiding speakers behind fake walls and props, to add a reality to static images.”

The spaces around the edge of the IMAX screen are not visually composed, as would first be assumed, with the intention of being a vague, empty ‘periphery' space. The dimensions of the IMAX screen actually allow an audience to ‘look around' the screen, breaking much of the directorial need for an audience's attention to be focused on one single image after another in a standard narrative sense. In this respect the films do bear an element of visual free interaction, but the narrative is better compared to a ‘ride' whereby there are any number of things to see at any one time, and each viewer's experience of the film will be markedly different from the other's based on what they observe at a particular time.

In a film format as big as IMAX a strange reversal takes place in that the sound takes the directorial seat, focusing people's attention on certain parts of the screen, or via an omni-present narrator's voice. In conventional film, sound is used to provide the ‘periphery' and the immersion. In IMAX the periphery is visual, the focus is lost and the sound must lead. This occurs through the spatialization of sound via the surround format; if we hear something unusual over to the left, our attention will be directed to the left hand side of the vast screen area, and so on. Speakers behind the screen allow this to be ‘spotted' very accurately. The rear surround speakers still perform the old function of creating a ‘periphery' as there is no screen space immediately behind the viewer.

Games certainly mirror this ‘directorial' use of sound in that the sounds of objects or enemies can often only be located by the use of surround sound. In the confusion of a huge onscreen battle, one of the ways of navigating through the chaos is to utilize the surround audio field in order to reveal where the next objective is. Much of the use of this is currently ambient in nature in gaming, due to smaller screens and also the fact that positionality cannot be used to its full potential while there are still gamers playing games on stereo or monaural sound systems.

The other main goal of an IMAX presentation, of course, is that of ‘spectacle', taking to extreme the sensations experienced by the first audiences of Lumière's Train Arriving at a Station in the 1890s who ran screaming from the auditorium believing the train to be coming right at them. Essentially at the core of this experience is a need to blur the lines of created and imagined space.

The IMAX film has to compete, like the mainstream theatrical cinemas, with the phenomenon of consumer DVD, which now account for a huge slice of the profits of any movie. IMAX has begun several strategies to broaden its appeal. For over ten years the company has been actively encouraging major Hollywood studios to adopt the IMAX format, with limited success. Now, though, with DRM technology, it is possible to re-master conventional films for the IMAX format. It is costly - every frame has to be enlarged from 35mm prints and manually cleaned - but this has proved effective in reviving the format. One of the biggest hits in IMAX cinemas was The Matrix Reloaded. In addition to this, IMAX has developed a smaller, cheaper large-format system for conventional cinemas.

As video games make the shift ever closer towards these larger formats, and as our homes embrace larger screens and more powerful sound systems in order to aid immersion, the audio will be able to fully switch from ‘periphery' descriptor to the ‘director' of our attention.

Ride film: Shock and Awe

	Disney's "Star Tours" is an example of a ride film.

The ‘ride film' is a relatively new entertainment medium, originally born out of the flight simulators used by the military, this has in turn given birth to an entirely new immersive and participatory entertainment industry.

Very often the ride film will tie in to a franchise or film license and extend a particular franchise's world vision, as in the case of Star Wars, Terminator, etc. In this respect, it also bears a great similarity to the extended narrative ideas of game franchises.

Audio is generally highly ‘magnified' and physically experienced way beyond reality in both ride film and IMAX film. Indeed, it could be said that many of these films could also learn the lessons of tension and release in producing a more convincing immersive experience. However, this is about entertainment, short, sharp and shocking. It is this where the subwoofer becomes the weapon of choice in the armory of the sound designer, to provide the necessary shock and awe for the audience, who generally are there to be ‘wowed'.

Tim Archer:

“Sound design and mixing for a ride film is more about high impact, fast moving pass-bys and letting go of reality. These ride simulators are usually set-up in a 5.1 configuration. The main difference is in the room itself. Not only are you dealing with a not-so-perfect acoustical environment, as in the IMAX or feature type theatres, you are also dealing with a large, radically moving, loud seating platform. The main approach is an ‘everything louder than everything else' type of entertainment. The challenge is choosing and blending sound effects that can create the right impact.”

This certainly explains much of the need for the loud volumes present in such rides; there is a certain amount of mechanical noise generated by the apparatus that needs to be covered up first in order to suspend the disbelief of the audience. Tension and release are the agents of narrative design here in the traditional ‘roller coaster' sense. Ride film narratives generally hit hard and continuously, almost deranging the senses with an onslaught of sound, immersive computer graphics and gut wrenching physical simulation. We are not talking about a subtle medium here. Yet in order to get the most out of this they do need to build up tension and there are moments for reflection and enjoyment of a floating sensation in the Star Wars simulation at Euro Disney. The use of surround in this medium is used more extensively, not to direct our attention as in the IMAX format, but to derange the senses, to confuse the viewer's sense of spatial awareness. Use of subwoofer for sheer physical impact is nowhere better illustrated and synchronized with both onscreen action and physical movement.

In terms of an overall aesthetic, video game and the ride film share an obsession with physicality. From rumble packs to the ‘derangement of the senses' experienced in such games as Medal of Honor. This is achieved through use of the surrounds and the sub channel. Currently the .1 channel is non directional, which adds to a loss of directionality associated with low-end. This is how a game will sound ‘cinematic' to most audiences. In video games, the sub channel will be used so much more than in a movie mix. This is generally tied to action in the game, and is also used as a physical reward for the player's interaction.

	Waiting in line for a ride film mirrors the latter moments of Medal of Honor.

Other sections of game audio are side by side with the ride film in terms of audio aesthetics. Although these depend, of course, very much upon the genre; a third or first person action shooter will make more use of sub than an RPG. Shoot-em ups may play on either a tension and release narrative structure, one subtle build-up and other part all out kill-death frenzy, or they may opt for the full out continual derangement of a ride film, in which case prolonged immersion may become a problem for the player. Ride films are designed very differently in terms of their experience. They last little more than five minutes after what may have been 2 hours of queuing up to get on the thing. In many ways even the queuing up and waiting for the ride can be mirrored in the tension and release of game play: lots of sneaking around, thinking, planning, and then five minutes of hardcore action as in latter moments of Medal of Honor. Higher quality home sound systems will enable developers eventually to better map and mix these peaks and troughs, to deliver more impact or subtlety where it is needed.

In terms of mixing content for theatrical and consumer surround, the main difference is the uncertainty of the end system as to which the game will be heard on, although THX are making tentative moves into the consumer surround market with their categorization of quality sound systems. With IMAX you have to mix in-situ on an equivalent size stage and this guarantees it will sound at its best in performance. In terms of surround the confusion of multi-point sounds in a huge battle is already up there with the ride film. The only thing needed is the assurance that the delivery system is going to provide an accurate reproduction of the mixing environment.

The major presentational problem between game and film is that films inhabit both theatrical (cinema) and consumer (DVD) spaces, while games inhabit only consumer spaces.