|
|
 |
 |
 | | | | |
 |
|||
 |
||||||
          |
 |
 |
||||
|
Designing Interactive Audio Content To-Picture
The Problem My purpose with this paper is to present a detailed analysis of - and a surprisingly simple solution to - a universal problem faced by game sound designers and music composers. The critical issue that this paper addresses is that there are currently no established tools or methodologies allowing interactive sound designers or composers to work "to-picture." The first half of the article explains the extreme importance of "to-picture" audio design environments for successful AV sound content creation, and the second half describes a cheap, fast, and incredibly effective way to turn the audio resources that you already have into the interactive audio design environment of the future. In the television and film industries, sound designers are able to monitor final visual cues from within their design environment while creating and mixing audio content. This design process is called working "to-picture," and explains why the effects just seem to fit so well. When you're at a horror flick watching the stupid teenagers decide to split up to explore the haunted house, you're watching the exact footage that the sound designers were watching when they were creating that "creaking door" sample. For linear media, to-picture design is supported through some form of video display (or with a video synchronization interface) by virtually every sound editing program, digital audio workstation, and music sequencer out there. (Heck, even low-end hobbyist multi-track tape recorders support SMPTE synchronization.) Unfortunately, this type of synching is just not available to in-game audio designers. Even getting to hear a sample in the game can be a multi stage process potentially requiring the involvement of several non-audio personnel (a programmer, the level designer who owns the effect, the level designer who owns the only level that is currently stable enough to run, that guy with the really annoying laugh, and the building's elevator maintenance staff). Actually being able to edit samples while the game is running seems clearly out of the question. This is really too bad, since not working to-picture is the direct cause of unnecessarily large numbers of expensive sound design callbacks in the gaming industry, poor aural experiences for legions of unfortunate gamers, and (most tragically) immeasurable frustration for game sound content developers the world over. Why Designing Sound To-Picture Is Critical Working to-picture is incredibly important because of the implicit first goal of AV sound design: the audio must fit the visuals. For some reason there is still a big misconception that game audio can simply be created to spec, which is just not the case. Don't get me wrong - a detailed audio spec is critically important! It makes sure that everybody is on exactly the same page about what each game sound is supposed to be doing. (For instance, a critical player awareness cue will be created much differently from an environmental sound event, since it serves a different purpose.) It's just that in this case a picture really is worth a thousand words, and a moving picture is worth a thousand callbacks. Creating sound content to-picture gives the artist an immediate and exact feel for what the aural effect needs to be, is inspiring, and automatically provides detailed and frame-accurate timing information. Sound designers will always work more productively to a visual. In Aaron Marks' interview with Tommy Tallarico Studios sound designer Joey Kuras ("Interview with Joey Kuras", Gamasutra, October 15th, 1999), Kuras makes the following observation: "The cinematics usually make it through on the first pass … we send it to the publisher or developer and they usually say that it is great and put it in the game. There are rarely revisions with cinematics because I'm looking at it." Conversely, "if you just give me a list of sounds, I'll do them but I won't know if they are right and then the revisions start rolling in." Mark Steven Miller's brilliant article "Annotated Scrawl from the Wall of the Audio Hall" (available at http://www.iasig.org in the "Publications" section) simply states: "SFX [sound effects] should be created based upon animated and implemented elements. Anything else is simply guess work." Making a sound effect for a visual you can't see is like trying to shoot a target blindfolded. Why Working In The Mix Is Critical A less
widely recognized effect of the absence of a to-picture game sound
design environment is that individual samples do not get developed
within their aural context. Audio artists are not able to properly
monitor the existing game sounds that a work-in-progress effect will
have to fit with. (The way in which various individual sound effects
fit together is referred to as "the mix." Setting amplitude
levels and EQs for the mix is referred to as "mixing.")
The scope of this problem will be a little tricky for non-audio-professionals
to grasp, so let's go back to our earlier hypothetical horror flick
example in order to illustrate the situation more clearly. In this case, the intensity of the effect relies heavily on a jarring scrape in a certain specific frequency range. This range is being partially "masked" (swallowed) by a music cue in the scene that has a lot of harmonic content in those same frequencies. In context, the sample is simply not very effective. The team is able to hear this right away and can take appropriate steps to fix the mix. They might replace the sample with one that works better with the scene, swap in a different music cue, aggressively EQ or gain tweak the mix, play with the pacing of the audio cues, etc. The point is that the film sound designers are able to work on a macro level - sculpting an evolving "audio scene" - rather than being stuck (as game sound designers are) on a micro "sample by sample" level. In a typical game design environment, the preceding situation would likely result in the sound designers (never having had a real opportunity to monitor their sounds in context) receiving callback after callback with the unhelpful feedback that the sample "doesn't sound right" or just "sounds bad." A sound that is perfect on its own is not necessarily going to be perfect in the game! And while this scenario may seem obscure to the uninitiated, mix problems of this nature are a primary concern for audio professionals in most fields. Isolating sound design from the mix prevents sound designers from getting full use out of their most important tool - their ears. ________________________________________________________ |
|
|
Features
The
Problem 