Gamasutra - Features - "Designing Interactive Audio Content To-Picture" [12.20.99]

Designing Interactive Audio Content To-Picture

Contents

The Problem

Adaptive Sound Design: Take The Problem And Square It

An Ideal Solution: Synchronizing to Actual Game Events

Budget Justification / Turbo-Charging the System

Executive Summary

Sound designers and composers for in-game content are not currently able to work "to-picture."
This is the direct cause of large numbers of expensive callbacks, poor aural experiences for legions of gamers, and immeasurable frustration for game audio content developers.
A) Making sound effects for visuals you can't see is like trying to shoot a target blindfolded; B) isolating audio design from the actual mix prevents developers from being able to create cohesive "audio scenes;" C) not being able to edit "adaptive" (truly interactive) sounds in their unique playback context makes creating them an exercise in frustration.
Synching a pro "sampler" (music hardware sample player / editor) to game events with MIDI creates a real-time "to-picture" sample editing environment.
This allows audio designers to: A) sculpt a sound to match its associated visual perfectly; B) ensure that the sample works in the mix; C) use actual game events to control playback of in-progress "adaptive" effects; D) work with the "big picture" while designing individual samples; E) design sounds that actually fit their context.
Your company probably already has the required hardware (a pro sampler, almost any sound card, and some MIDI cables).
Your sound department is probably already very familiar with using MIDI.
The cost of developing the code to use MIDI hardware output is offset by the value of finally using your audio resources to their full potential.
Game audio professionals are tired of getting slagged because their work doesn't sound as good as the movies - give then an environment in which they can compete! A lot of their talent and skill is currently going to waste.

References and Resources

Marks, Aaron. "Interview with Joey Kuras." (http://www.gamasutra.com, San Francisco, CA: Gamasutra.com, October 15th, 1999.)

Miller, Mark Steven. "Annotated Scrawl from the Wall of the Audio Hall." (http://www.iasig.org: Interactive Audio Special Interest Group, March 1999.)
Includes a complete audio project-planning guide.

Messick, Paul. "Maximum MIDI - Music Applications in C++." (Greenwich, CT: Manning Publications Co., 1998.)
An excellent book on Win95 MIDI programming and a great introduction to the MIDI specification. Includes a MIDI ToolKit API and C++ source code.

Links

http://www.digidesign.com
The makers of the ProTools Digital Audio Workstation.

http://www.emagic.de
The makers of Logic, a pro sequencer featuring "touch tracks," which allow MIDI events to trigger entire sequences.

http://www.iasig.org
The Interactive Audio Special Interest Group has a lot of great articles and maintains an industry news archive.

http://www.microsoft.com/directx/overview/dmusic/
DirectMusic is a part of DirectX versions 6.1 and later.

http://www.midi.org
The MIDI Manufacturers Association has MIDI information on-line and sells the complete specification for $50 via snail-mail.

http://www.silvershard.com
My web site. I provide interactive audio consulting services, including system and tool design, project planning, and audio fire-fighting.

http://www.symbolicsound.com
The makers of the Kyma sound design workstation.

Acknowledgments

I'd like to thank a few people for providing invaluable feedback, editing suggestions, and structural advice at key points during the development of this article: Pat Clark, Dan Oughton, and Mike Osbourne. Thanks! Also, gratitude and love to everyone I worked with at Pseudo Interactive (http://www.pseudointeractive.com), for supporting my first tentative steps towards exploring the audio frontier of non-linear / interactive delivery systems, for generously allowing me to share some of what I learned at PI in this paper, and for the memories.

Upon finishing his BMus (Composition) at the University of Toronto, Andrew immediately began composing music and designing sounds for a Toronto-based PC game development company called Pseudo Interactive. Through 18 months of their first long-cycle PC game and a short-cycle prototype he was Audio Lead and Lead Composer. For Pseudo Interactive's two most recent short-cycle prototypes he was Sound Programmer as well. Andrew left Pseudo Interactivein September of 1999 in order to focus on his independent research in interactive music technology. He pays the bills with contract composition and game audio consulting through his Toronto-based company, Silvershard (http://www.silvershard.com). His e-mail address is andrewclarkis@home.com.

________________________________________________________

[Back to] The Problem


				\|\|\|\|