Contents

The Problem Adaptive Sound Design: Take The Problem And Square It An Ideal Solution: Synchronizing to Actual Game Events Budget Justification / Turbo-Charging the System Executive Summary

Budget Justification

The Andrew Clark Method of to-picture interactive audio design provides a number of invaluable benefits. Synchronizing a sound editing tool in real time to actual in-game events allows the designer to:

The most important budget justification for such a system is that without it, you're wasting audio resources. If your sound specialists are not able to work in context, they are not working to their full potential. A lot of their talents and skills are going to waste. Game sound designers are tired of getting slagged because their effects don't sound as good as movie effects, so give them a working environment in which they can compete. (In fact, this system is more powerful than the to-picture sample editing environment available in film sound design, since it automates the synchronization process.) Similarly, your company has probably already invested good money in the basic audio hardware necessary for this system: a good quality pro sampler. This device could be working a lot harder for you.

If you don't have a pro sampler in your setup, the bad news is that getting a machine with good on-board sample editing capabilities costs a lot more than a software sample editor does. Non-pro samplers simply don't offer much in the way of this kind of functionality. Keep in mind the incredible value of a sampler's real-time interactivity and bite the bullet. It's not really that bad. An E-mu E6400 Ultra costs under $2200 US and is in fact used as a primary sample editor by a lot of film sound designers. Don't be stingy - make sure that the model you get has all of the sample editing tools that you need.

The real cost associated with this system is the development of the code necessary to route game events to your sound card's MIDI output port as MIDI messages. All I can say is that it's worth it. The good news here is that if your program already happens to be using Microsoft DirectMusic, functionality for accessing hardware ports and downloading MIDI messages to them is already included in the API. Of course, for a small sum you can always get me to come help you set up! http://www.silvershard.com

Turbo-Charging The System

Figure 4 shows my dream hardware setup for adding some real sample editing power to the system. This configuration adds a phenomenal amount of dedicated DSP processing to the environment in order to enable almost any type of to-picture sample editing in real time. With this configuration, any tweak can be auditioned right away, without the designer having to wait for effects to render.

Figure 4. The dream hardware set-up.

Symbolic Sound's Kyma system is a sound design workstation. It's like a sampler in that it's playable with MIDI events, and allows user configuration of the events' interpretation. It's like sample editing software because its large number of sample editing functions are accessed through a software front end (as opposed to the four-line LED interface you get with a typical sampler). It's like nothing else on the planet because it adds so much hardware DSP. This allows real-time tweaking of the sample edit functions that traditionally take the longest time to render (non-transposing time stretch, non-time-stretching pitch shift, complex spectral transformations, &c). The Kyma system doesn't cost much more than a pro sampler, weighing in at $3300 US for the basic configuration. On the down side, the consensus seems to be that it's pretty complicated to use.

Digidesign's ProTools package is another dedicated DSP solution that gives the designer access to a virtual rack of audio effect processors. These are called TDM effects, and are purchased separately. They are quite powerful (as good as rack units) because they run on ProTools' DSP farm. TDM plug-ins are designed by Digidesign and a number of third-party companies, and are of excellent quality. Again the interface is a software front end. The latest version of ProTools costs around $10K US though, and the plug-ins are expensive too. On the other hand, ProTools is an audio industry standard for Digital Audio Workstations.

Bonus Hypothetical Configurations

In this article I haven't written much about composing music to-picture in non-linear environments. Figure 5 shows a solution that, while as yet untested, should allow a composer to do just that. It relies on a feature that is (as far as I know) unique to Emagic Logic among pro music sequencers - something that Emagic calls "touch tracks." Available only in Gold and Platinum versions of the software, "touch tracks" allow the user to map MIDI events to trigger the playback of whole sequences. Game cues could easily be used to trigger work-in-progress arrangements interactively via this mechanism. Note also that Logic Platinum interfaces beautifully with ProTools hardware and TDM plug-ins.

Figure 5. Composing music to-picture in non-linear environments.

The final configuration example that I have provided isn't even a game audio solution - it's a setup that would hypothetically allow visual artists to work contextually on in-game animations. (Please see Figure 6.) The only reason that I have suggested it here is because it was recently brought to my attention that some animation packages support using MIDI input for certain controls. (Thanks to Mike Ottom, currently animating at Relic - http://www.relic.com.) I checked it out (very) briefly in the 3D Studio Max v.3 manuals. Sure enough, the package supports using MIDI events as motion capture controllers, and allows MIDI note IDs to be mapped to simple playback controls. I don't actually know whether using these features to synch animations-in-progress to game data would be practical or useful. This remains a topic for future research.

Figure 6. A solution for in-game animation.

________________________________________________________

Executive Summary