|
|
 |
 |
 | | | | |
 |
|||
 |
||||||
          |
 |
 |
||||
|
Designing Interactive Audio Content To-Picture
An Ideal Solution: Synchronizing to Actual Game Events I realized that if I could somehow hook my sampler up to actual game events, I would have a real to-picture sample editing environment. If the game were able to move that music keyboard pitch bend wheel for me, I could simply drive the car around and listen to work-in-progress samples in my sampler respond interactively. Sample tweaks would be an easy matter of reaching over and adjusting the instrument's settings. Then I could immediately see, hear, and feel how the edits were working in the game. If only there were a standard, real-time-event-based protocol that would allow the game computer to communicate with my sampler… While I was thinking this, an apple fell on my head, forever changing the way that game audio professionals would go about the task of creating interactive sound content. The key was the mysterious black wire connecting my music keyboard to my sampler. These two devices were already talking to one another via a standard, real-time-event-based protocol known as MIDI. While I was moving the pitch wheel on my music keyboard, it was using the MIDI cable to tell my sampler exiting things like: "0xE0 0x01 0x40!" ("Andrew just moved the pitch wheel to one notch above the center!") The sampler would then respond appropriately by pitch shifting sample playback. (Please see Figure 1.)
Another key ingredient was the fact that virtually every PC soundcard on the planet supports MIDI input and output, and can easily be hooked up to a sampler with a MIDI cable. As a composer, I was in fact already using this capability all the time with my music "sequencer" software. (A "sequencer" is a composition environment that provides a musician-friendly interface for recording and editing MIDI events.) The sequencer used the computer's MIDI output to transmit complex arrangements in real time for playback by instrument sounds in my sampler. (Please see Figure 2.)
From these two configurations, I extrapolated a third in which actual game events could be transmitted as MIDI events to my sampler in real time. All that was necessary to turn my sampler into a true to-picture sound editing environment was a tiny bit of code translating a specified game event (e.g. a change in RPM) to a corresponding MIDI event (e.g. Pitch Bend) and downloading it to the MIDI output port. (Please see Figure 3.)
To-Picture Adaptive Sound Design At Last! At the time, committing the audio programmer to learning a new multimedia hardware API didn't seem like a good idea. This programmer was not primarily a sound coder, and was extremely busy with mission critical implementations of other game systems. Also, the audio department itself needed a lot of higher priority playback and editing code, which took precedence when the programmer was available. Finally, no one was really sure how long this implementation would take, which made it hard to assess its actual cost. Eventually, Pseudo Interactive's audio programmer got too busy to write any audio code at all, so I took over the task. Soon afterwards, we were again involved in a project that required a large amount of adaptive sound design and - you guessed it! - an interactive car engine sound. As it turned out, it took me less than a day to modify my DirectMusic code to route player car RPM to a sampler via MIDI Pitch Bend output. Finally I was able to drive around in the game and have it control playback parameters for my work-in-progress effects! "Hmm," I thought, "the idle sound is way too high." I tweaked the transposition in my sampler and kept driving. "Hmm," I thought again, watching the car move and gauging the responsiveness of the sample, "the overall range of the pitch shift should be a lot larger." I adjusted the pitch shift range of the sample. "Hey, Rich!" I yelled. "Come tell me what you think of this for the basic engine loop!" Rich thought it sounded pretty good. I made him drive for a bit while I watched and tweaked sampler settings. "Thanks," I said, writing down the optimal transposition and pitch range for the sample. The events described in this paragraph took about as long to happen as they take to read. "Hmm," I thought, "that was worth it!" Rounding Out The System As it turns out, MIDI is a perfect solution for real-time synchronization between A) the editing environment provided by a pro sampler and B) game events. Your audio department already works with MIDI all the time, and MIDI does not have to be just for music. All that the MIDI (Musical Instrument Digital Interface) protocol really specifies is a standard for the transfer of real-time events. Since the way that a MIDI device responds to a particular event is device-defined, MIDI is already a common real-time control method for many non-musical applications (such as theatre and concert lighting). Pro samplers are designed to allow a lot of their MIDI implementation to be user configurable. The sound designer is able to decide: what samples a MIDI "play middle C" message triggers, their relative loudness, their basic pitches, whether they're looping, &c. In addition, a MIDI Pitch Bend message does not necessarily have to control pitch bend. It could just as easily be routed (in the sampler) to modulate the sample's volume. Since the sound designer already has control over all of the playback details in the sampler, the programmer only has to worry about sending a few simple MIDI message types: Note On (e.g. "play middle C"), Note Off (e.g. "stop playing middle C"), and Control Change (e.g. "the pitch bend wheel is 3 notches above center"). These messages correspond almost exactly to the audio Play(), Stop(), and Set() functions you probably already use in your game, and can be used interchangeably with very little setup. Each of these message types simply specifies an event ID (what note or controller it refers to) and a value. Presumably (hopefully!) your game editor already allows sound designers to link game events to sound content, and to set some playback parameters. Rounding out the system consists of adding A) a toggle for routing a given hook to MIDI and B) a couple of fields for specifying the MIDI event type, ID, and value (or range of values). (The setting of these values should be "musician friendly" - sound designers will want to refer to MIDI play event IDs by "note name" rather than numerically or in hex!) Once this is done, they can select the sounds they want to work on in their sampler at whim without requiring constant programmer involvement. ________________________________________________________ |
|
|
Features
