Mixing and Surround Sound

MIXING & SURROUND SOUND

Introduction Digital Audio Mixing Dolby Pro-Logic Encoding Latency Versus Underflow Things I Learned

When you design an audio engine for use in a video game, the biggest problem you are likely to face is balancing latency with the fear of skipping caused by underflow.

Latency is the time between when the player pulls the trigger on the joystick to shoot the laser, and when he actually hears the laser sound effect. It is increased every time your sound effect goes through a layer of software or hardware. First, there is the delay between clicking the switch and the game noticing the switch is down. Next, consider the time it takes to set up the track to play, maybe a wait while it loads from disk, plus the time it takes to mix the track into the playback buffer. Add to that the time the first buffer of sound takes to get through the queue of buffers in the sound card driver to the hardware to play out through the speakers. In poor circumstances this delay can easily climb to a second or more.

In an ideal world the sound card would be playing one buffer while a mixing thread is mixing the next. In practice there is going to be at least one more buffer in between, waiting to be played. One thing to note when choosing buffer sizes is that most operating system level interfaces have a preferred buffer size specified by the sound card device driver and woe to thee who dares choose another buffer size. Always leave the user the ability to adjust the buffer size, even if it is just through an INI setting. Technical support staff will be grateful.

For 11KHz stereo output, 4K is a good size. Each buffer plays for 4096/(11000x2)*1000 = 186 milliseconds meaning that you should be able to keep the average latency down to around 1.5 times the buffer time or 279 milliseconds. 186 milliseconds should be adequate time for most modern operating systems to be able to schedule a high priority thread to mix the next buffer.

While reducing latency is an important goal, there is a danger in smaller buffer sizes and shorter queues to the sound card. A poorly designed operating system might ignore your mixing thread for longer than that average of 279 milliseconds (especially on slower computers) causing the audio to skip CD style, playing the last buffer over and over again until you feed it a new one. This is underflow. The goal is to eliminate it. Weapons against this are well-designed operating systems, high priority threads to do the mixing, and more buffers between the mixer and the sound card.