Gamasutra: The Art & Business of Making Gamesspacer
Envisioning Our Interactive Audio Future
View All     RSS
October 23, 2014
arrowPress Releases
October 23, 2014
PR Newswire
View All

If you enjoy reading this site, you might also want to check out these UBM Tech sites:

Envisioning Our Interactive Audio Future

July 3, 2013 Article Start Page 1 of 3 Next

In this reprint from the final (June/July 2013) issue of Game Developer magazine, sound designer Damian Kastbauer shares this excerpt from the Oxford Handbook of Interactive Audio (due 2014), in which he blends narrative and design notes to craft an inspirational future vision for interactive audio.

Milestone 1: Power On

I'm standing on the bank of a river. It's late, and the heat of the afternoon sun is fading. The sounds of cicadas are all around me. Their placid chirruping, accompanied by the nearby burbling of a stream, sets a tone of peaceful tranquility. The simulation here is really good.

As I kneel next to the shore, splashing water on my face, the sound erupts in a perfectly modeled and synchronized symphony of harmonic fluids. Each drop of water from my fingertips that ripples on the surface is reflected in automatic sound synthesis and tied so tightly to the visuals that I'm struck by the fact that it was just a short time ago that we were locked into a sample-based methodology. It didn't happen overnight, but as I am standing on the shore of this digital river, it seems clear to me that audio technology has finally made good on the promises of transparent interactivity.

It should be said that I'm inputting this log in real-time, while working inside the beta release of our current simulation based on a historical re-creation of Earth circa 2012. The simulation has been developed using an industry- standard authoring application by specialists spread out across the galaxy. I'm currently reviewing the work done by our technical sound synthesis specialists who have recently updated some old sound models. It seems like ages ago that we began working with our library of reference materials to synthesize different aspects of Earth's soundscape into the living, breathing, and sounding world that is represented today.

Milestone 2: Soft Focus

I remain lost in thought as the sound of rushing water catches my memory. My mind is transported to a sunny day from my past. A reunion has brought family members together within a branch of an older beta version of a simulation located by a nearby stream in the countryside of Earth 2012. It was during this time in simulation technology that our industry was just beginning to iron out inconsistencies inherent within the burgeoning field of procedural audio, synthesis, and the advanced manipulation of dynamic sound: baby steps toward the expansive fully realized simulation I'm testing today.

As we laugh and carry on inside the memory of my mind's eye, the children play and chase butterflies along the edge of the rushing water. From the corner of my eye I can see my young daughter rushing in and out of cattails twice her height with her favorite doll. Her look of freedom and wild abandon while chased by cousins as the fronds whoosh back and forth brings a smile to my face. I move to speak and urge her to stay clear of the undulating, black, and treacherous stream edge, but they are gone before the words leave my lips.

I become lost in debate with various relations over the use of sound as a storytelling device. It seems we have finally found a way to effectively use all five senses to convey an emotional arc as part of an interactive experience. Of course, there continue to be new ways to channel and leverage our history through simulations, new ways of combining what has already happened with new technologies as a way of making sense of the future. Through the lens of creativity, we may finally understand who we are and where our civilization is headed.

Back at the reunion, the sun is beginning to set in the distance, and it's time for us to take leave of this place. I head toward the pack of children collapsed by the riverside in search of my daughter. My steps echo coldly in the grass. Her smiling face goes unseen, so I ask the gathered assortment of nieces and nephews where she might be. No one has seen her for some time, they say.

Milestone 3: Debugging Memories

In a moment, I feel my every fiber stiffen toward a heightened awareness of my surroundings. Nothing has changed, and yet in that instant it's as if all sound had been removed but for the deep churning of water. I look around frantically, my eyes darting between reeds as I began calling her name. People are gathered around me, I can see their mouths moving but can no longer hear their questions. My mind is sharply tuned to the sound of dramatically frothing whitecaps as I desperately attempt to keep my thoughts above water.

I wade into the sea of cattails, moving further and further upstream. Their fronds brush past me in a noiseless hush of extreme focus. I can hear others who have crossed a nearby bridge and begun combing the riverbank on the other side. My voice, becoming louder and more urgent, punctuating the ebb and flow of water to my left, sounds far away. A curious rocky outcropping comes into view, as the taste of saltwater reaches my lips. I see my daughter playing noiselessly with her doll in the shade of a dark black stone column. I call to her, but she doesn't look up. I quickly close the gap between us and suddenly recognize that the sound of the stream has disappeared.

My daughter becomes startled by the sound of my voice calling her name. She looks up as I rush to her and hold her in a deep embrace, my footsteps resonating with an alien quality. The sound, clothing brushing against her white dress, explodes within the vacuum of soundlessness I find myself enveloped in. I look into my daughter's eyes as she holds out a ring of flower stems that she's woven together. She looks at me and smiles. When asked by friends later she'll say, "I wasn't lost, I just found a quiet place to play." We emerge from the noiseless interstitial space, call off the search party, returning to the sound of the world around us.

These early simulations weren't perfect, but the feelings generated by them resonate just the same. The difficulty of authoring propagation for every explicit sound once meant that the process was subject to gaps in the handwoven audio fabric. Every inch was "painted" by hand with sounds. It's easy to see, looking back on the technology, how these "dead spots" could exist—these places where no extraneous sounds would reach. Thankfully, people continued to work toward accurately representing sound propagation within the simulation . These advances were, over time, able to leapfrog the manual process that had been bogging down the process . I have to say that the radical sound mix change brought on by my alerted state operated perfectly, in retrospect. It's hard to believe the leaps and bounds that technology has achieved, even during my short life span— especially as I have found myself deeply embedded in driving the modern state of the art in our re-creation of Earth.

Article Start Page 1 of 3 Next

Related Jobs

Crystal Dynamics
Crystal Dynamics — Redwood City, California, United States

Audio Lead
The Odd Gentlemen
The Odd Gentlemen — Los Angeles, California, United States

Sound Designer
Digital Extremes
Digital Extremes — London, Ontario, Canada

Sound Designer


Robert Schmidt
profile image
I may be off base but I feel that the advances in audio cards lags way behind that of video cards. I would expect on-board voice recognition and synthesis by now. I guess the market doesn't put as much pressure on the audio card industry. It seems to me that many audio applications could use the same sort of massive parallelisms as video cards. Perhaps nvidia should jump in.

Damian Kastbauer
profile image
I agree, advancement in audio processing support has lagged behind support for fancy pixels on the screen. Whether it's dedicated hardware, or pure CPU, the resources are usually reserved for graphics. Both the original Xbox and (rumored) Xbox One have dedicated hardware specifically for use by audio. Even without, the current generation has seen incredible leaps in by quality, technique, and dynamism, I think, due to the advancement of common and (somewhat) standardized audio middleware toolsets. It will be interesting to see how game audio technologists harness dynamic sound during the next console gegneration.

Xbox One (Durango): Sound of Tomorrow:

Kevin Fishburne
profile image
Physically realistic sound generation is nice and all, but best case scenario it'll match the amazing graphics which accompany entirely unrealistic and neglected mechanics. If you take away the graphics, and in your case the audio, what's left is a terrible simulation of what actually happens when your player does what they're allowed to. If games are defined by the interactions of the characters within them they fail harder in this respect than any other. I'm all for better audio, but as they say putting lipstick on a pig doesn't make it beautiful.

Have an example:

Unbelievably terrible. The first woman runs away, yet the others keep working in the mirror without even flinching or asking what happened. Despite being right in front of mirrors (wide field of view), they somehow don't see you. After you kill the first one (in the middle!), the others don't take the slightest of notice even with all the "slinking" sound effects. Pretty much all games are like this.

Damian Kastbauer
profile image
Your argument has little to do with audio (other than using this article as a soapbox for your opinion) and your example is an illustration of the worst kind as to the potential of interactive experiences.

Todd Masten
profile image
People will settle for audio that is 'good enough' but will force visual boundaries to the breaking point. It is the nature of the beast. Mainstream audio quality has actually devolved over the past decade as we switched from CD/DVD/Bluray to MP3 as our main source of aural delivery. I can sit with a 5.1 Bluray playing in my living room and marvel at how amazing the experience is over two channel highly compressed MP3, but most people do not care. I worked in the industry for years trying to push audio forward, but encounter budget constraints, memory constraints, resource constraints, and a general 'meh' attitude towards making the audio extraordinary. The industry is fine with 'good enough', so that is what it gets.

Rob Bridgett
profile image
Just re-reading this.
I like the sense of the creator/engineer eventually becoming some sort of 'pilot' of the whole experience. I think the speculative fiction you have conjured up (either consciously or unconsciously) blurs the line we currently have between game player and game creator, that is a very possible future (and in some senses the present) the gamification of game making.
Bold article. Great read.