In this reprint from the final (June/July 2013) issue of Game Developer magazine, sound designer Damian Kastbauer shares this excerpt from the Oxford Handbook of Interactive Audio (due 2014), in which he blends narrative and design notes to craft an inspirational future vision for interactive audio.
I'm standing on the bank of a river. It's late, and the heat of the afternoon sun is fading. The sounds of cicadas are all around me. Their placid chirruping, accompanied by the nearby burbling of a stream, sets a tone of peaceful tranquility. The simulation here is really good.
As I kneel next to the shore, splashing water on my face, the sound erupts in a perfectly modeled and synchronized symphony of harmonic fluids. Each drop of water from my fingertips that ripples on the surface is reflected in automatic sound synthesis and tied so tightly to the visuals that I'm struck by the fact that it was just a short time ago that we were locked into a sample-based methodology. It didn't happen overnight, but as I am standing on the shore of this digital river, it seems clear to me that audio technology has finally made good on the promises of transparent interactivity.
It should be said that I'm inputting this log in real-time, while working inside the beta release of our current simulation based on a historical re-creation of Earth circa 2012. The simulation has been developed using an industry- standard authoring application by specialists spread out across the galaxy. I'm currently reviewing the work done by our technical sound synthesis specialists who have recently updated some old sound models. It seems like ages ago that we began working with our library of reference materials to synthesize different aspects of Earth's soundscape into the living, breathing, and sounding world that is represented today.
I remain lost in thought as the sound of rushing water catches my memory. My mind is transported to a sunny day from my past. A reunion has brought family members together within a branch of an older beta version of a simulation located by a nearby stream in the countryside of Earth 2012. It was during this time in simulation technology that our industry was just beginning to iron out inconsistencies inherent within the burgeoning field of procedural audio, synthesis, and the advanced manipulation of dynamic sound: baby steps toward the expansive fully realized simulation I'm testing today.
As we laugh and carry on inside the memory of my mind's eye, the children play and chase butterflies along the edge of the rushing water. From the corner of my eye I can see my young daughter rushing in and out of cattails twice her height with her favorite doll. Her look of freedom and wild abandon while chased by cousins as the fronds whoosh back and forth brings a smile to my face. I move to speak and urge her to stay clear of the undulating, black, and treacherous stream edge, but they are gone before the words leave my lips.
I become lost in debate with various relations over the use of sound as a storytelling device. It seems we have finally found a way to effectively use all five senses to convey an emotional arc as part of an interactive experience. Of course, there continue to be new ways to channel and leverage our history through simulations, new ways of combining what has already happened with new technologies as a way of making sense of the future. Through the lens of creativity, we may finally understand who we are and where our civilization is headed.
Back at the reunion, the sun is beginning to set in the distance, and it's time for us to take leave of this place. I head toward the pack of children collapsed by the riverside in search of my daughter. My steps echo coldly in the grass. Her smiling face goes unseen, so I ask the gathered assortment of nieces and nephews where she might be. No one has seen her for some time, they say.
In a moment, I feel my every fiber stiffen toward a heightened awareness of my surroundings. Nothing has changed, and yet in that instant it's as if all sound had been removed but for the deep churning of water. I look around frantically, my eyes darting between reeds as I began calling her name. People are gathered around me, I can see their mouths moving but can no longer hear their questions. My mind is sharply tuned to the sound of dramatically frothing whitecaps as I desperately attempt to keep my thoughts above water.
I wade into the sea of cattails, moving further and further upstream. Their fronds brush past me in a noiseless hush of extreme focus. I can hear others who have crossed a nearby bridge and begun combing the riverbank on the other side. My voice, becoming louder and more urgent, punctuating the ebb and flow of water to my left, sounds far away. A curious rocky outcropping comes into view, as the taste of saltwater reaches my lips. I see my daughter playing noiselessly with her doll in the shade of a dark black stone column. I call to her, but she doesn't look up. I quickly close the gap between us and suddenly recognize that the sound of the stream has disappeared.
My daughter becomes startled by the sound of my voice calling her name. She looks up as I rush to her and hold her in a deep embrace, my footsteps resonating with an alien quality. The sound, clothing brushing against her white dress, explodes within the vacuum of soundlessness I find myself enveloped in. I look into my daughter's eyes as she holds out a ring of flower stems that she's woven together. She looks at me and smiles. When asked by friends later she'll say, "I wasn't lost, I just found a quiet place to play." We emerge from the noiseless interstitial space, call off the search party, returning to the sound of the world around us.
These early simulations weren't perfect, but the feelings generated by them resonate just the same. The difficulty of authoring propagation for every explicit sound once meant that the process was subject to gaps in the handwoven audio fabric. Every inch was "painted" by hand with sounds. It's easy to see, looking back on the technology, how these "dead spots" could exist—these places where no extraneous sounds would reach. Thankfully, people continued to work toward accurately representing sound propagation within the simulation . These advances were, over time, able to leapfrog the manual process that had been bogging down the process . I have to say that the radical sound mix change brought on by my alerted state operated perfectly, in retrospect. It's hard to believe the leaps and bounds that technology has achieved, even during my short life span— especially as I have found myself deeply embedded in driving the modern state of the art in our re-creation of Earth.