Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Gamasutra: The Art & Business of Making Gamesspacer
Envisioning Our Interactive Audio Future
arrowPress Releases
July 28, 2021
Games Press
View All     RSS

If you enjoy reading this site, you might also want to check out these UBM Tech sites:


Envisioning Our Interactive Audio Future

July 3, 2013 Article Start Previous Page 2 of 3 Next

Milestone 4: Audio Beta

I return to testing inside the current-day simulation. I stand up from splashing water on my face and dry my hands on coarse cotton pants. This elicits a similar soft explosion of sound as on that day so long ago. I'm reminded of how long it took us to get the subtlety of movement and material interaction to accurately reflect the complex dance of physical sound .Tying into the cloth simulation was the easy part. Once we had all of that data on our hands, it was purely a process of sculpture (that is, mostly subtractive). Sound models became more complex still with the addition of liquids and temperature... but at the end of the day, our flexible synthesis models stood up to the barrage of parameterization from each of the systems.

I step back onto the path and continue on to a nearby watermill. I'm reminded by each footfall how long it took to get these footsteps right. Everything is now handled inherently by the procedural models in place, which take all sorts of factors into account: mass, velocity, materials, viscosity, and a variety of other things. What began as a purely academic pursuit swiftly blossomed into a creative endeavor thanks to some early front-runners in the field. Once the underlying mathematics were in place, the tools quickly followed. This development allowed early designers of sound to move beyond hard science running behind the scenes. Enabled by smart and creative authoring applications, the sound of footsteps and movement was able to transcend the synthetic quality of early experiments and emerge as a realistic representation.

As I approach, I note the churning sound of water on the undershot-wheel paddles grows louder with each step. I'm struck by the quality of dynamic frequency filtering over distance. The simple 3D point sourced, volume- only attenuation curves of yesteryear have long ago been replaced by a complex matrix of real-time distance modeling, taking into account everything from air density to temperature and humidity. The spinning wheel splashes on effortlessly while some unseen turbine labors to convert the power of nature into energy. Of course, this one is just for show. However, it wouldn't surprise me to find that it powers all of the lights inside. In such a complex woven fabric of interaction, you can usually count on everything being designed with a purpose.

I walk to the door of the water mill and reach for the handle with trepidation; the last time I tested it, it didn't accurately convey the weight and material type, so I sent the sound designer back to the drawing board. Turning the knob I hear the rusty mechanism come to life with a sharp snap of metal, hollow-wooden resonance, and deep unsettling creak. Each individual aspect of the sound takes into account the compositional breakdown occurring: damp location (as evidenced by the deep rust flaking off each hinge), wood density and thickness, and the age and tenacity of the steel. The door closes with a deep thud that reverberates as I let it close behind me . That was much better, I think, as I send off a friendly note to the designer who is a million miles away. Within moments I've received a response which reads like a sigh of relief.

With so much to do, it's hard to get caught up on the little things. Thankfully, the work we've been doing on the underlying physical modeling propagates throughout the entire world. Imagine if we had to place sounds by hand for every river or door; the worlds we're creating are just too large for such a labor-intensive process. While we do try to take a handcrafted approach to sounds that require something unique, the expectation that every nook and cranny sound convincingly Earth-like in this simulation involves more than just a few sound files spread across the face of the planet.

Milestone 5: Stress Test

Back inside, there is the pervasive, low-end rumbling sound of rushing water coupled with an oppressive clattering. I'm standing in front of a succession of wooden gears locked together in a spiraling groove of perpetual motion. The hollow "thonking" sound they make as each tooth locks into place is a direct result of the latest wood models developed by the foremost modal synthesis historians. Piggybacking on their research into old technologies and material types has given us a satisfying level of detail to the rich characteristic of wood. With their research in place, we were able to further embellish the machinery using several creative tools running within the simulation.

The whole contraption is shuddering with an unbelievable rumble that I can feel in my gut. With a gesture, I engage the authoring tools, and an interface for interacting with the world appears in front of me. As it springs to life, ready for action, I quickly navigate the controls and begin slowing down the water flow outside in order to hear the change in sound.

As the internal mechanism begins to wind down, there are no artifacts in either the pitch or timbre as the gears begin slowing to a halt—this isn't a simple sound file representation. You see, the only remaining samples of an actual watermill were recorded at the low sample rate of 192 kHz/24 bit, but we were able to use feature extraction across a diverse sample set and mine relevant data from these recordings and use it to inform the creative application of various processes and models. These samples were critical, since we had never seen a working watermill in person, and ended up affecting the overall sound presentation.

As things grind to a halt, I notice a gentle whistling sound finding its way through the cracks in the thatched roof overhead. Wind was the first and easiest dynamic synthesis we could apply to these simulations—we could apply pitched and filtered noise of different colors in combination with reflectors and deflectors, both abstracted within authoring toolsets and programmatically based on geometric representations within the different environments. This technology was very futuristic at the time, and what it lacked in "natural" sound factor, it made up for in its ability to be modified in real time, using parameters from the simulations. As the technology progressed, the randomness of nature swiftly took the place of consistently sloping pitch curves and unreal-sounding representations.

My footsteps echo convincingly on the hollow wooden floor as I resume my test and calibration procedures. I find the watermill synthesis and modeling holding up well under these extreme circumstances. I stop my testing just short of flooding the entire mill in order to listen to the resulting forces of nature on the tiny primitive structure. Safe within the confines of the simulation, the properties of the debug shard that instantiated when I enabled the authoring tools gave me a unique perspective on the resulting mayhem. Distanced from the confines of the simulated physical world within this unique instance, I'm free to run amok. I'll save the creative joy of destruction for another day. I start the river gently flowing again and exit the tiny building and continue my testing.

Milestone 6: Content Complete

Outside, the sky has blossomed into a majestic eruption of purple and pink at dusk. The late afternoon cicadas have all been replaced with a chorus of crickets and occasional bird chatter. It's been so long since we were restricted to a single looping audio file that I don't even notice how diverse the soundscape is. While there continues to be a place for field recording as the basis for building a library of granular models and anomalies, it's no longer possible to capture these sounds in their natural habitat. These sounds may have existed in nature a long time ago, but that time has since past. Luckily we have access to a wide variety of artifacts from the 21st century, including field notes, recordings, and skeletons.

Some of the environments initially proved difficult to re-create. Isolating elements amidst the noise of then- modern culture proved easy enough at first. When it became more difficult was when we began to note peculiar behavioral changes between recordings taken from different decades in many of the vocalizing birds and animals. As the rise of industrialized society began to take hold, so too did the sound of a new era of machinery and technology. These sounds become an inexorable part of the complex auditory fabric of the earth and, over time, completely modified the speech and frequency range of vocalizations wherever nature overlapped with industrialized society. The difficult parts then became understanding the complex interaction that developed over time, and finding ways to realistically represent the ensuing soundscape. The result is a blending of both natural and manufactured sounds in a complex cacophony quite unlike anything heard since.

I watch the sun slowly slipping behind the rolling hills off in the distance and I'm struck by the true beauty that the 2012 Earth embodies in our simulation. Everything is represented with a rhythm that resonates throughout, from the tall grass gently swaying in the breeze, to the water wheel working in ceaseless syncopation. It seems that in this moment, there was a balance between the elemental forces at work and the swiftly encroaching hands of progress. It's impossible not to judge the years that have transpired in the interim as tragic when faced with such beauty. We all hope that this experience can serve as a future roadmap for how to proceed as a society, now that the damage has been done.

Article Start Previous Page 2 of 3 Next

Related Jobs

Sucker Punch Productions
Sucker Punch Productions — Bellevue, Washington, United States

AI Systems Designer
Xbox Graphics
Xbox Graphics — Redmond, Washington, United States

Senior Software Engineer: Performance Tooling
Xbox Graphics
Xbox Graphics — Redmond, Washington, United States

Senior Software Engineer: GPU Compilers
Insomniac Games
Insomniac Games — Burbank, California, United States

Senior Hard Surface Outsource Artist

Loading Comments

loader image