Austin,

You start off giving us your qualifications as an "audiophile" but you don't seem to list any work in audio on video games, so I'll assume that you have not actually done so. You also spend the rest of the article describing "adaptive music" from a marketing standpoint, so I'll assume that is what you actually do in the industry. The article you are "responding" to made no claims about using adaptive music to market games, so I'm not really sure where you were coming from when you wrote this, but as an audio designer for games and film, I felt the need to comment on some of your statements.

First off, you finish your second paragraph with a sentence that seems to contradict the rest of your thesis - "Left 4 Dead has the best adaptive music of any game I’ve played, and arguably the game would not be nearly as good as it is without this adaptive music."

You said it right there - "the game would not be nearly as good." If that is not a statement "selling" us on the quality of the game, I don't know what is.

Next paragraph you state - "All this aside, adaptive music in games is a tough sell.  From my perspective as a marketer, adaptive music simply will not sell games." Redundancy of these sentences aside, you are probably correct, adaptive music on its own will not sell a particular game. I have never seen any media product sold on the merits of its audio and this is, in my opinion, simply due to the fact that people don't realize how audio is affecting them as they have certain experiences.

Film editor and sound designer Walter Murch once said that while visuals come in through the front door of our perceptions, "sound is sneaking in the back door". As you played Left 4 Dead, the music changed depending on the circumstances of the gameplay. These subtle changes affected your overall perceptions of the gameplay in ways you probably didn't even realize, heightening the tension and thrills you were experiencing killing zombies. This tends to be such a subconscious manipulation of your senses that it doesn't even register as a factor for why you were enjoying the experience. This is a hard concept for most people to wrap their heads around, and I agree, certainly hard to sell in a marketing campaign.

You continue - "If we assume limited time and budget (and by extension design and programming hours), video game designers can get a better return on investment by investing in good graphics, a great storyline, and excellent level design.  These three things make a game playable and award winning." I agree absolutely, this is indeed what game designers should be doing. This is generally why studios work with audio designers to create the music and sound effects. It is true that the "investment" in game (or film) audio can be significantly less than that of the design, art, and programming side of the process. Amazing visuals will indeed "sell" a game. Smart and intuitive level design will win awards. Audio in general, and music in particular, colors the experience of each one. Design and art create the settings in which players move through a world, sound effects and music tell us how to "feel" in those settings. Adaptive music simply makes the experience more organic and, yes, immersive. Music that changes and adapts to the experience of the player will entrench that player deeper in the world the game has placed them in.

You go off the rails a bit with this statement - "I would argue that no matter how good the programming of the interactive engine, this engine could not score a good game trailer." Why would anyone want to? Trailers are linear and non-interactive, thus don't need adaptive music.

"Spending valuable budget on making the music in trailers excellent is far more important than good music in game from the point of selling units." In all of the projects I have worked on, the marketing budget tends to be wholly separate from the production budget. Look at any movie's reported "budget" on IMDB or BoxOfficeMojo - this is just the production budget. Double that number and you are a lot closer to what was really spent on marketing and other studio costs. If you have a game the company wants to make money off of a product, I guarantee they will throw whatever money they deem necessary at marketing to sell it to consumers, especially if it has a larger production budget.

"Using the same music in game and in trailers is simply more cost effective than programming a special audio engine and then having someone separately score and record the trailers." I'm not sure what you are really saying with this statement. As I have stated, marketing tends to have their own budgets. If you look at marketing campaigns for media products, there tends to be a very high percentage of advertisements that don't even use any music from the product they are selling. Most movie trailers tend to use scores from other movies simply because when the trailer is being cut, the score isn't ready yet. Game trailers tend to use a lot of "stock" score or licensed music. If the game (or film) is a sequel, then that music is generally used. Also, trailers are never created "in-engine" anyway - they are video captured, cut together, and given a sound pass after the fact so I don't see whether or not a game has an adaptive music engine would effect that at all.

"Most people now listen to their music on their mp3 players through shitty headphones, while they are doing other things." Very true, but when a person plays music while they are doing other things, they tend to select a type or style of music that fits their activity or mood. My mother probably isn't going to put on Snoop Dog while she is cleaning the house, Elvis is more her speed. Even if someone has set their Ipod to shuffle, they may still skip over a song they might not want to hear at that moment or whatever reason, thus "adapting" the music to their experience.

"Music is simply not important enough to most people to think about it.  Trends in consumer electronics confirm this point." I'm not sure what trends you are referring too. Simply look at Apple - the iPod and now the iPhone have completely turned that company around from a struggling PC competitor to a home entertainment giant. Retail media stores are folding because of Amazon and iTunes deliver their music to them faster and more conveniently. Music matters to people.

"Because nobody really knows how to program one of these adaptive engines, the initial cost of one of these engines would be phenomenal.  When factoring in all the time it takes to record individual instrument tracks, adaptive audio is simply too cost prohibitive for most games." No one? Really? Fmod has an adaptive music system integrated into their engine. Wwise contains an adaptive music system. Even sound engines not specifically designed with adaptive music systems can be used to create adaptive music with some creativity from the sound designer. To your other point, music tends to be recorded separately anyway. On my current project, we recorded the stings, brass, and percussion separately and had them mixed together for the final pieces. I even have those pieces as separate tracks in case I wanted to use them differently.

Your entire article shows a complete lack of understanding of your topic. Your main concern seems to be how to market something in the easiest and most established ways possible. I suggest you stick with that.

~Michael Kamper

Michael,

I agree with you whole heatedly. There are many problems with his statements. If you look at his profile you see that he is only a student.

I always find it interesting that people, even those that have been in the industry for years, still consider the audio for a game "simply not important enough to...think about..."

I wonder what the studios who have nice budgets for full orchestra's, recording studios, engineers, etc. would have to say about it since they spent all that money on something that is "just not that important".

Good response, Micheal.

-Darren

I think it's weird that nobody mentioned this talk at Austing GDC last year:
http://www.gamasutra.com/php-bin/news_index.php?story=20249
A Generative, Adaptive Music System for MMO Games

And I quote from there (about this thing being hard to sell)
" But Larson thinks that it's merely a matter of how you pitch. "I think we get less success as audio people saying, 'This is going to be so cool, give me time, money, and people.' We have more success saying what problems we solve, and I am killing the problems of [players] turning off your music, and killing your audio budget." "

There are no technical details at the article, but I was there and the idea was simple enough to be tried at home :)

Thanks Mike--you saved me a lot of typing! :-) I think Gamasutra should have vetted this before throwing it out.

That said, the "nobody knows how to program..." line was worth the read. :-) He's going to be quite upset when he finds out dynamic mixing has already shipped.

jeff

"There are many problems with his statements. If you look at his profile you see that he is only a student. "

Hey, that's not necessarily an indication of knowledge.

Jeff - just to make sure you understand, this is a Member Blog. Anyone can post a Member Blog - in fact, we welcome it.

I should probably start out by qualifying some of my statements. When I talk about “adaptive audio,” I meant to imply “cinema quality” adaptive audio. Many games already have some form of adaptive audio, even if it’s just playing a different audio track when an enemy approaches. In this post, I’m attempting to address David Vink’s article on truly cinema-like adaptive audio in games. I realize now that point was never adequately made in my original post. Calling it “adaptive audio” was perhaps a bit too broad.

If this change clears up some of the problems people take issue with in the original post, then great. For those who are still unsatisfied, some qualifications seem to be in order. A list for some of the more simple responses and rebuttals:

• By calling myself an “audiophile,” I was expressing the point that I am an individual who is enthusiastic about high-quality and high fidelity audio, not someone who works in the field.

• When I mention Left 4 Dead in the second paragraph, I was attempting to imply that the audio in game made the game better, not made the game. One would be safe in arguing that the adaptive “director” that placed enemies and items made Left 4 Dead the game that it is today. Left 4 Dead would still be an awesome game without the audio engine.

• I am by no means trying to underplay the value of sound design in games. As someone who is auditory in nature, I tend to notice good sound in games, and it enhances my enjoyment of a game much more than visuals. I am simply arguing that cinema quality adaptive audio would not be cost-effective, even if I would immensely enjoy it.

• Admittedly, my statements about music in trailers might be a bit off, at least as far as scoring for trailers.

• The “trends in consumer electronics” I was referring to with the iPod and shitty speaker phenomenon refers to the quality of the music, not its impact or popularity. My point in this statement was to get across the “it’s good enough” attitude people have towards audio. Anyone who has done even the tiniest bit of sound work knows that listening to low-bitrate audio through an iPod with the stock headphones leaves out so much of what is actually present in the music, yet it is acceptable to the vast majority of people. The same goes for large screen TV’s as highlighted in the original post. “Music matters to people,” but high-quality music does not.

Please correct me if I’m wrong, but these are the steps (at least from my conception of this process) that would be required for a cinema-quality music experience in games.
1. Short samples of music would need to be recorded/collected (in the same way one might create loops for something like Sony’s Acid Pro).
2. Each sample would need to be categorized by a variety of factors, not limited to mood, theme, feel, and appropriate setting.
3. The audio engine would need to be able to not only interpret the correct audio for the current mood, but it would need to anticipate what will happen in the next several seconds, as some samples will no doubt be better at transitioning into each other.
4. The audio engine would need to put together samples that are aesthetically pleasing to listen to. It would also need to keep track of what samples were used, so they didn’t get repetitive.
5. The audio engine would need to keep track of what melodies played during each event (for example remembering what it played when it encountered a NPC or area) so it could keep consistent musical themes for people/places/vehicles/etc.
6. As an extension of number 5, the audio engine would have to be able to subtly alter themes to keep the music interesting.

This sounds like a massive undertaking; please correct me if this is a mistaken assumption. I’m simply arguing that from a cost/benefit perspective, this type of audio engine wouldn’t be worth it in terms of units sold.

Having multiple pre-recorded tracks can come close to a cinema-like experience (think Left 4 Dead), but for open ended or sandbox games, the amount of pre-recorded music necessary to keep things fresh would be astronomical. Pre-composed music can also never perfectly transition between tracks unless that transition was taken into consideration when the music was composed.

The current process of designing sounds for games is not entirely foreign to me, as for several years I was seriously considering going to college for it. I have the utmost respect for how music influences humans on a level they are not truly conscious of, and how sound designers accomplish that. The core of this editorial is about business decisions, not creative ones. I’m looking forward to seeing how adaptive audio matures in the future, and would love nothing more than being proven wrong with a cinema-quality adaptive audio system in an upcoming game. I’m also looking forward to exploring this game in terms of profitability, and comparing it to others without said adaptive audio system.

Simon,

The problem was the fact that his post was "elevated" with the "Featured Post" label, putting it prominently on the front page, which indicated to me that Gamasutra "endorsed" his incorrect understanding and framing of the topic.

If Austin had published this blog post anywhere else on the internet, I might have simply ignored it. If he had simply wrote a post about how difficult it is to market adaptive music (and audio in general) to the masses, I would have agreed with him wholeheartedly. That may have been his original intent, but that is not what was written.

The reason I wrote what I did, and I apologize to Austin for being harsher than I should have been, is because it is one thing to have elements that I believe immensely enhance the gameplay experience dismissed as "simply not important enough to...think about..." in a blog post on a website, but it is wholly another for a website that that is entrenched with the GDC (through its shared parent company Think Services), an organization that prides itself on the technical and spiritual evolution of the medium, to seemingly "endorse" these views. What this says to me is that the editors of Gamasutra (or at least the blog editors) have just as little of an understanding of the topic as the writer in promoting it as they did. In fact, I would go so far as to say that the post violates Gamasutra's Terms of Service since, in my opinion, it is certainly "false, inaccurate, or misleading". That was truly the reason that I felt I had to write what I did.

Audio continues to be one of the most misunderstood (at best, dismissed at worst) aspect of games. The only way that its importance in the grand scheme of game production and theory can grow is if members of the audio community nip these kind of harmful misunderstandings in the bud. Maybe I'll write a Member Blog about it.

~Michael Kamper

ps - I noticed in preparing this response that Gamasutra changed its header since I bookmarked the site a few years ago. It used to be "The Art and Science of Making Games".

Simon,
Thanks for the clarification--much appreciated! I think my response is roughly the same as Michael's.

jeff

Austin,

That was a much clearer and more pointed argument. Thank you for clarifying some of your viewpoints.

I agree that it would be difficult for game music to achieve the same kind of perfect emotional synergy with what is happening visually in an interactive setting. It can be done (and has been done) on a smaller scale with important moments that can be predicted and planned for, but continuously changing music based on player interaction is much more difficult, and for most games, probably not even desired.

I would also agree that the "Director" aspect of L4D was a huge part of what made that a great game. I would argue that the game was elevated to "awesome" by the adaptive music. Can you play the game (or any game) without the music? - yes. Is the experience going to be wholly different? - yes. Music that works with the player and responds to his actions grabs a different part of the player's brain than the visual gameplay and sinks the player further down into the immersion of the experience. A well done score and detailed music implementation will turn a great gameplay experience into a fantastic one, and the player won't even realize why most of the time. That's the Catch-22 of Audio - if we do our jobs well, no one will even notice we did anything at all.

Your “trends in consumer electronics” is valid but off-topic. Having music color their experience is important to people - how they choose to listen to the music is not. Admittedly, having your work listened to on crappy speakers is painful but inevitable. That is why it is common practice for sound designers and music mixers to listen to their work on high end pro speakers and crappy $50 consumer models. You have to make sure that the sound you are producing translate to any kind of output device. You certainly can't count on your game being played by everyone on a $10,000 5.1 theater system, so you better make sure your work sounds good coming out of the mono speaker of a 30 year old CRT TV. However the music reaches the listener's ears, the inclusion of it is still important.

Your breakdown of how an adaptive music system might work is ambitious but probably overly complex. A lot of what you describe as the audio engine needing to determine would probably be preset by the sound designers. The samples wouldn't need to be broken down so minutely as you describe. A designer would be the one who would break down and arrange tracks by mood, theme and feel and in an "aesthetically pleasing" manner. The level of "anticipation" you describe was implemented very well in the game "Dead Space" where tension strings would build or lower depending on player proximity to a "fear emitter".

Mostly, you seem concerned with preventing repetition and keeping the music fresh and interesting. This is a huge factor of course in all aspects of gameplay since there is a lot of repetition that happens over the course of a game. Music already relies on looping cues over combat sequences or similar moments which can get very repetitive depending on the length of a track and how long the battle goes on. Find a safe piece of cover to hide in during any battle sequence of any game and you are bound to start hearing the same music repeating over and over again. No amount of adaptive music will ever prevent that (especially when a player is 'opting-out' of the experience in such a way), but what it can do is make the experience seem more varied and reactive to their actions.

And since you are so hung up on the cost/benefit ratio of creating such an engine, I would suggest taking an in depth look at how much is actually thrown out of a game from all departments during production. I would argue that audio takes up a pretty small fraction of that cost/benefit pie chart. You are looking at probably one audio programmer working with one or two sound designers when creating and implementing such an engine - that's not much when looking at a 40-100 person dev team.

To be sure, the cost of the actually composing, recording, and creating the individual tracks for an adaptive music system would be greater than simply working with a simpler score or slapping licensed music into the game. In order to achieve the amount of material needed, you would probably need to sacrifice using a big name composer and a live orchestra in place of an up-and-coming composer (or do it yourself) using synth and samples. Music budgets are growing for larger projects though, so given the right amount of money and talent, the raw material for such a system could be created.

Of course, not all projects call for a full 100 piece orchestra. Depending on the type of game, the audio director might be better served finding a talented Ska band to produce their tracks. Indeed, not all games would require an adaptive music system to deliver the emotional backbone that music can bring. But for the project that it would be appropriate for, the benefits of increased immersion and deeper emotional connection of the players with their interactions in the game world far outweigh any additional costs such an undertaking might endure. It simply enhances the player's experience which is always something that will sell a game (but perhaps not fit well into a 30 second commercial spot on Adult Swim).

~Michael Kamper

A few peripheral points that haven't been brought up.

1. Austin talks about how the game trailer is an important part of selling a game--and it most definitely is. However, he omits any notion of word-of-mouth promotion by those that have bought and played the game. Good marketing can give you a quick jump out the gate--but it can do nothing to help you finish the race to profitability once word is out that your game sucks. And focusing resources only on areas that are useful for flashy advertising is a great way to ensure that your game sucks.

2. Austin also argues that development money is better spent on graphics, gameplay, story, etc. Unfortunately he fails to see how audio enhances visual perception. (Give a player two identical environments but one has sound and ask which looks better. The answer is the environment with audio.) A quick search of the internet will find many studies confirming the phenomenon. Since audio production is much cheaper than video/visual, the return on investment is significantly higher.

http://smartech.gatech.edu/bitstream/1853/3395/1/99-28.pdf
http://www.physorg.com/news137743564.html
http://www.nature.com/nature/journal/v407/n6806/full/407906a0.html

3. Finally, we come to the role of marketing in video games. It is my firm belief that the marketer's job is to sell whatever the designer deems the coolest/best/most innovative game he or she can make. A good marketer listens to the designer and finds a way to sell the game (even if not all the features are promoted). A bad marketer says, "I don't know how," or "This doesn't fit our formula," or "I can't sell this," and asks for changes, cuts, and/or additions based on what is easy (such as cutting a flashy trailer).

-Dave Steinwedel