Non-Game Examples of Adaptive Music

Note that the formal definition of adaptive music does not specifically mention video games. As it turns out, this type of composition does in fact crop up outside of game music. You may be surprised at the history, richness, and breadth of music that falls into this category. Here is a small sampling of a number of non-gaming adaptive music approaches.

Mozart’s Musikalisches Würfelspiel (1792)

Adaptive music more than 200 years old? Yup. In Mozart’s combinatorial “Musical Dice Game”, parts are generated a measure at a time by rolling dice to pick randomly from a table listing multiple potential versions. The number of potential variations of this piece of music “is so large that any waltz you generate with the dice and actually play is almost certainly a waltz never heard before. If you fail to preserve it, it will be a waltz that will probably never be heard again”. ^{[Gardner, 2001]}

Wolfgang Amadeus Mozart - Music Game Innovator

Stockhausen’s Piano Piece XI (1956)

This piece consists of a single page showing nineteen discrete musical sections. The performer is instructed to play the sections in any sequence, as the mood takes her. But the end of each sequence includes specific performance notes regarding how to play whichever section is picked next. ^[Morgan91]

Earle Brown’s Available Forms II (1965)

In Available Forms II, two orchestras rehearse a total of thirty-eight “composed orchestral events”. During performance, two conductors cue these segments “in any combination or sequence”. The conductors may also modify their “ensemble, tempo, and loudness”. ^[Brown65] Brown combined the flexibility of small group improvisation with the awesome expressive power of full orchestral forces.

Synesthesia LLC’s “Interactive Dance Club” installation at SIGGRAPH 98 (1998)

This dance club environment installation allowed participants were invited to experiment with a large variety of non-traditional interfaces – hooked into musical responses designed to fit in “immediately and identifiably” with the DJ’s current club mix. (Actually, an “Experience Jockey” (“EJ”) was responsible for maintaining larger scale structure and direction of this unique audio experience.) Dance club lighting effects with a twist triggered musical phrases when overhead light beams were broken in one “zone”; another featured an array of light sensors on the floor that allowed shadows to influence melodic elements. Trigger pads, proximity sensors, pedals, and other interfaces were used to influence musical activity in other areas. ^[Ulyate02]

…Why are non-gaming examples important?

The really nice thing about many of the non-gaming examples listed in this section is that you can actually study the scores. Most have been published in one or more editions, so you can likely go to your local library and check out the details of their “implementations” for free. And there is a lot of interesting material out there.

Brown, for instance, experimented with a wide variety of indeterminate and “variable form” scoring techniques for many years. A number of other 20th century composers (such as Feldman or Boulez) have also experimented with aleatory music elements. [Morgan91] As a bonus, there also tends to be readily available analyses and discussions of these non-game works in secondary sources. (For instance, try Googling “Mozart ‘Musical Dice Game’”.)

By contrast, the “score” for “X-Wing”’s adaptive music system (for instance) is accessible only via Lucas Arts’ proprietary iMUSE editor. The actual details of the implementation are hidden from the general public; only a handful of people have ever peeked “under the hood”. This opacity is typical of just about any video game adaptive music score.

Further Clarification: Non-Linear Music that is Not Adaptive Music

A look at a two approaches to musical indeterminacy that aren’t adaptive should help to make our definition even clearer.

John Cage’s Music of Changes (1951)

Although the score was originally generated through “chance operations”, ^[Morgan91] the generated score itself is considered to be the definitive form of the piece. There is only one representative performance version of this piece. Only if John Cage’s system for generating this music were actually considered to be the piece would it fit into the category of adaptive music.

Witold Lutoslawski’s aleatoric notation techniques

From the 1960s onwards, Witold Lutoslawski used orchestral score notation techniques that involved indeterminacy; however, his aleatoric techniques were used to describe particular aural effects, which were intended to sound (essentially) the same from performance to performance. ^[Morgan91] This type of effect should not be considered adaptive music.

Questions for Consideration

What, if any, jazz music performance traditions should be considered adaptive music systems? Should classical Northern Indian rag improvisation practices be considered adaptive music?

Is truly interactive music a subset of adaptive music?

Is John Cage’s 4’33” an adaptive music composition?

Other Potential Applications for Adaptive Music

The video game industry is the driving force behind adaptive music technology development right now. However, as adaptive music technology and techniques mature, the craft may well find other important applications. Some for-instances follow.

An Adaptive Symphony

The same kind of structural and expressive variation that is currently possible using computers and synthesized performers is theoretically also possible using a traditional orchestra in performance. The MIDI output of a software adaptive music system under the control of a conductor/composer/improviser figure could fairly easily be displayed as traditional parts on networked laptops for each member of the ensemble. The result would be spontaneous, cohesive, orchestral improvisation. Realization of the composer’s inspiration – as it strikes.

The Democratic Dance-Floor

Club-goers could influence the flow of music by interacting with the DJ and an online adaptive music system using web-enabled cellular devices. At its simplest level, the play-list queue could be displayed on a large screen, with participants merely voting on what track gets played next. With a more involved implementation, any number of performance parameters could be up for grabs: real-time control over the bass line’s resonant filter, dynamic cross-fading control, muting and un-muting of various parts, assigning parts to new instruments, control over builds and breaks, and general ebb and flow within a piece of music. The mood of the music could quite literally follow the mood of the room (or at least of the majority).

Summary

This article proposed a formal definition of adaptive music, and examined it in some detail. The scope of the definition was clarified via a number musical examples that fell in- and out-side of the adaptive music category.

Will the Real Adaptive Music Please Step Forward?

Currently, adaptive music approaches and implementations are characterized by their diversity. At first glance, the examples in this article tend to have more that sets them apart than groups them together. And none on their own could really be considered representative.

But the single essential question still remains: “how can a game composer score a scene intelligently and compellingly, when she doesn’t know what is going to happen, when?” The technologies and techniques that provide general, implementation-unspecific solutions to this challenge are the ones that will consistently find traction and evolve the craft as a whole.

And “what exactly are we talking about?” seems like a good place to start.

References

[3DSoundSurge01] “3DSoundSurge Press Release: Monolith’s 3D Engine to Feature the Interactive Power of Microsoft’s DirectMusic™”, (3DSoundSurge, 2001), online:

3D SoundSurge (accessed: 10 December 2002).

[Boyd06] Andrew Boyd and Robb Mills, “Implementing an Adaptive, Live Orchestral Soundtrack”, lecture at the Game Developers Conference, San Francisco, 2006.

[Brown65] Earle Brown, “Introductory Remarks” to Available Forms 2 for Large Orchestra Four Hands (98 Players), (New York: Associated Music Publishers, Inc., 1965), 1.

[Doud03] Chuck Doud, “Composing, Producing and Implementing an Interactive Music Soundtrack for a Video Game”, lecture at the Game Developers Conference, San Jose, 2003.

[Gardner01] Martin Gardner, The Colossal Book of Mathematics: Classic Puzzles, Paradoxes, and Problems: Number Theory, Algebra, Geometry, Probability, Topology, Game Theory, Infinity, and Other Topics of Recreational Mathematics, (New York: W.W. Norton and Company, Inc., 2001), 632.

[Morgan91] Robert P. Morgan, Twentieth-Century Music: A History of Musical Style in Modern Europe and America, (New York: W. W. Norton & Company, Inc., 1991).

[Ulyate02] R. Ulyate and D. Bianciardi, “The Interactive Dance Club: Avoiding Chaos in a Multi-Participant Environment,” Computer Music Journal, Volume 26, Number 3 (Boston: Massachussetts Institute of Technology, 2002).