Adaptive Music

What is music?
In a freshman "Introduction to Ethnomusicology" course I attended at the University of Toronto, the entire first class was dedicated to attempting to agree on a definition of "music." Easier said than done! Luckily, since this paper is a solo effort, I get to pick the definition without argument. Music: structured patterns of sound in time.

What's in a name?
In many ways that was a terrible definition. For one thing, it relies heavily on the fact that you already know what music is. (The word "music" is a symbol with which you already have years of personal experience.) Really, its sole purpose is to focus our attention on some fundamental elements of music, without getting distracted with sticky issues like art, creativity, culture, human involvement, and intent.

Sound Structures in Music
Part of the reason that music is so appealing is the number of levels of patterns that are going on at once. Sound comes from very fast air pressure changes at rates of roughly 60 to 23000 times a second. Phrases involve the arrangement of these sounds on a much different temporal scale. The sounds become melody and rhythm, and harmonic motion. Structure is a higher level organization of phrases. (E.g. classical Sonata form, or the ordering of verses and choruses in a pop song.) Very different parts of the brain process and appreciate these different elements simultaneously when you are listening to music.

Equivalent structures in language
A non-musical structural equivalent of a sound would be a word in language, e.g. the word "sound". The equivalent of tone (the quality or "color" of a sound) might be font. This sentence is a good example of a phrase. This paragraph could be thought of as a substructure, and the essay as a whole as a much larger architectural structure or form.

Structures and Adaptability in Language

This article explores adaptive art in the context of language. This is because language is much easier to demonstrate and discuss in the written medium than music is! So, despite the title "Adaptive Music," this article doesn't talk about music at all - except by analogy.

An Adaptive Poem
I used to write video game music for a living. It was a very exciting job, but it also had its share of frustrations. For instance, part way through one particular project, I discovered that the lead programmer hadn't bothered installing a sound card in his new computer. As one might expect, this was making it kind of hard for him to test the interactive score I was writing. (True story! The next part is made up though.)

I decided that it might be a good idea for me to write some interactive poetry for him to work with temporarily. (We could even keep it as an option in the release version of the game, for the hearing impaired.) Like the music, the poetry would change in real time in order to reflect the changing action of the game and to enhance the atmosphere. It would be of extremely high quality, and interesting enough that the player wouldn't get sick of it after a mere eight hours of game-play.

An adaptive poem?!!
Writing adaptive poetry sounds impossible, doesn't it? Well, it's essentially the same challenge that an adaptive music composer faces… but more on that later. For now, let's try to prototype an adaptive poem and see what happens. The first step will be nailing down the tone and style, and what we want the piece to communicate. The following example is a linear poem that will help us figure out what we want the interactive version to do. (The game, by the way, is called Seal Hunter and features the heroic Captain Brave-O.)

Example: Linear Poetry

The cold dawn breaks on the arctic ice
Gleaming hard on a cruel device:
Captain Brave-O's clubbing hook
Dripping blood from the lives it took

The evil seals are everywhere
Breathing our precious human air
Soon the air will all be gone
And no more human breaths be drawn

Then: BAM BOOM BANG, KER-CHUNK, KER-SPLAT!
The hook is WIELDED like a BAT
CAPTAIN BRAVE-O's in the FRAY,
HORDES of EVIL SEALS to SLAY!

His breathing slows, the battle done
This war, he knows, is far from won
And though there's not a seal in sight
It's far too quiet to be all right

Adaptability at the word level
The action and mood of Seal Hunter can change quite suddenly. Captain Brave-O could be walking around being brave and heroic one minute, and then diabolical seals could leap out at him the next! Ideally, our accompanying poem should be just as dynamic. Unfortunately, since the game player will be controlling Captain Brave-O's actions, we have no way of knowing in advance what is going to happen, when. One way to try to deal with this is to compose a poem that is flexible at the word level, so that it can change moods at any time.

The next example randomly selects words from one of four word lists. Each list contains only words that are appropriate to a particular game mood. When the action of the game changes, the source word list is also changed, thereby changing the mood of the poem on the fly.

Example: Adaptive Word Texture

(The level begins as Captain Brave-O enters heroically. Poem mood: HEROIC) Captain Brave-O brave hero marching gleaming brave hero Captain brave adventure marching hook Brave-O brave brave hero (The landscape changes; seals are nearby. Switch poem to CREEPY) danger fear shadows lurking seal evil danger caution scary (Seal surprise attack! Switch poem to COMBAT) bam ker-chunk ker-splat hook bat fray slay bang ker-splat bat slay slay ker-chunk bam ker-splat slay slay slay (The last seal is subdued but the territory is still scary. Also, Captain Brave-O's health is low. Switch to CREEPY, INJURED combo) injured hurt danger fear injured lurking seal evil danger caution shadows

That was surprisingly effective and very dynamic; however, aesthetically it might be considered somewhat lacking. It was more like word texture than traditional poetry -- there was no real attention to rhyme or meter. Let's see what happens when we try a less random ordering of words in an attempt to address this:

Example: Adaptive Word Texture with Meter and Rhyme

(The level begins: HEROIC)
mighty gleaming courage gleaming
roaming courage hero marching
Brave-O mighty Captain marching
Gleaming courage courage roaming
Mighty courage Brave-O Brave-O
Gleaming mighty roaming hero
Courage Captain gleaming marching
Hero mighty --
(Surprise seal attack! COMBAT)
fray seal bam bat
slay slam spray splat
hook fray squeal slam
bat seal slay bam
fray slay fray seal
splat…
(The battle ends.)

The second example was essentially the same as the first, except for two small changes. This time, the word list for a particular mood consisted only of words sharing the same foot. Additionally, I added special lists of rhyming pairs. Every fourth word was selected from this list, in order to form rhyming couplets. The result was a kind of pseudo phrase structure (but without any real sentence-level meaning).

This more structured version came at a slight cost. Notice that when the sudden seal attack happened, it broke the rhyme and rhythm scheme of Brave-O's strut poem. If we'd waited for the line to finish, the attack poetry would have been late and much of the effect would have been lost. Having to make this kind of compromise makes this solution less dynamic than pure word texture. On the other hand, patterns of rhyme and meter do have an undeniable attraction.