The non-diegetic side of the framework

Interface

The first category of the non-diegetic side of the IEZA framework, Interface, consists of sound that represents sound sources outside of the fictional game world. Sound of the Interface category expresses activity in the non-diegetic part of the game environment, such as player activity and game events. In many games Interface contains sounds related to the HUD (Heads Up Display) such as sounds synced to health and status bars, pop-up menus and the score display.

Sound of the Interface category often distinguishes itself from sound belonging to the diegetic part of the game (Effect and Zone) because of interface sound design conventions: ICT-like sound design using iconic and non-iconic signs. This is because many elements of this part of the game environment have no equivalent sound source in real life. Many games intentionally blur the boundaries of Interface and Effect by mimicking the diegetic concept. In Tony Hawk's Pro Skater 4, Interface sound instances consist of the skidding, grinding and sliding sounds of skateboards. Designers choose to project properties of the game world onto the sound design of Interface, but there is no real (functional) relation with the game world.

Affect

The second category of the non-diegetic side of the framework, Affect, consists of sound that is linked to the non-diegetic part of the game environment and specifically that part that expresses the non-diegetic setting of the game. Examples include orchestral music in an adventure game and horror sound effects in a survival horror game. The main difference between Interface and Affect is that the Interface category provides information of player activity and events triggered by the game in the non-diegetic part of the game environment, while the Affect category expresses the setting of the non-diegetic part of the game environment.

The Affect category is a very powerful tool for designers to add or enlarge social, cultural and emotional references to a game. For instance, the music in Tony Hawk's Pro Skater 4 clearly refers to a specific subculture and is meant to appeal to the target audience of this game. The Affect category often features affects of sub-cultures found in modern popular music, but the affects of other media are also found in many games. Because most players are familiar with media such as film and popular music it is a very effective way to include the intrinsic value of the affects.

The second dimension of the framework

As we have seen, the first dimension distinguishes categories belonging to the game world (diegetic) and those who are not belonging to the game world (non-diegetic). But there also is a second dimension. The right side of the IEZA framework (Interface and Effect) contains categories that convey information about the activity of the game, while the left side (Zone and Affect) contains categories that convey information about the setting of the game.

Many games are designed in such a way that the setting is somehow related to the activity, for example, by gradually changing the contents of Zone and Affect according to parameters such as level of threat and success rate, which are controlled by the game activity. We also gain an insight concerning the responsiveness of game audio: only the right side of the framework contains sound that can be directly triggered by the players themselves.

Summary

• The IEZA framework defines the structure of game audio as consisting of two dimensions. The first dimension describes a division in the origin of game audio. The second dimension describes a division in the expression of game audio.

• The IEZA framework divides the game environment (and the sound it emits) into diegetic (Effect and Zone) and non-diegetic (Interface and Affect).

• The IEZA framework divides the expression of game audio into activity (Interface and Effect) and setting (Zone and Affect) of the game.

• The Interface category expresses the activity in the non-diegetic part of the game environment. In many games of today this is sound that is synced with activity in the HUD, either as a response to player activity or as a response to game activity.

• The Effect category expresses the activity in the diegetic part of the game. Sound is often synced to events in the game world, either triggered by the player or by the game itself. However, activity in the diegetic part of the game can also include sound streams, such as the sound of a continuously burning fire.

• The Zone category expresses the setting (for example the geographical or topological setting) of the diegetic part of the game environment. In many games of today, Zone is often designed in such a way (using real time adaptation) that it reflects the consequences of game play on a game's world.

• The Affect category expresses the setting (for example the emotional, social and/or cultural setting) of the non-diegetic part of the game environment. Affect is often designed in such a way (using real time adaptation) that it reflects the emotional status of the game or that it anticipates upcoming events in the game.

4. Discussions and future work

In this article we have described the fundamentals of the IEZA framework, which we developed between 2005 and 2007. The framework has been used at the Utrecht School of the Arts (in the Netherlands) for three consecutive years as an alternative tool to teach game audio to game design students and audio design students. For two successive years we gave our students the assignment to design a simple audio game⁵.

The framework was only presented to the students of the second year as a design method. We found that the audio games developed in the second year featured richer sound design (more worlds and diversity), better understandable sounds (for instance, the students made a clear separation between Interface and Effect) and more innovative game design (games based on audio instead of game concepts based on visual game design). The students indicated that the framework offered them a better understanding of the structure of game audio and that this helped them conceptualize their audio game designs.

The framework offers many avenues for further exploration. For instance, it is interesting to look at the properties of and the relationships between the different categories. An example of this is the observation that both Effect and Zone in essence share an acoustic space (with similar properties and behavior), as opposed to Interface and Affect, which share a different (often non-) acoustic space⁶.

In many multiplayer games, it is only the acoustic space of Effect and Zone that is shared in real time by players. Such observations can not only be valuable for a game sound designer, but also for a developer of a game audio engine. It is also relevant for designers incorporating player sound in games, because whether or not the player sound is processed with diegetic properties, defines how players perceive the origin of the sound.

An insight we discovered while designing with the framework is that the right side (Interface and Effect) is more suited to convey specific game information such as data and statistics, whereas the left side (Zone and Affect) is more suited to convey game information such as the feel of the game.

The IEZA framework is intended as a vocabulary and a tool for game audio design. By distinguishing different categories, each with specific properties or characteristics, insight is gained in the mechanics of game audio. We believe the IEZA framework provides a useful typology for game audio from which future research and discussions can benefit.

⁵ Games consisting only of sound. For more information please visit: www.audiogames.net.

⁶ Interface is the part of game audio that is the most likely to remind the player that he or she is playing a game. Therefore designers often add diegetic properties such as acoustics or similar sound design to make the interface category less intrusive.

References

Chion, M. (1994). Audiovision, Sound on Screen. New York: Columbia University

Press.

Collins, K. (2007). "An Introduction to the Participatory and Non-Linear Aspects of

Video Games Audio." Eds. Stan Hawkins and John Richardson. Essays on Sound

and Vision. Helsinki : Helsinki University Press. (Forthcoming)

Folmann, T. (2004). Dimensions of Game Audio. SHHHHHHHHHHHHH audio blog.

Retrieved November 5, 2004 from:

http://www.itu.dk/people/folmann/2004/11/dimensions-of-game-audio.html

Friberg, J. and Gardenfors, D. (2004). Audio games - New perspectives on game

audio. Paper presented at the ACE conference in Singapore, June 2004. Retrieved

August 2004 from:

http://www.sitrec.kth.se/bildbank/pdf/G%E4rdenfors_Friberg_ACE2004.pdf

Leeuwen, T. van. (1999, October). Speech, Music, Sound. London: Macpress.

Salen, K. and Zimmerman, E. (2004). Rules of Play: Game Design Fundamentals.

Cambridge, Massachusetts: MIT Press.

Stockburger, A. (2003). The game environment from an auditive perspective. In:

Copier, M. and Raessens, J. Level Up, Digital Games Research Conference (PDF on

CD-ROM). Utrecht, The Netherlands: Faculty of Arts, Utrecht University.

Weis, E and Belton J. (1985). Film Sound: theory and practice. New York: Columbia

University Press.

Games:

Grand Theft Auto III (PC). Rockstar Games. Released May 21, 2002.

FIFA 07 (PC). EA Canada, EA Sports. Released September 2006.

Tony Hawk's Pro Skater 4 (PC). Aspyr, Beenox. Released Aug 27, 2003.

New Super Mario Bros (Nintendo DS). Nintendo.Released May 15, 2006.

Tetris (Nintendo Game Boy). Nintendo. Released June 14, 1989.

Rez (PlayStation 2). United Game Artists, Sony Computer Entertainment. Released November 21, 2001.