Playing by Ear: Using Audio to Create Blind-Accessible Games

Gameplay Considerations

After conquering the navigation problems, making all of the gameplay accessible via an audio-only interface was easy and fun. One of the goals in our game is to collect a set of objects, so we had to figure out how to tell players what they need to collect. We associate a name sound with each type of object. The name sound is just the narrator reciting the name of the object (for example, "chicken" or "water balloon"), so telling players what they need to collect is just a matter of stringing together a narrative introduction ("To complete the whatchamacallit, you'll need . . .") with the name sounds for each item on the list.

We also play the name sound when the player bumps into an object. This improves gameplay for both blind and sighted players, especially for objects that might look or sound unfamiliar. I played Quake 1 for several days before I figured out that the floating blue Q-like thing made me do more damage to enemies. I had never noticed the tiny text message "You got the quad damage" scroll by on the status bar; I would have been clued in more quickly if I had heard "Quad damage!" announced when I ran across it as happens in Quake 2 and 3, instead of a generic beep sound.
We could also use name sounds to implement an audible inventory, reciting the list of objects that the player is holding. However, we've chosen to limit the number of objects a player can hold to just two (one in each hand), so instead we just play the objects' idling sounds once when the inventory key is pressed. We put artificial stereo panning to good use again, playing the left-hand object's sound in the left ear and the right-hand object's sound in the right ear.

We follow a couple of general design principles to ensure our game is fully accessible to blind players. First, we make sure that if two items look different, they must sound different. That isn't usually a problem; most objects in the real world make unique sounds, if they make any sound at all. We just avoid populating our game with items that make no sound.

We also make sure that item or game state changes are accompanied by audio cues. For example, items make a "grabbed" sound when they are picked up. Pick up a chicken and you hear it squawk. While it's in your hand, it will make a disgruntled clucking noise, instead of its normal, "I'm a happy chicken" noise.

Stuff We Haven't Figured Out Yet

As I write this, there are still a few problems that we haven't solved and a few solutions that we haven't tried. The thorniest issue is the up/down, front/back problem.

We are using the simplest possible stereo spatialization algorithm for 3D sound sources, which makes it impossible to distinguish whether a sound source is behind or in front of (or above or below) the listener. Preliminary experimentation with the HRTF (head-related transfer function) algorithms built into DirectX is discouraging - the more complicated algorithms sound better but aren't good enough to tell players whether objects are ahead of or behind them. We are currently experimenting with nonrealistic techniques to indicate the fore/aft position of objects with respect to the listener. Since our game doesn't require unrestricted, up-and-down 3D movement in order to be fun, we're not going to do anything to indicate the up/down position of objects.

Player-generated text, such as player names or text chat, is a problem for which we don't yet have a solution. The standards for accessing text-to-speech functionality under Windows are just emerging, so even though we can assume that all of our blind players have a speech synthesizer for converting text into speech already installed on their systems, we have no way of sending text to the synthesizer to be spoken. We may end up licensing a synthesizer to include with the game, but for now we're simply avoiding features that would require text-to-speech conversion.

We intend to reintroduce doors to the game, because the simple mechanisms of allowing doors to be open or shut and locked or unlocked will add strategic elements and make the game more interesting. When we do, we will probably modify the occlusion algorithm so that closed doors muffle sounds coming through them. That, combined with hallway and room noises, should solve the problems we had earlier with blind players being unable to find the exits in the level. We will still have to figure out how to tell a blind player there is an open door nearby that can be closed or locked.

As mentioned earlier, we are using a quick and dirty hack to approximate true environmental audio. Implementing EAX environmental audio is on our list of things to do, but has been a low priority because we can't assume that our players will have an EAX-capable sound card. We think that supporting EAX will increase the quality of the game, but don't think it will improve accessibility or gameplay.

Better Games for Everybody

Oxo's Good Grips brand kitchen tools were designed for people with arthritis or other joint problems (see www.oxo.com/eyeonoxo for the full story). They've been hugely successful selling them to able-bodied people. I don't have arthritis, but I own their potato peeler, ice cream scoop, and cheese grater. Designing products that work for people with disabilities creates products that work better for everybody. All of the techniques we've used to make Zform games blind-accessible can be applied to any game. None of them makes the game any harder for sighted people to play; on the contrary, most of them either help to reinforce the graphical interface or make the game more interesting and fun.