Takeshi Shimada builds libraries for Nintendo games. When he was presented with the Brain Age project a few years ago, he and his team faced multiple challenges in finding, developing, and fine-tuning the technologies that would be needed for voice recognition and handwriting recognition.

In a session at the 2007 Game Developers Conference called “Rethinking the Development Timeline,” Shimada (speaking in Japanese with English translation) revealed the precise nature of these challenges and showed how his team dealt with them. He shared Nintendo’s process of how to manage that workload against a ticking clock.

One of the first challenges that came to Shimada from the design team was whether one could play Brain Age while holding the DS sideways, like a book. The technology team had not tested the DS to be handled in this way, and they didn’t know how the recognition technologies would potentially be affected by this twist.

After Shimada assured the designers that the product would not be negatively affected by turning the DS to open horizontally, his team began tackling the much bigger issue of finding engines that would support the other input elements of the game: voice and handwriting.

Handwriting recognition was a major hurdle in the development of Nintendo's Brain Age: Train Your Brain in Minutes a Day

Although finding companies that specialized in this type of technology wasn’t too difficult, choosing which ones to use required much consideration, Shimada said. The task was to find a tool that offered fast recognition speed, had good memory, wasn’t too high in cost, and didn’t need too much heavy processing. If the processing cost was too high, it would have a negative affect on the battery power of the small handheld device.

In early 2004, Shimada and his team began the serious work of tuning the engine. “At that time, we had already decided to release the product in the spring, which would mean we had a mere three months to finish it,” he said.

After deciding on the technology, the team encountered more unexpected challenges. For one, the voice recognition technology had been calibrated to recognize adult voices, and while Nintendo didn’t necessarily want Brain Age to be a children’s game, the company did want to appeal to the largest audience possible, so the game would have to be able to understand a wide range of pitches and tones.

To re-calibrate the engine to recognize children’s voices, Nintendo had to find children to create the input. Shimada said he resorted to asking other company employees to bring in their children so he could record their voices--20 children saying 130 words in all. The recordings had to be completed in both noisy and quiet environments, too, which presented another challenge.

Shimada and his team realized that the places they imagined people would play a Brain Age, like buses, public parks, and schools, weren’t ideal locations to set up recording and development equipment. So to simulate the ambient noise of a public space, they recorded the children directly outside the Nintendo building in Japan.