We immediately gave the prototype to our peers for review; we knew what we had was not perfect, but we certainly did not feel it was fundamentally flawed. Watching people play it -- or fail to play it -- came as a great shock. At this stage there existed an obvious disparity between the team playing it comfortably and enjoying it and the inability of our users to even grasp the basics.
The team had between us designed and implemented the entire system, and so had an intrinsic understanding of its functions; the way and form in which the information is presented, what the information represents, and how it should be interpreted to perform the correct actions and bring about an enjoyable experience. In linguistic terms, we were the ideal speaker-hearer, understanding without exception the meaning of the language -- in this instance a video game interface -- which we had created.
The feedback we received from our test audience was varied and occasionally conflicting; some blamed themselves for failing to understand the gameplay, others claimed tweaks should be made to the tutorial, that the music tempo should be slower, that failure conditions should be less stringent, that they didn't know where to place their fingers next, or that the beat should be better visualized.
We took the feedback from the test subjects on board -- both their verbal feedback, and that which we observed from their actions during play. We built a new prototype, still restrained within the ideals, had new users play it, collected more usability feedback, and repeated the process until we had a refined and much improved version of the initial prototype. At this point, with the help of the tutorial, almost all users were able to pick up and play it.
A refined version of the original prototype, still featuring the move and rhythm bars. Later these were merged in to one.
This success, in retrospect, also indicates a larger failure; over the course of these design refinements, we had partially achieved the ideal of usability through minor adjustments at the expense of some of our other ideals. The screen had become cluttered as new feedback elements crept in, confusing the eyes of the user. Rather than making the game more accessible, we had in fact made it more complex.
We took the game around to publishers and others who we wanted to partner with us to assist with marketing and licensing and found that in less controlled conditions that the game was, rather embarrassingly, not at all accessible; we had created an esoteric system which could only be understood through a lengthy tutorial.
After this realization, we called a meeting to discuss the future of the project and within an hour we had identified the root cause of the problem and produced a solution. We had been incorrectly interpreting the feedback from our tests and addressing the symptoms of a problem rather than the cause. What the users were asking for (the minor changes) and what they really needed (a more prescriptive core mechanic) were very different things.
The solution was irritatingly simple: to rebuild the game as the alternative button-based prototype, and disregard our self-imposed design restrictions -- i.e. the design ideals.
The button system dictates the required actions to the player through the established iconography of the button -- something to press or to touch. Meanwhile, the closing circle suggests timing. These elements were the key to producing the desired interactions between the video game and the user.
Now, the user was not required to learn each move they were expected to use; instead the core mechanics allowed the user to simply follow along. This, in turn, slightly reduced the theatricality of the experience, but greatly increased the accessibility of the product and the enjoyment users experienced.
We started the development of the new prototype from the ground up in a new engine, and took on board our previous mistakes. We now did not impose our ideals on the product, but instead built, tested, interpreted, proposed solutions, rebuilt and retested, with both the prescriptive and implied feedback of the user always leading the design.