Playtesting Mobile Games at the DMV
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.
We can't afford to make a game that isn't fun, which is why we've been playtesting Patchmania since the first month of development. Playtesting helped me focus the design of our puzzles, gauge the level of difficulty, validate the controls, iterate on the UI, and even develop our story and characters.
There are a number of ways you can playtest a game, but we think we came up with a new one – testing your game at the Department of Motor Vehicles (DMV)! The DMV is an ideal place to playtest a mobile game: on any given day, there are hundreds of people, of all ages, from all walks of life, stuck in one place for hours, who will do just about anything to alleviate their boredom. There are also no tables in the waiting area, so people don’t bring their laptops, and aren’t able to get any useful work done.
At one point in the early days of Patchmania’s development, I spent an entire week at the DMV! I would arrive early in the morning with a backpack. In my kit were several phones and tablets loaded with the latest build of the game, a charger, some snacks, a clipboard and notebook, some multi-choice surveys, a little printed sign, and large brown envelope to hold the survey results. Then I would simply find an empty seat among a large group of promising potential testers.
At first it was slow going – starting up conversations with complete strangers doesn’t come naturally to me, and it was occasionally awkward to explain what I was doing there. However, I quickly began to get creative to overcome my own boredom. Overall, I found most people were receptive and friendly to playing games while they wait, and through trial and error I learned what worked when approaching people. Rather than limiting myself to the people sitting immediately adjacent to me, I would frequently move seats, and used a small handheld sign and eye contact to get people’s attention. The most successful variant of my sign read “Bored? Play my puzzle game prototype!” in big block capitals. To my surprise, people would often get up and move seats – unprompted – just to get a chance to try out the game!
I also experimented with different signs and incentives. At one point, in an attempt to get players even faster, I had a sign that offered people $5 for playing the prototype. I worried that the financial incentive might taint the data, but to my surprise, most people didn’t actually want the money, and would actually refuse it afterwards – particularly if they had a good time. It turns out that the offer of $5 just gave people an additional excuse to break the ice and try the game out, even if they didn’t really intend to collect.
Once I got the hang of it, I would get 20 playtests a day, 100 in a week. This might not sound like much compared to driving thousands of people to play a prototype via Facebook ads and collecting data remotely, but for some kinds of learning – like figuring out if your game concept is fun (and if it’s not, why not!) – I believe there’s no substitute for actually watching someone experience your product, and talking to them about it. You need that unfiltered and brutally honest feedback from a complete stranger, and you can’t usually get this from friends/family/colleagues, or remotely over the Internet.
Here’s how a typical DMV play session would go:
1-2 minutes: introduce myself, explain why I’m doing playtesting and what I’m looking to get out of it. In our case, it was important to explain that my prototype had no sound, temp graphics, and that it was all very unfinished. I also found it sometimes helped to tell a white lie and say that it wasn’t my game, and that their feedback wouldn’t hurt my feelings. After getting that out of the way, it was easier for players to focus on whether it was fun rather than being distracted by the lack of presentation.
- 20-25 minutes: let them play the game uninterrupted. Give them the phone/tablet, resist the temptation to touch the screen if you see them struggling, and don’t take it away until the test is over. I also used this time to chat about what other games they play, take notes, and to probe their understanding of the game whenever I saw them get stuck or confused.
- 3 minutes: ask them to fill out a brief one-page multi-choice survey anonymously, and to slip it into a brown envelope (I would include a bunch of dummy blank responses in there for the first person of the day to increase their feeling of anonymity). A well designed survey will yield quantitative data that you can track over time to see if you are improving.
- 5-10 minutes: debrief. Ask them about their first impression. Find out what they liked and didn’t like. If you’re focusing your testing on improving a particular area, tailor your questions accordingly.
It’s worth noting that some of the most useful feedback came from unexpected sources. I selected players that I could see already had modern smartphones – I didn’t want familiarity with touchscreens to be a factor in the results – but other than that, I didn’t discriminate. Some of the most insightful feedback came from construction workers & bikers – who despite their overt manliness weren’t uncomfortable playing a game about cute little bunnies!
So there you have it – if you’re part of a small team and working on a mobile game (or app), playtesting at the DMV can be very productive. It helped me iterate to something fun quickly and cheaply, and built confidence in what I was making. It also forced me to get out there and meet my potential customers very early, to learn all I could about them. The data I gathered also proved convincing when approaching potential collaborators to work with me later on. Finally, while the DMV certainly doesn’t condone this practice, most of the time I went completely unnoticed by their staff, and I found that if I was respectful and didn’t hassle anyone, they were happy to let me entertain people while they wait.
Patchmania is an adorable free puzzle game that launches worldwide for iOS on March 19th, 2015. To learn more, visit our website. Here's a screenshot from the finished game: