My Message close
GAME JOBS
Latest Jobs
spacer View All     Post a Job     RSS spacer
 
May 18, 2013
 
Sony Computer Entertainment America LLC
Sr. Network Systems Engineer
 
Amazon Game Studios
Sr. Game Designer
 
Amazon Game Studios
Quality Assurance Manager
 
Treyarch / Activision
Technical Animator
 
Amazon Game Studios
Game Development Engineer
 
Amazon Game Studios
Game Graphics Engineer
spacer
Blogs

  Playtesting in Public
by Tadhg Kelly on 06/24/09 12:18:00 pm
2 comments Share on Twitter Share on Facebook RSS
 
 
The following blog was, unless otherwise noted, independently written by a member of Gamasutra's game development community. The thoughts and opinions expressed here are not necessarily those of Gamasutra or its parent company.

Want to write your own blog post on Gamasutra? It's easy! Click here to get started. Your post could be featured on Gamasutra's home page, right alongside our award-winning articles and news stories.
 

We're about to do something scary. We (me and my company that is) are taking the step of playtesting the core of our game to see if it's fun. Like any game in development there comes a time when the designs all fall away and good intentions are left by the roadside, and all that remains is what you have. Is it any good, is it not?

The main difference is we're doing our playtesting in public.

Simple Lifeforms is a social game company. We make games that work on the web through a social network like Facebook or Myspace. We primarily develop in web technologies like PHP or MySQL, and our game is hosted like any other web site or application.

For some developers that choice of platform is incredibly limiting as PHP and Flash are not exactly swish as game engines go. Maybe not. On the other hand, for us it's incredibly liberating.

As a social game designer, I can do something that most developers really can't: I can get my game in front of players today, hear their opinions today and fix their problems today. Obviously not for all cases in all instances, but that's the principle. Change game rules, add a virtual good, fix a market, balance a spell, I can do it all today. And they in turn can tell me what's wrong. Today.

So my game may not be a Halo killer but it is also not stuck in the mire of retail-politik. It's not crunching endlessly for milestone after milestone with only internal subjective quality assurance for guidance. It's not doomed to the artificial half-light of focus groups. I can just ask my customers what they think and then can quickly respond. What that really means is that I can build the game from the ground up.

This has been a surprisingly hard lesson to learn. I came up with a full design for Spell Souls in February. I wrote a wiki, wireframed schematics, created spreadsheets with various quantities and attributes like a regular designer might, but we quickly realised somewhere around mid-April that this was a big mistake. Details were not read, feature confusion crept in quickly, wireframes were not fully played, and nobody understood the game but me.

The reason this was (and is) a mistake is I was thinking in terms of design-develop-tweak. Design the game, make the prototype and content, and then tweak. That's how it worked on every game I've worked on in the last 8 years, but oh so wrong.  That's how retail games work in the old release-and-patch model. We realised that Spell Souls is not a retail game, it's a service game, and that makes the whole approach different. I read a lot of good advice, from blogs such as Startup Lessons Learned, and we talked about the importance of getting the game in front of players as fast as possible. Spell Souls is attempting something a bit unusual, so how can we know that the core game mechanic works?

We certainly can't afford to just sit on it for 9 months and continually prototype. So I worked to scale things back. And then I did it again. And again. Scope revision is normally a final activity and though a team tells itself that cut levels, enemies or content will go back into the game if there's time, it very rarely ever happens. Gone is usually gone. With Spell Souls, I could cut to get at the core of the game to playtest it, but gone is not gone.

My game does not have to be fixed for features like a retail game. I'm not selling something in a box or a downloadable game on Xbox Live. I have no long lead times to worry about, not approvals and no corporate structure requiring buy-in. Web applications tend to settle on their core functionality after some iteration but they can always change. Facebook recently changed its whole front page interface and features to focus on new priorities, so why can't I change Spell Souls to do likewise? The only reason is if customers don't want that.

So a lot of our expanded features are sitting on a roadmap while we try and get the core right first. This is something we can not only afford to do, it is something that we should do. After all, what if the core of the game sucks? What if it's great but players like it for a different reason than we assume? What if it's partially good but there's one very obvious thing missing? All of these are huge questions that any developer has to, at some point, ask of themselves.

The main difference for us is we get to ask you too, and I'm pretty sure that you'll be much more brutally honest than we would ever be with ourselves. As I say, it's scary stuff.

If you want to join the Spell Souls playtest, join the Group on Facebook.

(@tadhgk)

 

 
 
Comments

Jason Schklar
profile image
Great post. A couple of thoughts I have:



1. There are other ways to iterate with user feedback *in real time* in order to make game improvements before release. I (and others) do rapid iterative usability testing with game developers and publishers where we iterate on code and content in order to identify issues, experiment with solutions, and validate fixes. This means that we all watch people playing the game together, discuss issues that we see in real time, get "workers" (coders and content providers) to try out fixes, and test again with those fixes. Once you have a workflow in place, you can be putting new builds in front of new players fairly rapidly (like multiple times per day). This kind of approach is more appropriate with games-as-products (as you allude to above) because they can't just let a huge portion of their user base experience the game for free.



2. You can learn different things using different methods -- and you can use different methods depending on where you are in the ship cycle. Focusing intently on a few users (ideally one at a time) will allow you to probe deeper into a product in a more controlled situation. You can generate interesting directions and research questions that you can "test" later on in a much larger release (and test multiple variants if you're not sure what direction to take via A/B testing on subpopulations of your community). You can also save your community folks a bunch of pain (and yourself a bunch of grief) if you do N=1 (or N=5) usability testing early on to catch serious blocking issues that prevent large numbers of people from discovering the core fun of the game. Why have 2,000 people complain about a confusing menu option that causes them to lose their save games when you could have fixed it beforehand?



Don't get me wrong, I love the loosey-gooosey approach you talk about where you don't have to wait for some "corporate structure requiring buy-in". But you can also take that approach with smaller usability tests. I help my clients build quick and dirty observation labs where the development team can watch people struggle, fail, and try to co-opt/use the game in cool ways no one ever thought they would have before. Really, all that's needed is a good source of newbie participants, a set of specific user experience goals, a speakerphone and a few video cables. And, of course, some elbow grease :)



Thanks for the post -- please keep 'em coming.



Cheers,



J

Tadhg Kelly
profile image
Hey Jason,



Sure, all true. Whatever works. :)



T


none
 
Comment:
 




 
UBM Tech