Editor's Note: This article is the second in a
series of interviews and round-ups from the GDC Europe, which was held
August 27-29 in London, UK. In this second dispatch, John McLean-Foreman
interviews another three developers who spoke at the event: Bill Fulton,
Zeno Colaco and Harvey Smith.
Bill Fulton was a founder of the User-Testing
Group for Microsoft Games, which uses psychological research methods to
get feedback that improves the usability and fun of games published by
Microsoft. Since 1998, the group has tested 12,000+ gamers playing 80+
different games, including most Microsoft titles like Age
of Kings, Combat
Flight Sim, Halo
and Rallisport Challenge.
Prior to working at Microsoft, Bill did four years of post-graduate training
in cognitive & quantitative psychology at the University of Washington,
studying how people form judgments and make decisions, and how to meaningfully
quantify theoretical ideas.
What does "usability" mean to you?
Usability work is essentially an attempt to see how
consumers respond to the game prior to you [the developer] finishing the
game, and buying the game and, , so you have a chance to revise prior
to shipping. The basic ideas you want from the productivity world is that
you want your version 1 to really feel like a version 2.
We use two major research methods to get information
from people. The devil is completely in the details. It's all about how
you get the right information from the right people at the right time.
So, something comes online, and you have the time and schedule to actually
fix it. You figure out who that segment of the game is being targeted
towards. In the case of a tutorial, you would bring novices to the game,
not experts of the genre, because you're not trying to teach experts.
If, however, you were dealing with an advanced feature of the game, you'd
bring in experts because they're the people that are going to find the
advanced features and are going to want them to be fun.
Why would a development team need psychologists?
Why not have the designers watching the game testers instead?
For the same reason that you wouldn't hire me to
go and do your art for you. Doing research with humans is extremely tricky.
Anyone who as ever filled out a questionnaire and seen a dumb question,
or seen through to what the answer is desired to be, knows how easy and
transparent it is to figure out what people are trying to get at. Often,
if they know what you're trying to get at, they will either intentionally
tell you what you want to hear (I'm sure that anyone who has been in a
focus group has seen that), or sometimes they will purposefully tell you
what you don't want to hear to antagonise you. You don't always know which
is which. So, psychology has worked out a lot of methods on how you get
information from people, how people tend to respond in research situations,
how to know when they are suspicious or when they are going to give you
incorrect answers and what you can do to minimize it. A risk is being
run that people are going to behave in a way that doesn't reveal their
true feelings and thoughts.
Some companies can't afford to set up a psychology
department. Is there a book that developers can study so that they can
get more accurate results from their own tests?
That's a very difficult one -- the books that I know
of are designed for academics, not necessarily for lay people. I think
that the best introduction to thinking about the effects of design, as
opposed to game usability (for which there are not books yet), is Donald
Norman's The Design of Everyday Things. Basically it's a book that
looks at bad designs in common tools, like the telephone, light switches,
and teapots. The people who designed them failed to think about the user's
tasks and what the user's decisions are. My favorite example is doors.
They're terrible. Half the time you don't know whether you should push
or pull a door. You sometimes don't know which side the hinges are on
when they give you the long bar across the middle. It's because the people
designing those doors didn't think about how people would really use them.
This book is more for planting the seed of thought, "How am I doing
that with my design? How am I failing to remember the kind of people that
I'm trying to suit?"
Another book on the subject is be Jakob Nielsen's
Usability Engineering. He's considered one of the biggest figures
in the field, but he's very academically oriented and very productivity
oriented. I don't think that there are good texts for games specifically.
One of my colleagues wrote a chapter for a usability
handbook, which is basically what we consider the opening shot to the
game usability field. He's written a methodology of how and why it's different,
how it's interesting and important, and you can access it on our website:
We're trying to start the usability field into thinking
Do you feel that usability studies will make a
better game and increase its sales?
Do I believe it makes a better game? Absolutely.
We can show people unable to do something in one case, like in the A-1
level [which was supposed represent enemy difficulty levels and read "A.I.",
but testers thought it said A-1 and didn't know what that meant], and
all people succeeding after the design is fixed. We have very good evidence
of how our changes actually affect consumers more so than virtually any
As far as tracking sales, that's the perpetual question.
I don't have a good answer, but we are beginning to examine it now. We
now have enough data and bandwidth to look at the relationship, but quite
frankly I think that the correlation is going to be muddled at best. The
only thing I can say is that I find it interesting that people ask us
to justify usability because they view it as an additional cost. But how
do you determine how much improvement your product will realize by adding
an artist, or another tester? No discipline knows how adding one more
person or doing something slightly differently is really going to affect
the bottom line. You use your best intuition, you use your judgement and
you say, "I believe that adds value."
After your speech I overheard someone saying
that through usability you're not designing something new and creative,
but are in fact designing games for the "lowest common denominator
I think anyone who describes a consumer as
an idiot should perhaps not be in the games publishing business. They
should make games that they enjoy playing, and there's nothing wrong with
that. Lots of people take up hobbies that they enjoy. But, if you're in
the business of making games for other people, then one ought to attempt
to find out what other people like. To me, it's very basic. Even if you
throw away the profit motive, as an artist you want lots of people to
understand and enjoy your work. Unless you believe they're too stupid
to understand, say, modern art -- then therefore modern artists are making
art for modern artists, and they don't care. As long as that's fine for
them then that's great. If they don't make something that can appeal to
someone like me who is uneducated, then I will be disinterested in their
work. I assume that most people's egos would prefer a million people to
love their game, to have said, "I spent 50 bucks and I'm proud to
have done so because it was fun." as opposed to, "Oh… those
are for other people."
In his current role as Vice President,
Publisher & Developer Relations at SCEE, Colaço is responsible
for the day-to-day management of SCEE's European licensee community that
has over 250 active developers and 40 publishers on the PS2 alone. This
includes licensee relations, product planning and the development of strategic
partnerships. Colaço and his team are a focal point for emerging
talent and new businesses looking to break into interactive games.
What was your talk about?
My talk tried to give some guidance to developers
about what I think publishers want in the next big games, and tried to
coach developers on how to pitch their title to publishers. Granted, that's
slightly outside my general remit, being Vice President of Sony's third
party division, [which] really looks after the platform edge.
How often do publishers try to look beyond the
presentation to see what value is underneath?
We at Sony don't believe that we're necessarily
the best ones to tell people how to write games. The talent and the creativity
are the people themselves, and what they want to bring to the table. I
think it is important and necessary to treat game development as a business.
Whether it's the developers themselves employing someone who's got some
exposure to that [part of the industry] to represent them, or even a third
party, it's becoming more important. Tried and tested methods are there.
It's about hooking up to one of those and trying to deliver it.
A lot of publishers are risk averse and as a result,
they're putting the same game on the shelves again and again. Doesn't
it makes sense to take risks at least part of the time?
I completely agree. As you probably saw in the panel,
I felt it was something that needed to be said. Risk aversion is one thing,
but "no new I.P.s" (Intellectual Properties) is going to lead
to the death of great creativity. One thing that we do as a platform,
and I'm sure publishers are looking at it as well, is that we want to
try and help risky products. Something that pushes the genres on our platform,
we tend to try and give a better voice to. We support many of these titles
when they are coming out, sometimes even with a direct marketing contribution
from the platform to the publisher involved.
I think developers should try to use and leverage
the platform companies in the process of pitching. For instance, perhaps
you're working on a title that is slightly different and is pushing a
particular genre -- say, it's a music product. That is a smaller genre,
but it might expand the platform's capabilities for some of the consumers.
Clearly a music product is probably not going to do the same sales as
a football product, but if that product were brought as a concept to my
team, they would be happy to give feedback on how to maximize and broaden
that genre. That meeting in turn could be used as leverage with a publisher
who could get into early discussions with the platform company to say,
"If we brought this to your platform, would you support us? Would
you give us exposure in your catalogues, or maybe in your in-machine demo?"
Or even, as I said, to request support for the marketing costs.
It's almost like incubating creativity in that way.
But the developer has to [take the initiative] in the first place. The
developer has to show some sort of originality to both platform [and publisher].
I think sometimes developers go straight to the publisher and miss out
on the platform holder, and they then lose any leverage that the platform
holder might have been able to give them.
Harvey Smith is the project director
on Ion Storm's DX2: Invisible War.
He held the lead designer position on Deus
Ex. Prior to that he worked as lead
designer of FireTeam
(an Internet squad game developed by Multitude, Inc.), and at Origin Systems,
as a designer, associate producer and quality assurance tester on such
games as System Shock,
and the CD verson of Ultima VIII:
What was your talk about
at the GDCE 2002?
My talk was about what
I call "systemic level design". After working on Deus Ex
and seeing the contrast in the two different ways that we worked on levels
(one was more systemic, one was more special case), it drove me to want
to talk about when one was the right answer versus when the other one
was the right answer.
How do you define "emergent
We talk a lot about game
interactions on a class-to-class basis. When the player does one thing
in the environment, and it causes an interaction between two other game
elements that provide a second order of consequence, that maybe the development
team didn't predict. [The game] behaves in a rational way that surprises
the player. We call that emergent gameplay.
The example I used in my
speech was an enemy unit that explodes when it dies, and a bunch of containers
that the player could open with explosive resources or lock picking resources.
Players figured out that if they lead that enemy unit over to those containers
before delivering the deathblow, that the resulting explosion would open
the container. We call that emergent gameplay.
In your session you
mentioned that Deus Ex players created ladders using
the removable proximity grenades and climbed off the map. How do you then
make the player finish the level and go where they're supposed to?
Ultimately our game is
still a combination of special case moments and systemic moments. We still
say, "Get to the exit" or, "Go talk to this guy" or,
"Go do X". How you do accomplish those goals is more or less
up to you, however. We reward you for goal completion, not goal completion
Will designing levels
like this give players an experience they've never had before?
The sense of possibility
that players talk about when they play Grand Theft Auto 3 -- or
any game that tries to make a really in-depth environment where the world
is not just full of one-to-one interactions that were prescripted -- is
a very powerful thing. It's empowering to the player. The gameplay itself
becomes largely about exploring the possibility space. So yeah, we think
it's pretty innovative. As game worlds get more complex, keeping this
angle going will mean that every game experience will be richer and richer.