|
When I was at SEGA, I sat in a "release" meeting with my boss, the
head of all of development and the QA manager. I listened as the head
of development lectured me on the strength of our titles and how we
have set the bar very high and so we can't possibly release a title
with bugs.
His Director of Development took over and
pontificated on how he and his boss have years of experience and know
when a game is ready for the market and would be happy to teach me
these skills if I was willing to listen.
Who'd say no? I said something that sounded like: "Yeah sure, can you sign the release papers now?"
They
both looked at the QA manager and with no further prompting said: "QA
has signed off on it". With that, the two big guys signed. It was at
that moment I realized, the ultimate judge of a titles worthiness for
manufacturing had nothing to do with these two big wigs.
Make
no mistake, I learned a TON from both of them and I highly respect
them. But they really had nothing to do with the release of my title.
Even the QA manager was really just reporting the results from the
testers. These are the very same testers that we paid only 12 dollars
an hour... we used to say: "Want to test a game? Wait, are you
breathing? Yes? OK, you're qualified". Now that was 1992, games are a
lot more complicated but the attitude towards testers has not evolved.
"Want to test a game? Wait, are you breathing? Yes? OK, you're qualified"
I have two stories to relate to you about testers.
In
1992 I was the Producer on Evander Hollyfield Boxing on the Game Gear.
One of the features I really pressed for was multiplayer. I really
thought this was the killer part of the Game Gear and we didn't use it
enough. But, it did require a LOT of testing, so did a lot of other
titles that released about the same time.
At one point
I'm sitting in my cube and I noticed a lot of new faces in and around
the test department. They had opened the flood gates and were trying
to get all of the titles tested. I wandered into what was our main
conference room to make sure they testers were really... testing. Of
course, the new testers needed some pointers. While I was helping
them connect multiplayer, I had the following exchange with one of
them:
Me: "Like boxing games?"
Tester: "Si"
Me: "Every play any boxing games before?"
Tester: "No"
Me: "What were you doing yesterday?"
He turned to another tester and said a few words in Spanish and then did a pantomime that resembled digging, and said: "Ditch".
Yes, they had hired ditch diggers to test games.
Here's
a bit of trivia: Evander Hollyfield Boxing was the first SEGA game,
Game Gear or Genesis, that included a list of the testers in the
documentation. I insisted, that was a lesson learned.
The
second story is sad. I worked for SSI for a short time, long story
short, it wasn't a good fit both in project and management. Some of my
lifelong friends that I made there were testers. They were amazing,
They could find bugs like all other testers, but they were also history
buffs. They could tell you that the tank in "that game", has the wrong
muzzle velocity for that era in winter. I can't tell you to this day
how they could even SEE that on screen.
I firmly
believe that one of the downfalls of SSI was when they moved the
company from their location in Sunnyvale, to be in the same building as
all of Mindscape. They didn't move the testers and that cost them
years of experience that could not be easily replaced. Those testers
were the heart and soul, and I don't know if it was the ego of the
manager, which I suspect given the poor managers, or the costs... but
not moving the testers I believe was the steak to the heart of the
company.
So if you take nothing else away from this post... VALUE YOUR TESTERS.
Your thoughts?
P.S.
OK, one more. In 4 years at SEGA I was asked hundreds of times if they
were looking for game testers. I would always hand them my card and
say: "Sure, go home... play a game you don't like, the same level over
and over for 8 hours straight, then call me and I'll help you get a
job". I never got a call.
|
And also as an aspiring game designer, I know it takes the right kind of people to balance prototypes too so...
Testers FTW!
MMO places are a little different, basically requiring you to have played their specific game a ton (It is impossible for me to play every MMO to max level, so this can be hit or miss).
It is all a bit wacky and random from company to company. It is fun to talk to different people and have them be surprised that every company doesn't do it the same way they do it.
I've talked to people that don't consider the general QA people to even be really part of the company, to places where they actually aren't (QA being a pool of temp workers from an agency hired on demand, with only the leads being real employees). On the other hand, I have seen places where QA gets tons of respect and fancy swivel chairs.
I wonder which places have buggier games... hrmm?
I'd have to disagree with your point that QA is the ultimate decider of whether a game is done. If it was entirely up to QA to release a game, there wouldn't be ridiculous crunch time. No, it's the schedule, be it the original one created at the beginning of the project, or the hundredth revision of it near the end.
A schedule is the driving force behind crunch time, but it's not the programmers who decide that crunch is over... it's the testers. The only exception is when a company sends a product out even with bugs, then the schedule would indeed decide the product is ready for release.
And as a side note, I did a blog post awhile ago that makes the point that crunch time = Producer failure.
Mac
I can't disagree that testers verify bug fixes -- that's why they're paid the big bucks -- but QA has very little to do with the overall product cycle. It is rare that a title ships "when it's ready."
I can only recall one or two times when I had been given the luxury of a complete and thorough test cycle. Every other time, a deadline (i.e. the schedule) was dangled over my head. The sign-off sessions you brought up were more formalities to an unavoidable end. The QA approval always had caveats.
Just to be clear, I'm not discrediting your blog piece; QA definitely plays a role in the process. The part that I have difficulty with is the amount of responsibility you've placed on QA. It's bad enough that testers are paid the lowest in the industry (and unchanged since 2007), but to say they hold the biggest sway in the development of games comes across as almost a joke to me.
The industry may value QA as much as you do, but it has an odd way of showing it.