QA is extremely important. Those are not my words. That's a quote from a session at GDC this year, and it's something I have not heard anyone disagree with. Ever. So then why is Quality Assurance considered just a stepping stone into the rest of the industry? Why are QA testers generally ridiculously underpaid? Why do they keep getting fired after the title they worked so hard on is launched? Why were so many people genuinely surprised that my company sent two QA representatives to GDC this year?
Before I start, I would like to remind you that what you are about to read are my personal thoughts and opinions, which I have formed by listening to and reading articles by people in the industry. I will, however, also share some of the philosophy we apply in my QA team at Paradox Interactive.
So here we go. Let's start by looking at what you need to make a successful game! To me personally, there are three core things you need to get right:
In the list above, QA is most definitely involved with at least the third point. At Paradox Interactive, QA does the second and even the first one as well. Do you see what I'm getting at here? If you want to make a good game, you need good QA testing.
Since QA is quite essential, it needs to be inhabited by the right people. Unfortunately, to begin with, not all people working as testers are the right people for the job. QA does not mean playing games all day, it means testing them. That's a huge difference. If you don't realize right from the beginning that you will have to spend your days breaking down the game brick by brick while examining every side of those bricks, you're going to have a hard time. You will most likely not sit on a sofa with a controller in your hands, balancing a Coke on your belly. QA has come a long way from just goofing around and occasionally file a report or two. While some companies still have bug hunting as their QA department's main responsibility, many have moved in the same direction as we have at Paradox Interactive. We look at usability, perform user experience tests and have mandate to comment on the “fun factor” of the game, for example.
Furthermore – this fallacy that QA is just a stepping stone into the rest of the industry has to go. I appreciate that a background in QA is great for a producer, and of course everyone is free to pursue the careers they want in whatever field they are interested in. But if you know that you want to work as a game designer, please apply for game designer positions and not QA. If you know right from the beginning that you do not intend to stay for at least a few years, please don't even bother.
One way to get rid of part of this problem, I would like to argue, is to raise the qualifications required for anyone to get a job as a QA tester. No other disciplines in the industry pick people off the street, so why should QA? The bare minimum, if you ask me, should always be a formal game education – a background in software testing should not be too much to ask either. Experience in other parts of the industry should be encouraged as well. A tester that has a background in animation will be more capable of finding animation bugs, and a modder might be able to break your editor in a more effective way.
Because let's face it – if nothing else, we're potentially dealing with a lot of money here. The less experienced your QA team is, the bigger the risk that they miss those really critical issues that end up costing you a lot of money. Just to give you a concrete example of what money we are talking about here: In one of the titles we have been working on, we discovered a server issue in beta. The title was not that big compared to your average AAA game, but that single issue still ended up costing us roughly 130.000 USD. That might not sound like much to some of you who work on bigger titles, but for this project it really was. Imagine if it had not been discovered before the launch of the game. I won’t even try to calculate the possible lost revenue we could have faced due to pissed-off gamers. An experienced tester will be more likely to anticipate issues and even preventing them from ever happening, if they are involved early enough in the development. Don't make the mistake of assuming that testing games is easy. You keep learning things all the time – and the more you learn the better you get, obviously.
I do not, however, blame QA testers who glance longingly at the greener grass on the other side of the office. Because so far, QA has hardly been a good choice for people who want to make any kind of career. You become a tester, and then you can basically either become a manager or leave QA – and we risk losing a lot of capable people due to this. Management simply isn't for everyone. You could be the greatest tester in the world but a terrible people-person, and then testing is what you should do. It's not in your nor the company's interest to make you a manager in that case. That's why you need a career path for these people as well.
At Paradox Interactive, we have just started implementing a system for this. You start off as a QA Tester, doing what all testers do – test the games. Then, when you're bored and up for a new challenge, you can specialize on a specific area and become a QA Specialist. Examples of areas in our team are AI testing, user research and audio. After that, you can eventually become a Senior QA Specialist – which we however have none of so far.
While still in the cradle, this initiative has proven to work very well. The team seems very happy to have been given the opportunity to focus on an area they are interested in and the overall quality of the games we put out is slowly improving. In addition to this, we can show job applicants as well as the rest of the company how serious we are about what we do, and in a very concrete way how much competence our testers actually possess. Hopefully, this will also make our testers more interested in staying with us for a long time.
Talking about making testers stay with the company – working with titles such as the ones our development studio creates (grand strategy franchises such as Europa Universalis and Crusader Kings), just wouldn't be possible if we had fresh testers for each title. They are very complex and take so much time to get into that the experience the testers gain over time is priceless. Fortunately, since we have the advantage of having several titles in production at a time, it's never a question of having to keep a team on hold during off-season. We always have something to do. In the rare case that we have less projects, the specializations themselves are a great fall-back. Instead of being idle, the testers are encouraged to educate themselves – attend lectures, watch videos, read articles and books, and so on. We even allocate one day a week especially for those kinds of tasks, if we don’t have too many other things to do. That way, they will be even more able and efficient when the work-load is getting heavier again.
Of course, I do understand that not all studios work that way and keeping the QA team on board between titles could create a lot of expensive down-time – but in a perfect world, all QA testers should have full-time employments. They should be treated with the same respect as everyone else, and their experience with the company's titles should be appreciated. However, our world is unfortunately far from perfect. Many testers are still unable to land anything but contract jobs – some might even sit on six-month contracts for several years in a row, without ever getting employed for real. In my opinion, that's just not the way you treat people you care about.
Before I end this rant, I would just like to say that I don’t want to sound like I’m putting Paradox Interactive, our QA team or our way of testing on some kind of pedestal. We have a lot of areas that we are still working on improving, and you’ll probably find that our games – just as any other games – are not perfect. Ultimately, it’s about resources and how you use them. We have decided to focus on having fewer testers that, if they weren’t to begin with, are given the opportunity to grow to become experts in their field – rather than having a huge team with less qualified testers. It’s a concept that has worked quite well so far, but we still have a long way to go. The most important thing, though, is that we are really trying to move forward all the time. To us, QA is truly important.
So, let me just finish by asking you again: If QA is as important as most of us agree that it is, how come it's treated the way it is at many companies? Also, how do we attract the right testers, and how do we make them stay? If we all, people who work with QA as well as people who do not, could just start thinking about how we talk about and interact with QA testers as well as the discipline as a whole, I'm sure we'll all win in the long run.
We all want to make a hit, and without QA you'll be sure to get an S before that word. Please stop pretending like that's not the case and start showing QA some respect.
Anna Jenelius, Assistant QA Manager, Paradox Interactive.
Thanks to Mattias Lilja, Susana Meza Graham and Joe Longworth.