| Ron Dippold |
|
|
If you're not reading In the Trenches you really should. I'm enjoying the comic, but the real gold for anyone in the game business is in the stories from QA people:
http://trenchescomic.com/tales/post/drive-fast-and-crash-a-lot When you say 'How did QA not catch this?' or 'Didn't they have any QA on this?' the truth is they probably had QA and QA probably caught it, but they decided not to fix it. I respect QA. I cringe every time something show up in my Jira folder, but then I suck it up. I also treat QA guys with respect and strangely my bug reports are pretty useful. So yes, I totally agree with Brandon's article here. It's like getting a prostate exam or tooth cleaning - not pleasant, but you're better off getting it than not. |
||||||
|
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
| Daniel Martinez |
|
|
This article is really just the tip of the iceberg as Brandon mentioned. There is a stereotype that "QA does nothing but play games" and while that is technically true, it's still a job none the less and does not warrant compensation that puts sweatshops to shame. I don't ever sit in front of the same game for 8 hours feeling my retinas burn away for leisure, much less if the game is buggy and requires the constant repetition of arduous, mundane tasks (searching for mapholes anyone?). After the first 40 hours or so of being on a project the novelty wears off and then the game of sleep vs work begins. With that said, if QA is regarded as a cancer separate from the dev team and the company as a whole, what makes developers and publishers think that QA will put forth their best efforts? A happy employee is a productive employee. In the worst of cases, all QA employees really want is some respect, recognition, appreciation, and maybe some munchies every now and then. :)
|
|
|
| Jr Hawkins |
|
I'll start off saying there's a huge difference between internal QA, and publisher QA. Most internal teams I've been involved w/ have tried to treat QA like developers (at least the internal QA team). Publisher QA is a mess. This is where you hear all the horror stories from QA, and I'm afraid to say it's well deserving. Honestly I think it's a lot worse than most developers realize, and that's pretty scary.
So why don't developers just keep internal teams? QA tends to be the largest team on any project, and for an internal studio it's hard for them to justify keeping 20 guys on who have no work to do since most of QA is only needed for the 2nd half. It's much easier for a publisher to organize QA teams since they'll have 5-6 projects a year instead of one project every 2 years. I'm not really sure why publisher QA has so many problems though. Maybe it's the pay is too low to get quality employees, or maybe it's just bad management, but it seems every publisher QA facility in the world is crap. The worse thing about it I see publishers throw giant teams onto projects when only 20-30 testers are needed (and that's being liberal). Then the team is 1/2 as productive, and cost 3X as much. |
||||||
|
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
| Joe McGinn |
|
Good article, couldn't agree more. Always try to involve QA in the design and other processes, and they are as much a part of the team as anybody. Also agree that if they don't "want to get out" they need a viable career path. A great QA lead is incredibly valuable.
|
|
|
| Paul Boyle |
|
I would love to support this article. QA guys are people, and they deserve fair treatment. However, tbh, Brandon's suggestions would lead to making worse games, not better ones.
Unfortunately, there are problems with both his suggestions: 1st, "Get Bonus" Hiring people who are not 'scrubs' and giving them a better career path, not laying them off, essentially amounts to paying them more money. The fact of the matter is that most projects don't solve all their bugs, pre-ship, NOT because they aren't found, but because they dev team ran out of time. The only way to get more dev time, usually, is to have more money. Less money = less dev time. Hence, 'higher quality' QA would, in general, lead to a bigger bug backlog, and a fewer bugs fixed. The key misunderstanding here is that QA is NOT your first line of defense against bad reviews. It's your LAST line of defense. Its there to catch all the things that made it through the cracks, the little things. If your game has huge issues that are going to significantly affect its metacritic rating, these issues problem stem from failures that are present before QA can and should get involved. Yes, QA is important, but its job is not to help you build a game that will score higher - it's there to help you prevent a game that will otherwise do well from failing due to uncaught bugs. 2nd "Quality Time" Most development teams already have too many people contributing ideas and suggestions without having development responsibility. Sure, QA has a perspective, and I've never failed to hear it voiced, there's a certain percentage of bugs that are 'buggestions' in every project. But if you really lack for ideas and perspective on your team, that's a failure of design and production, which should be addressed, not the lack of a QA person pointing those things out. |
||||||
|
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
| Jeremie Sinic |
|
It's a vicious circle I have witnessed too often already in my short career: internal QA people are often hired without any specific qualification, then they are not considered worthy of being listened to, then they quit and are replaced by new faces without experience...
I think the QAs should work directly under the Producers, so that their voice really counts. Currently it feels like QA's power is only in informing developers about issues and bugs. They unfortunately don't have their say when it comes to taking decisions like fix or not fix. QAs are the closest people to end users: ideally they should be the ones with the last word. |
||||||
|
|
|||||||
|
|||||||
|
| brandon sheffield |
|
|
I definitely don't have all the answers here - and due to its origins in the magazine, this had to be under 1,000 words, so there's a lot I couldn't cover. Really I just hope to open up the dialog about QA, and the situation can improve - I'm sure better ideas will come forth in the comments here than the ones I presented above.
|
|
|
| Scott Williams |
|
This is a really interesting read and tackles quite a few key points. For me as a QA Manager I found that the career path in QA was hard to obtain. It took years working on temporary contracts until I got my first permanent lead role and it went from there.
When recruiting for my last project I only hired experienced testers. This was the strongest team I ever created and the reception from the development team was great. The problem doesn't lie here, it's convincing top management that the team is worth keeping around for future projects and that they should be paid much better than todays rate. Unfortunately when the end of project comes, QA is one of the first on the chopping block regardless of talent and feedback from the production team |
|
|
| Glenn Storm |
|
|
QA is an essential specialist position; a critical part of development able to provide invaluable information to each team member, particularly the designers and producers.
While the artists paint, model, and animate; while the audio crew mix, tune and tweak; while the engineers organize, implement and optimize; while the designers research, explore and refine; while producers gather, communicate and cooperate, _someone_ needs to help be the eyes and thumbs of the player. Everyone on the development team is ideally an advocate for the player, especially the designer, but the only position dedicated to being an objective observer, the only specialist with the bandwidth capable of giving the kind of feedback your reviewers will give post release, is the tester. Bugs are a development team's golden eggs; each one is an opportunity to make the reviewers' experiences better, to make the game better for every player. And that opportunity is directly proportional to the severity of the bug. Welcome the bug from QA with gratitude, not just respect. |
|
|
| John Flush |
|
Unfortunately too many business owners see QA as the department that tells you that your other departments (Development) can't do their job right. As long as that is the view of QA and the solution is viewed that "Development should just do better" QA will always be at the short end of the stick.
That and the revolving door of QA. Having gone to a few testing conferences over my career it is amazing to see the churn of ideas. It seems like QA gets reinvented every 5-10 years with new nouns describing the same workflows and processes because the people that figured it out 10 years ago got tired of never being listened to and moved on. |
|
|
| Simon Ludgate |
|
|
As someone who's worked as both a QA and a reviewer, I have to say: these are not in any way, shape, or form similar jobs. While working in QA I've often seen things I'd absolutely smash a game for as a reviewer, even things that could be fixed quite easily, but if I enter them in as "bugs" all I get are angry managers telling me "that's not what your job is."
One of the big problems is that Quality Assurance is poorly named. Most of the time, it's "technical requirements checking." For example, if you were to publish a game on a PS2, no UI element could enter the 15% outer edge of the screen, because it could get clipped off by an old CRT. So "QA" people "play" the whole game with tape on their screen, making sure nothing goes out of bounds. There really is no glory to this kind of task. |
||||||
|
|
|||||||
|
| Ariel Gross |
|
|
We treat our embedded audio QA tester as a member of Team Audio. I believe he's invited to all of our meetings, and we take his input seriously. He is truly part of our team, and his contribution is totally valued. Also, he's just an amazing dude. Brendon Ellis, WE LOVE YOU. /salute
I agree with the sentiment of this article, too. There is a sort of stigma about QA with some people, and it's a shame, because they really do make our games better in a huge way. |
|
|
| Nathan Mates |
|
It's not that QA is overloaded. It's that mathematically, they're a very thin line of defense. Let's do the math. Numbers are approximate, based on my experience in AAA games. Let's have a 25-person QA team working 100 weeks on a game for 45 hours a week. (Yes, QA generally starts smaller and ramps up, but let's smooth that out.) That's 112,500 hours of eyeballs on the build, but most of those hours aren't on the final (or even near-final) build. Now, for a AAA game, selling 100K copies in the first week or two is pretty normal. Assume each of those buyers plays (on avg) 1 hour each. Guess what -- consumers have now have a pretty comparable number of hours playing the game as QA, and way more hours playing the final build than QA ever had. And if the customers play 2-10 hours you're way behind.
When it all comes down to the bottom line, QA is not a monolithic force. QA is comprised of people, who may be good, bad, or indifferent. I've worked with some QA who's worth their weight in gold, and some QA who aren't. (Worst case: a two-week fight where QA insisted that unscrewing and disconnecting the network adapter from the Fat PS2 was comparable to pulling the network cable or controller. It only ended when I bought one of those adapters for my PS2 at home and quoted chapter and verse from its manual saying to power down the PS2 before [dis]connecting the adapter.) QA managers who can really enforce best practices on QA -- like writing up a bug report with somewhat passable English spelling, grammar and sentence structure, or discouraging them from re-opening a bug 5 minutes after I mark it closed, saying that my checkin somehow didn't make it into last night's overnight build are great. Work with developers, and we'll work well with you. |
|
|
| J Z |
|
As QA that works outside of the game industry (telecom) I can tell you that no serious QA professional would take a second look at the average salaries of QA in the game industry. Why submit myself to tortuous repetitive testing for minimum wage when I can be pushing the boundaries of technology for other companies and making twice the money with half the time spent?
I love games and game development, I would love to get into that industry. But the path would never be through QA... and the only way I could see anyone half-interested in those lay paying QA jobs is if their endgame is to be a developer... thus resulting in what the article said, good QA don't stick around. As for general QA comments, there's always going to be scrubs, and those worth their weight in gold, even outside of the game industry in higher paying QA jobs. I think QA management is the real dark horse here and should not be discounted. I was very disappointed to see even the QA leads having subpar pay. I would look for the best of the best of QA management if QA salaries on a whole will never really approach those of development. |
||||||
|
|
|||||||
|
|||||||
|
|||||||
|
| David Phan |
|
|
Tighten up the graphics on Level 3.
http://www.1up.com/features/top-5-game-design-college |
|
|
| Mitchell Cronin |
|
There's a lot of good points in this thread.
While I certainly can't support keeping a huge 20-30 man internal QA team after a project finishes, I think there is a lot of merit to keeping a group of people full time depending on the size of the studio and how the work is spread out. While a lot of QA initially starts as 'random kid off the street', if you keep them around through several project cycles they'll develop skills that will eclipse anything you can get out of that random kid on the street situation. In addition you can take the time to train them on other skillsets so they can do more than just test the game. Where we are at QA is taking a more active role in testing the internal development tools and outsourcing pipelines. This kind of work can bridge the gap between projects and gives more incentive to keep a portion of internal QA on after a project. Still, the major issue is career path advancements and incentives. There are lot of good reasons to keep on QA and there's probably still many other creative ways to utilize QA that haven't been thought up yet. However there is little to no incentive for someone in QA to want to stay in QA. Ask anyone in QA and they will want the shortest path out. (These days that is getting much longer. With many universities offering game development degrees now QA isn't the awesome foothold in the industry it once was.) It isn't until they reach supervisor or manager levels that the salaries can even be considered an 'adult wage'. I am not arguing that freshly hired QA testers need a massive pay increase, but there needs to be greater incentives to stay in QA. For the testers that stick it out and build and improve their skills there needs to be compensation for it, and incentive to actually just...stay in QA. This will help filter out the scrubs from the standouts. You can essentially use someone's temp contract as an extended interview to really see their work habits instead of just taking a gamble after a two hour interview. Once into a full time role they need a respectable wage with opportunity to advance beyond just becoming a manager. As mentioned above there could be people with programming backgrounds who help test internal tools and pipelines, and are expected to do more than just hit the crash and post it. There can be focused compliance testers who know all the ins and outs of Microsoft, Sony and Nintendo's rulebooks. You can have tech art focused testers who do more than just point out the holes in the world but know the ins and outs of how a streaming system behaves. You can even have QA design and write testing tools to help automated some of this process. The best person to write QA tools is someone who's had hands on experience testing the products they work on. There are lots of possibilities for QA. No, it doesn't make sense to keep all of them around, but there is a lot of merit to keeping small teams on to build skillsets and knowledge. As a lot of you have mentioned one trained and awesome QA tester is worth way more then piles of kids on the street. The position does have a low barrier of entry, but once someone has proven themselves why not offer a legit career path with adult wages beyond just becoming a manager? |
|
|
| Joshua Sterns |
|
|
Great article. Glad to see attempts at generating awareness. Personally, only my time in retail was worse than my experiences in QA.
I don't have the answers, and my experiences are limited to a two year period. I do have some general suggestions that may be fun to discuss. In regards to the difficulty of hiring QA. Change the interview/application process. Have potential employees write a number of paragraphs/pages. Do more then ask if they like videogames. Find out if they have a college degree, or if they have a writing hobby. The application processes I experienced were a joke, and did little to indicate what my abilities were. Attracting top talent is also difficult when the pay is so low. Keeping QA around full time is too expensive. Then diversify the job description. Have QA do more then test. Is there really nothing to do for this department before the home stretch of development? Also having QA in at an early stage may help prevent some of the buggestions. Many of the buggestions I saw were a result of a lack of communication between the devs and QA. I also have a hard time understanding why QA is too expensive, but paid time off, bonuses, benefits, etc. for other departments are not. Overall I believe many of the issues with QA stem from the hire/fire cycle, low wages, and lack of respect from both sides. Address these problems and discover what a good QA squad can do. |
||||||
|
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
| Jeremy Tate |
|
So, I've been working as a Tester for 13 years now. (I'm not a fan of the term QA, since we don't assure quality, that's the job of the entire team). I find a common thread is that, even with teams where you have a stated respect for Test, few developers know what to do with a tester early in the process.
Personally I find I have to constantly be vigilant about progress the team is making, and if I'm not a part of conversations early in the process, I try to insert myself and provide as much value as I can. However, it sure is nice to work on a team that has committed to having that customer advocate early in the process. I was lucky that I learned my craft at a publisher which was part of a large software company, where Test is expected to be its own career path. That's not saying that you can't move on to other areas from Test, but if you really want to focus on having more and more impact as a tester, a clear avenue is there to follow. I've since left that company and found many places that don't have a good attitude towards Testers, if I set the expectation early on that I'm not there to move somewhere else in the company, I get a lot more respect and it helps change the culture. |
|
|
| Heinz Schuller |
|
|
As a developer, if you're not making QA a core part of your development team and treating them as such, then you're doing it wrong. A well-run QA department is an asset at nearly every stage of the development process. The key to this is working with QA directly to define & set metrics cooperatively. Then they can help you do your job better when they know what to look for.
QA can run build-verification tests on your tools to help keep your team at peak productivity. They can do competitive research on similar products and provide you with a wealth of data. Given the deadlines we all work under, it's fair to say these folks have saved me countless hours and saved my butt on numerous occasions. |
|
|
| Adam Dials |
|
I was a Test Director in the military simulations community on a sim called OneSAF. In the military, QA is treated with a significant degree of respect (partly because some of the testers are retired operators who can kill you in a variety of ways :) ).
I’m a long-time gamer and modder and hope to one day work in the industry. No insult is intended here and I genuinely respect the work the industry does. But the impression I get, which may not be correct, is AAA game testing is haphazard. It seems throwing large numbers of enthusiasts at the game is the norm. Is this the case? In military simulations, test engineers examine the sim at a number of points. First, we evaluate designs. Before a given design is implemented, we conduct a design review to focused on several things, like requirements traceability, realism, and usability. If we say a given design is not a good idea and present a cogent argument supporting our position, the design may be modified. This heads off problems before significant effort is invested. Next, our test engineers write test cases. The lead test engineer does a traceability analysis on the requirements, comes up with a test plan to ensure every requirement is examined, then assigns the construction of those test cases to test engineers. Test writing is divided between areas of expertise. We have technical test engineers, functional test engineers, usability folks, etc. The technical folks write load tests, system and platform tests. The functional folks look at combat systems and mechanics. And so on. Before the tests are executed, all of them are scrubbed - the development team gets a look at them as does management to help in getting the tests focused correctly. Finally, our test engineers execute the tests. All the engineers step through their tests, and the test lead compiles the results. A management board convenes and categorizes the bug according to severity with developers and testers in the room. Funds/effort/time is then allocated according to severity and ranked importance of the requirement. Is this level of rigor employed? Some of these testing techniques have to be used, but maybe you guys do it on the developer side of the aisle. Any perspective would be interesting to hear. |
||||||
|
|
|||||||
|
| Michael DeFazio |
|
|
Does anyone think Obsidian and specifically the QA department who tested FONV did not raise flags because of the games bugginess before it was shipped? Playing the game normally (not trying to break anything like a QA engineer would do) had me experience terrible freezing, corrupted savegames and and serious companion AI issues (to the point of breaking the game twice)... and I only played the game for 40 or so hours. Everyone I know who bought the game day 1 had almost exactly the same issues or worse. (How could they not have known?)
Smart money says they released the game KNOWING it was broken. (And I have heard that Bethesda basically moved up (rather than back) the release date to push the game out the door, even though the devs knew it was a mess). I know people might think that I am putting the blame on Bethesda here, but to be frank, I waited and waited for patches to fix these issues and they took WAY to long... I never finished the game because even after some of the first patches, I still got freezing issues and corrupted save games... (Loosing 40 hours of progress was a huge turn off so I eventually gave up). The idea that Obsidian's QA department didn't recognize the problems with the game's bugginess which contributed to the low metacritic score seems like a farce. |
|
|
| Sean Conrad |
|
Does anyone actually believe the average QA tester is making $45k a year!?
|
||||||
|
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
|||||||
|
| Christopher Marker |
|
I cannot agree more. I worked for a while as a QA tester at a major publisher and I was never made to feel like a valuable part of the development process. Instead I felt like worthless badge trash, and was seeking another job frantically. The pay wasn't even enough to pay my bills living with 2 roommates in a dump of an apartment. Quite literally the cheapest in the area. The minimum-required-by-law benefits were a joke, and cost 20% of my weekly paycheck that was already too little to pay my bills. I don't know of a single person who actually took any of them. Then add in the fact that I had to worry about being let go every couple weeks, and knowing I will be let go no matter how well I do, it was just a terrible experience. Honestly you have the same pay, and better job security working a cash register or flipping burgers. To move up out of associates tester was something like a 5 year process, and only happened if you displaced one of the full career supervisors.
We didn't come within a 100 miles of the Devs. Hell we would be fired, possibly sued, if we did speak to them personally, since we didn't have clearance to enter that part of the campus. It as standard operation assumption that QA testers would steal IP, cause leaks, or even try and steal equipment. It was honestly only the job some kid who lived at home, who didn't know any better, could even afford to do, and unsurprisingly, most of the saff was 18-20, lived at home, and this was the first job. And the supervisors, the only true "full time" guys didn't test, just yelled as us to not talk all the time. And of course working with "summer job" types who didn't have a clue, some of whom didn't wash regularly too boot, was just as poor as being treated like trash by the company. Of course they wouldn't have to have treated the staff like children if they had compensation and working conditions that could allow actual professionals to even survive at the company. I feel like at that company, they might as well have tired prison inmates to the do the job, since that is how much they valued and trusted their temp QA testers. |
|
|
| Matthew Cooper |
|
|
Sean, it refers to average in the mathematical sense. You're probably reacting to what you perceive as the median or mode.
|
|
|
| Michael Liaw |
|
I've worked in QA for 6 years now and I've been treated quite well at my company. I find that treating QA as QA and not as QC (Quality Control) has made all of the difference. We aren't constantly in the game playing it for our job. We are also analyzing the design docs, working with developers on specific issues they have, and generally supporting the other developers in making the best game possible in whatever way we can.
One thing that makes it hard for QA to be treated well and looked upon as a profession, not just some high school dropouts, is that most companies do not bother trying to improve upon their QA processes it seems. The game industry as a whole tends to think getting enough people tossed at a game playing it will guarantee that all bugs will be found and ensure that the quality of the game is good. This is definitely not the case, regardless of how much time you have to fix the bugs found from doing this. If people are able to find bugs before code is generated or content has been developed, the savings are immense and this is where I find QA is the most useful, specifically in the early stages of a game where most people would rather not have QA. Hopefully the videogame industry as a whole can grow to better understand how to effectively use QA within a game development cycle. |
|
|
| John Caminiti |
|
I worked QA at a major publisher for three years and it was a terrible experience. We were treated like poorly by the company, the pay was unlivable (I think the most I ever made with overtime was around 25k a year) and employees in other departments called us the monsters in the basement (the QA department was kept in the basement). At this company, no matter what your background and education was, once you’re QA, you are only looked at as QA. People were desperate to move out of QA but, the only people that moved out were the ones that had the right friends, or it was just pure luck. And if you moved up in QA or became too good at your job, you wouldn’t be allowed to move up because you had became too valuable. Most testers are kept as temp employees through a temp agency, so they can easily be fired after the project ends (next time you finish a game, watch the credits and watch the large number of names under Volt).
QA is definitely not the correct term for the job. There is no assurance of quality that is performed, even though a test team is the greatest focus group you can have. The test team (QA group) is an incredible resource for feedback and focus testing but is looked down upon so much that they are completely dismissed. Apparently it’s a much better idea to get a few teenagers to come in and pay them each $50 for two hours to focus test the game, then it is to get feedback from a group of people that know your game inside and out (and at $50 for two hours, focus group kids make more than double then a testers hourly wage). You are paid to test the game, if you try to make a suggestion to better the quality then you are shot down because you are only a tester and it’s up to Production or the developers to worry about the quality. All the testers really do is to make sure you don’t fall through the map, make sure the menus and gameplay work, the game can be saved and the game can be completed. Oh and we also have to correct the mistakes written by testers from outsourced companies in foreign countries (they may be cheaper to hire but the quality of these outsourced foreign groups are terrible, so if you get a really buggy game, the testing may have been outsourced). Our QA group worked with Production, which is a terrible idea, we didn’t normally have direct contact with the developers, Production was always the middleman. Every bug we entered had to be approved with Production before it was sent to the developers and there are a lot of bugs that would not be sent out to the developers, usually due to trying to hit a ship date. I came to loathe the term “Will Not Fix” (WNF). QA is a terrible place to work for the most part, and it’s not at all a good starting point to break into the industry. Companies will hire absolutely anyone to test and pay someone a terrible wage and let them go when a project ends (although if publishers would learn to not release all of their titles in the last 4 months of the year, and spread them out throughout the year, you can have better groups of testers working on your games constantly). QA is looked down upon and treated with disrespect even though it’s such an important part of the game development process. I have seen companies/publishers hire kids right out of college for Production positions and would not even consider people in QA, even though people in QA are far more qualified. From my personal experience, the only people that have respect for the testers are the ones that have been testers themselves. |
||||||
|
|
|||||||
|
| Mathieu MarquisBolduc |
|
It may sound obvious but you need to have several levels and types of QA, just like there are several levels and types of programmers.
Game Tester, Development tester, Support Specialist, are just a few types. Your entry-level game testing job will never be great, just like entry-level programming sucks. The important thing is that people have opportunities to grow and move up within their skill set. In all my gaming jobs testers (QA, whatever) HAD a career path, should they have the skills and will to climb it. Maybe its more of a Canadian thing? I dont know. I know at least one studio lead that started as QA... |
||||||
|
|
|||||||
|
|||||||
|
| Fiore Iantosca |
|
|
I've been in software QA since 1996 in many different software shops. It's the same everywhere.
QA does not get respect and when projects need the dates moved, QA ALWAYS ALWAYS ALWAYS SUFFERS. |
|
|
| Kym Brainard |
|
I will admit that QA is generally disrespected by members of the Dev team, and even worse by producers. You'd not believe what I hear beyond my cube wall. They also do need to be paid better than what they are. However, many aspects of this article, I cannot agree with, and think are over-exaggerated. In the end game development is a business like any other. It's all about supply and demand. There are less spots on a team for QA, and far more unemployed floating around, and it's also a job you do not need to have a college degree to do.
Most QA is brought on towards the middle of a project because that's when there is actually something TO test. They're let go at the end because, there's nothing for them to do. You can't seriously believe that small to midsized studios, that only work on 1-2 projects at a time, can afford to keep them on the payroll, while they sit around catching dust. For that matter, at most studios, many artists, and programmers are let go towards the end of a project as well, because they're not needed in pre-production for the next project. As far as 'nowhere to grow into,' that statement also not fully sound. The rest of the development team generally has a college degree, or some kind of official training. Albeit, there's more to QA than 'just playing games.' FAR more! But in the end, it's a job that you are capable of hiring someone out of high school to do, and with several weeks of training, they'll at least be suitable to hold their own. If you, as QA, have a desire to go beyond that job title, then you need to do as the rest of us have, and either train yourself, or go to school. Heck, half of my studio is filled with Designers, and Producers, that started as QA... It's not impossible. If you feel the desire to stay QA, but get paid more, then you should still get extra training into say, programming, scripting, etc. Make yourself someone that the Leads feel the need to keep around, even during downtime, because you're the only one who knows how to set up great test units, or can look through code to give actual analysis over something that's broken, etc. |
|
|
| Tony Forte |
|
QA and devs need to function hand and hand. Period. If it's production QA vs devs, or internal QA vs devs, whatever... there is no progress made on either front. Mistakes are made on both ends, and if there is no professional respect for each other, including QA respecting devs, negative progress is made.
Devs need to accept QA is here to point out their mistakes. Suck it up you need to re-design something or whatever. You are paid to develop over a period of time, not to make a set of features once then dance around QA feedback. QA needs to deflate their giant ego and focus on technical integration and improving their own skillsets. Become something where "kids off the street" are only used for focus tests and full time/long term contractors can design, run and execute game-breaking test scrips and ad-hoc strategies. Pay increases can not be justified for un-skilled labor. BUT publishers or devs also need to see the value in someone who gets paid to play games, broken or not. Qualitative feedback should be taken at face value, but look at the face who is giving it to you. Literally a random person or a video game specialist. |
|
|
| Aaron Haaf |
|
An interesting takeaway note from all the Game Production Workshops (6 months for a game in Unity with several classmates) in my school has been that QA immediately beginning development has made a large difference. All of the "successful" games have made a huge push for internal and "recruited" user/playtesters.
One of the biggest things has been the large amounts of feedback for the design of the core mechanics. It might have been the inexperience showing of the teams, but it seems that people of any discipline have a hard time of seeing what's wrong right in front of them. A platformer that was pushed out last quarter that I had the pleasure of testing, had it's controls VERY clunky, and this wasn't addressed at all despite many complaints from these QA'ers. It really did hurt the rest of the game. |
|
|
| Abel Bascunana Pons |
|
|
I think QA is closely related with game design. QA detects issues after they have been implemented. A game designer should see potential issues before they are implemented. That's why a good QA guy can become a good game designer if he has some capacity of abstraction and a great deal of experience playing games from all genres.
About all the QA stories we've heard of and lived through personally, i can only say... a QA guy must inevitably end learning the meaning of the phrase "estoic calm" if he wants to be happier with his job. Sometimes we may think our observations are too right not to be taken into account, but making a game is not as easy as saying "change this or that", and involves the coordination of many busy people in different departments. To top this, deadlines are really tight and many QA efforts end up down the drain. We must live with this. |
||||||
|
|
|||||||
|
| Daniel Martinez |
|
|
There was an article on Gamasutra a couple years ago that said if QA ever unionizes, it will not be because of pay, working conditions, or the like, but because of credits. I worked on a couple of projects where no one in QA was credited, in one project even the development team and admin was not credited, only the licenses that had been acquired were given credit and that was to avoid a lawsuit.
|
|
|
| Jeff Lindsey |
|
On the related topic of "why games release with bugs", in my experience, it starts with the publisher or client. They want more features and fast progress, and force teams to accrue massive technical debt in order to meet their expectations. They're not held accountable during the payoff of that technical debt (team crunch, KS'd bugs etc), so they have no real interest in addressing the bigger issue (until buggy releases can be quantified in revenue lost). In fact, nearly every contract I've worked with explicitly assigns technical debt to the team - "Alpha: all features complete, placeholder art and bugs exist".
I'm a big fan of constant visibility on technical debt and constantly paying it off. I would rather be stuck having to justify doing less (but doing it better) to stakeholders than setting my team and customers up for the inevitable mess. |
|
|
| Roberto Estrella |
|
I heard amazing things from Nintendo's QA working in Pre-production and along with the design team from beginning to end, does anybody have an article about it to support it?
|
|
|
More: Console/PC, Social/Online, Smartphone/Tablet, Indie, Exclusive, Recruitment, Audio, Design, Programming, Production, Art