|
I started my first game art outsourcing studio in 2000. We were one of the first large-scale art outsource companies in the United States. This was back when the term outsourcing was associated with other industries like vehicles and electronics, not video games. Developers didn’t outsource like they do now. They either brought in contract artists temporarily or leaned on other game developers to lend their art teams when they were in-between projects. Outsourcing was not the ubiquitous term it is today.
Back then our biggest job was teaching game developers what outsourcing could do for them. Our pitch was that developers didn’t have to staff up quickly, carry those employees through lean times, or spend all their time managing a giant team of artists. Obviously it was a sensible decision, considering 90% of all developers outsource now.
Now, we no longer have to make that pitch. Developers already know the benefits of outsourcing. They do it all the time. The challenge we face today is convincing them to keep that work in the United States instead of sending it to a company in China. Convincing developers to keep this work local is a much harder sell. To start, let’s change the term for outsourcing locally so we are all on the same page. We’ll call it localsourcing. It’s got a sexy ring to it.
Sending that work to China is cheaper than localsourcing. The day or hourly rates for an outsourcing company in China can be 25% to 50% less than a stateside studio. Exchange rates and cost of living play a factor in this, but central to this is the fact that China is still a communist country and they aren’t as subject to playing by market rules within their own country. Worker’s salaries aren’t necessarily dictated by the market. They’re simply dictated.
Causality aside, the common consensus is that outsourcing game art to China is cheaper than localsourcing. I am here to tell you firmly, “not necessarily.”
There are costs associated with outsourcing that are not directly related to the asset you are purchasing called secondary costs. Secondary costs are the costs that are not built into the price of your product, such as your time, transportation, materials, etc. If the grocery store 30 miles away sells toilet paper for 50 cents less than the one down the street is it cheaper to drive 60 miles there and back for a cost savings of 50 cents? No, of course not. You already know the gas alone negates any savings. You also factor in the wear and tear on your car and most importantly, at least from my side of the street, your time. What is your time worth?
Since this is a game industry blog and not a fecal hygiene blog let’s bring it closer to home. An outsourcing firm in Shanghai agrees to build you one castle for $160, which you both agree is 8 hours of work. Done deal. You’ve got a budget of $160 and you need a castle. They want your $160 and they can build you a castle. It’s a win for all.
This number is for example purposes only. No studio, in China or the US, or anywhere in the world would take on a job for $160. That I know of, at least.
Before you get started, you need to provide them with everything they need to build a castle that works for your game. Since they are an asset factory, designed to create assets to exact specs, you need to give them a specific polygon limit, plenty of photos of castles, concept art if you have it, a description of what the castle needs to look like and its purpose, where the entrance is located, how many parapets it has – all very important information. You pack it up in a zip file and send it off in an email. All in all, you’ve spent about 4 hours of your time on this castle, so far.
Since the company is in Shanghai, and you sent the package at 1:30 pm you need to wait till 5 or 6pm till they get into work and receive your package. About 5:30pm you get an email saying they got the package and they will take a look at it. You write them back, letting them know you are looking forward to seeing their work and to please let you know if they have any questions. That’s another 20 minutes. It’s now 6pm and you’ve got a date with Season one of Battlestar Galactica.
You get in the next morning and there is an email waiting for you from Shanghai. They looked at your specs and your reference and they have a couple questions for you before they can start working. You answer their questions best you can, then go about your day. Since they are in Shanghai, you don’t expect to hear back from them till about 5pm or so. 6pm rolls around and another email pops up from Shanghai thanking you for all your help and information and they will get started on your castle right away. You email them back letting them know it was your pleasure and to not hesitate if they have any more questions. This is another 20 minute exchange and now it’s time to go home and see what those tricky Cylons are up to.
The next morning you get in and there is a castle model waiting for your review. You unzip it, eagerly looking forward to finally filling that blank spot in your terrain with a majestic Arthurian fortress. It’s going to be beautiful. You can’t wait. Unfortunately, the castle doesn’t look all that hot. The folks in Shanghai tried to use your photo reference as a stone texture. Plus the polycount is twice what you specified. You take a few snapshots of it, draw some paintover notes in Photoshop stating all the things that are wrong with it and write up a review making sure you ask why the castle is twice the poly limit you originally specified. You send the email off at noon. You just spent about 3 hours total retrieving and reviewing your castle, and you still have a blank spot in your terrain where a castle should be. The tension in your neck is turning into a migraine. Is it time for Battlestar yet? No, it’s only 12:30pm.
Around 6:30pm you get an email from Shanghai. The whole thing was a misunderstanding. They thought the photo reference you sent was supposed to be used for the texture. There was a misunderstanding with the poly limit. They are very sorry and will fix this right away. You understand, these things happen. The whole exchange takes about 30 more minutes and you decide to grab a six pack on the way home. You’ve earned an IPA with season two.
The next morning you receive your zip file as promised and inside is a jumble of amorphous polygons that looks like it may have once been a castle, before it was sieged by a dozen catapults and a bad optimizing spell. Again, you write up a review, trying to be as clear as possible, spending extra time building a sheet with the concept and a screen capture side-by-side. All in all, the whole ordeal took about one hour of your time.
6pm arrives with an email in tow, asking you if you have time for a phone call. You agree. If you have to write another in depth email review you just might lose it. They call. You all talk it over. It appears their artist just ran the optimize tool till he got it down to half the polygons and sent it off without fixing the mapping or apparently even looking at it. They are very sorry. So are you. It’s 8pm now and you have just spent 2 hours trying to communicate over the phone with the Producer in Shanghai. You are eager to get home. You remember there is half a bottle of Makers Mark on top of the fridge left over from Thanksgiving. You end the call and head home to the comforts of whiskey and Captain Adama’s stoic grace. Things are looking up.
Your elation is short-lived. The next morning finds you staring at the worst looking castle you have seen since the hand-made cardboard one from your high school play on Don Quixote. When the work is still far off after the third revision you do what every Outsource Manager and Art Director has done since the beginning. You pull it in-house. Defeated once and for all, you walk down the long cubicle hall to the artist’s quad and ask your buddy Jeff if he would please build you a castle. You beg. You offer a six pack of Stella and your genuine Macfarlane Metallica action figures in trade. He finally agrees when you throw in your Evangelion series on VHS. 8 hours later you have the best looking castle in the history of video games. Jeff is awesome.
So how much did that castle cost you? $160? Not even close. You paid the Shanghai firm $160, but I just counted about 12 hours of your time and 8 more hours for Jeff who ended up building you a pretty sweet looking castle. Not to mention the beer, action figures and VHS collection. Of course it is tricky calculating what your time is worth, but if we go with the industry average of $10k per man month (which isn’t necessarily what you make, but what many publishers pay developers per head per month) then you paid about $500 of your own time and $750 of Jeff’s time in the making of that castle, plus accessories. Now you can add the $160 you paid to the Shanghai firm. That castle ran you upwards of $1,360. Again, not including your bartering items. It’s also not putting any value on the total, absolute drop in your quality of life due to frustration, headaches and lack of time spent doing what you love, like watching Battlestar Galactica.
These are the secondary costs of outsourcing, which is something many people - especially the ones allocating the budgets - often overlook when deciding on which studio to partner up with. And by no means is this limited to overseas studios. A local studio of inexperienced artists can rack up those secondary costs just as fast as any overseas studio if you’re not careful. It’s understandable choosing to go with a studio that charges less. We do it all the time in our daily lives from tires to phones – we are seduced by the promise that we can receive better quality for less money. Unless you are an Outsourcing Manager or an Art Director and have spent many hours reviewing work from a sub quality outsourcing firm you can’t possibly know the kinds of issues that pop up on a daily basis. Imagine if that castle was a whole level’s worth of assets, or a set of characters. Since outsourcing in games is relatively young, these issues have only existed, or just recently come to light in the last couple years as more and more developers trade war stories and compare notes. These costs have to be considered. Now more than ever. A missed ship date can be the equivalent of throwing your game, and people’s jobs out of a moving truck.
Whether you outsource or localsource your art, keep in mind the secondary costs. How much extra are you willing to pay? Not all your budget is going towards the assets you are purchasing. Often, much of your budget is going to secondary costs, so choose a studio that is least likely to accrue those secondary costs. Time zone, language, quality, and experience are all key factors in accruing or avoiding secondary costs. If a studio has a higher day rate, but can keep secondary costs at a minimum by managing the project and communicating well, it is worth paying extra for.
The CG for Battlestar Galactica was done in Canada by the way. Same time zone as the studio that hired them. It’s good work. Just saying.
Paul Culp is the Studio Director for Oregon-based video game Art and Animation firm, SuperGenius. www.supergenius-studio.com
|
I'm an indie. and have been for the past 9 years. In that time span I've dealt with studios and single artists all over the world and haven't found a simple solution yet. Good art and the right deal is where I find them and I value them whenever I do.
But there's more to it than that. I've had, in the recent past, a truly mind-blowingly terrible experience with Chinese contractors. And not just any random contractors - this was a company that my employer had been working with for years, and had been happy with. It turned out, however, that the items we needed from them for our project were simply beyond their range of cultural understanding. You could tell with everything they sent that they just plain didn't "get" what they were working on. And no amount of explaining ever seemed to do the trick.
It's like the castle example above - an American or European contractor will immediately understand what a good castle looks like. He understands what details contribute the most to the overall look, and what details should be neglected because they're not worth the effort. The Chinese contractor will *not* get this. They do not have castles. You need to explain to them everything bit by bit, so you spend a lot more time communicating and getting upset. Time and again, this thought will run through your mind "damn, I thought this was obvious" - well, it's not obvious to them. To make things worse, you will often get the feeling that the guy you're talking to may well be the only English speaker in the company, and he translates all your feedback into Chinese for his artists. He may do this well or badly - it's beyond your control.
On a sidenote - I used to live in Macau, China, as a teenager. There was tons of stores with pirated music and films (and literally one place in the whole city to buy original stuff). Looking at these pirated works, there was one thing that always, always blew me away: spelling mistakes. The cover art always had spelling mistakes. How this happened exactly, I don't know - sometimes, you saw what appeared to be an exact copy of the original cover, that someone scanned in, added some Chinese writing, and printed out. And yet, there were spelling mistakes in the English. How does that happen? The answer is simple, and illustrates perfectly the kind of problems you can have with Chinese contractors: to them, our writing is exactly as alien and incomprehensible as Chinese writing is to us. So, they don't recognise spelling mistakes. Similarly, they don't recognise which fonts are appropriate when.
These are examples of basic cultural differences that, believe me, have a real impact on the work they produce. They are skilled, hard-working, dedicated, they really try to do the best job possible - but frequently they just don't get the criteria you're working with.
I can understand a name being misspelt because they want the Latin characters to spell the same thing as the Chinese characters - for example, my last name would become Ma-gu-chi (or something along those lines, I don't remember any more). However, when they take an actual English word and change its spelling, then that just does not work. It's not a case of Pinyin transliteration, it's a case of basic ignorance.
Incidentally, the albums I saw, I couldn't even understand where the spelling mistakes came from - when making the cover for their pirated version, they obviously had the original cover to work with. The conclusion is obvious: this had nothing to do with Pinyin, it had everything to do with someone not recognising Latin characters and simply not caring about it.
Jakub's post was a relevant, well articulated anecdote that supported the article's main idea and elucidated the problems of outsourcing with real world examples.
Kevin, your post was a "cultural understanding" back-door-brag fail that adds nothing to the conversation.
Contracts didn't matter, as i found out, because the amount was too small to make it cost effective to hire a lawyer for.
I suffered a little more than $10,000 in damages, not including all the money the game would have made if it would've been completed.
I found a new guy in Virginia, USA and this means that i could at least take him to Small Claims Court if he defrauds me to enforce my contract but i doubt i'd have to since he's into the project and we seem to be developing a friendship.
I hope my story prevents some tragedies in the future.
Words to the wise; Buyer Beware!
Firstly, any remote vendor, in america or otherwise is not going to be 100% familiar with your project and your assets, just because they're 30km's from you, does not mean they have a hive mind like the borg and know everything you are thinking, so the mistakes you illustrate in terms of poly count and concept of what you want are more due to lack of management and poor direction most of the time. I've had many situations where the vendors in china are the ones creating the prototype for games and also the concepts which form the basis of the vision for the game. With good direction anything can be accomplished. Also, if you were to hire someone into your own offices, you wouldn't lock him or her into a room, where you slip instructions under the door, hoping he "gets" what you want. I would assume you would mentor and instruct that person, so maybe try doing the same with your China team, afterall, that's how they should be treated. Talent comes down to the individuals on your team, not the company portfolio, so maybe try a kick off meeting, spend two weeks in shanghai with your team and put your management skills to the test. This has worked great for my studios, I don't know about yours.
Secondly, there's no mention of scalability issues; anyway you cut it, managing 100 character modelers is not the same as managing 5 character artists in your studio. China is not a cloning machine where they make 100 artists based on your genetic makeup, so take some time to understand how the team structure is setup and write your documentation according to that, instead of a set of instructions for yourself and your clone army. Once you delve into that a little, you might learn something.
On that note, splitting a pipeline accross multiple vendors, which is often needed when you need expertise talent, like 100 character artists, no one vendor in china even has that many; you'll find that studio support and organization is critical to the success, spreading this volume of work accross multiple countries, china/india/america only means more overhead, so if it can be isolated and grouped to one set of vendors in one country and time zone, this actually makes things more managable, and is a big factor in where the work goes. So it's not just a matter of costs, it's sometimes just a matter of logistics and scalability. It just doesn't make sense for me to setup 5 studios in china to get 95 character artists, and then set up an additional one in america to setup another 5 character artists.
Thirdly, using the same example, time zone is not as big a hinderance as your article tries to describe. When you have a workforce of 100 artists, who need direction and feedback; you'll be glad that you have the entire day to go through and review all the work, while the artists on the other side of the world are asleep. It's also nice to see the results of your feedback the following day; and just as easily as receiving a bad castle asset, you could also receive and exceptional one.
At the end of the day, great art talent is great art talent anywhere in the world. How we use that talent and how we are able to manage it, is also a challenge, anywhere in the world. So stop blaming the country or the artists, start looking at solutions on how other people have managed to gain great results. I can give you a laundry list of examples of American vendors, who, by my surprise, don't even read the instructions they're given, yet it's in English!
The real secondary cost of outsourcing is actually more dependant on YOU more than it is on them.
My experiences with outsourcing (well, I worked in a company that contracted out some work to Indian companies) so more of my boss' experiences and how I felt about it..
Part of the program we developed (a sort of settings screen) was to be developed by that Indian company. It ended in disaster. The code was very buggy, too complex for what it needed to be, almost unreadable (I suspect they did this on purpose to be able to maintain it themselves) and needed a make-over quickly. It went way overtime and budget and in the end the company was dropped and some of our internal programmers were tasked with restoring the whole thing. This soon proved to be almost impossible since the design seemed fundamentally broken. So some of my colleagues ended up completely rewriting the whole thing anyway.
Needless to say, the outsourcing wasn't a success.
My boss however decided to go ahead and have another Indian company build another project.
As far as I know (I didn't stick around until it was finished) it was once again over deadline and very buggy. However they seemed more solid than the first team and the project looked like it was actually going somewhere/would get finished somehow.
Me and a few of my colleagues who did a code review of that settings screen almost died of laughter when we saw some of the code and comments.
Just my 2 cents.
Working with American companies may alleviate a few if the language and cultural issues. However the the challenge of the subjective nature of art, managing asset schedules, and art directing team members won't go away with an american company.
The take away here should be that there is no magic "outsource" button. There is a cost for all outsourcing solutions. The trick is creating pipelines and methodologies that reduce feedback loops and improve communication across all teams.
I agree, language and cultural aspects are always barriers, but this is also a known and controlable risk, if the results of your quality tests yield the results you want, I assume any project moving forward with a chinese vendor, takes on this risk knowingly. But here's a few tips I've learned over the years to help alleviate some of those headaches.
1) audit the translantaion pipeline - you'll find most of the art workforce has no direct internet access, and the leads have limited english, while most of the work is done through the translation team. So ensure there is a sufficient ratio of translators to art team, and if possible try to get a lead that has english capabilities so they can start the work more earlier as opposed to wait for translation.
2) Find out what the final translation delivery system is. If it's a word doc, that has been cut and pasted together with low res images of your original feedback; that's a warning sign. The art workforce, reads this once, files it away in a folder and often rarely looks at it, this opens opportunity for plenty of human error. It's also hard to track feedback changes this way. In my case, I've mirrored our external forums system, into their internal system, so full translation and history tracking is visible internally and externally. Makes onsite reviews a breeze as well.
3) I find it rare that there are any true cultural gaps (except maybe in management style, but that's another story) that prevent someone from doing proper art. Even the castle example by one of the posters above is a weak argument; by his logic, we shouldn't hire anyone that doesn't have a european background and degree in medievel architecture. Trust me, when I say, there's plenty of people in china that have seen Lord of the Rings, and they know what orcs and elves are, don't forget almost half of WOW's subscriber base is from China. That said, getting past this barrier is simply a matter of good art direction and understanding the background of your art workforce. comparative analysis using visual reference (call out sheets) that the art team is familiar with, will go a long way, and yes you'll need to get to more specifics on what you want, but it's not something you wouldn't have to do if you had an artist in your own studio who wasn't interested in castles either.
In conclusion, I like to find out the details of what's going on. It's easy to generalize that it might be language and cultural issues, and maybe sometimes it is, but in my experience I've found it often has more to do with process. At the end of the day, like any project, you get out what you put in.
2) Find out what the final translation delivery system is (and I assume manage it/accommodate it by "mirror(ing) external forum systems into their internal forum systems") This is more work, more hours = secondary cost. This doesn't support your argument, it supports the argument put forth in the article.
"Even the castle example by one of the posters above is a weak argument..." It's not a weak argument, it's a real world example. It really happened.
Most of the posters here, myself included, have had bad experiences outsourcing. Trying to leverage thinly veiled accusations of racism in an to attempt to... what? Disprove the things we've learned through experience? Well, it's absurd and intellectually disingenuous.
I am not from the gaming industry but in a typical outsourcing scenario in say IT, there is always somebody from the outsourcing firm, someone with expertise in the specific area, who sits at the client headquarters, to understand all the requirements clearly and ensure that they are conveyed correctly to the offshore chaps. Am sure this needs to be done for any firm, whether it is a Chinese, American, Indian or Philipino outsourcing firm, the first step and the biggest chunk of time of the project is spent on clearly articulating the requirements to the vendor. Something that cannot be done as easily as by sending a document through email. If only life were that simplistic!
Ultimately, it boils down to which outsourcing vendor you chose to partner with. There may have been one bad experience with one firm, but that is certainly not representative of the entire non-American outsourcing industry. In fact, the general trend if you see in the offshore industry, is increasingly shifting towards a 'collaborative approach' wherein the outsourcing firm no longer positions itself as just a 'low cost' alternative but more as a 'strategic partner' with influence on the business and even a stake in profits in many cases. How is this even possible if these were simply 'sub quality outsourcing firms"?
Besides, if you end up having to beg the guy down the hall to make you a castle, but you still pay the Chinese outsourcing firm, then something is wrong with your contract!
I work a lot with outsourcing, but outside the games industry, on regular IT projects. What you're describing is the sad reality from all projects I have taken part.
They always ended up being rescued by the onsite team, but since the ones in charge only care for the final result, the project is always a "success".
I wonder when the industry will finally learn that this is not the way to go.
Regardless, outsourcing overseas poses many obstacles that I don't believe are worth overcoming. Not anymore. Why would I send work overseas when I could send it to a local studio, avoiding those issues? It just doesn't make sense. If I have an art budget, I'm going to spend it on a US studio that I trust. If I was in China, I would prefer to work with a Chinese studio. India, likewise. I advocate localsourcing, whole heartedly. If you do the research, there are most likely very reputable, reliable studios in your own relative timezone that can take on the work and alleviate all or most of the issues that come with outsourcing. Take advantage of it. Beyond being practical, you are contributing to your local economy. That is always a good thing.
Aside from the issues with overseas studios, when you have a set art budget, you want your expenses to stay within that budget as much as possible. Otherwise you are taking from your other budgets. You may have $100k to spend on outsourcing, but if you dont pay attention to all the time you and your team are putting into it, you are taking from your in-house budget. By the time your game ships, it all comes out in the wash. You may have stayed within budget on paper, but you'll notice the countless hours spent on "administration" or something else considered non-billable. That needs to be accounted for. A good studio understands this and will want your "administration" time to be minimal. A studio like mine wants your art budget. We want you to spend your art budget with us. But only your art budget. If at the end of the project we find we have taken a substantial amount from a different budget, robbed Peter to pay Paul, then we have failed.