Gamasutra: The Art & Business of Making Gamesspacer
What Went Wrong? Learning From Past Postmortems
View All     RSS
October 22, 2014
arrowPress Releases
October 22, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
What Went Wrong? Learning From Past Postmortems

April 22, 2009 Article Start Page 1 of 5 Next
 

[In this feature, originally printed in Gamasutra sister publication Game Developer earlier this year, EIC Brandon Sheffield and the editorial staff compile the most-repeated 'what went wrong' issues from the development of games -- spanning Rock Band to Final Fantasy XII and beyond.]

Postmortems -- articles written directly by the development team on the creation of a particular video game -- have been a staple of Game Developer Magazine for well over a decade.

With so many authors revealing what went right, and of course the car-wreck style appeal of what went wrong with the development process, patterns start to emerge. It becomes clear that a lot of the same mistakes are being made over and over again, and that some companies are even repeating their own mistakes.

With that in mind, we decided to round up every "what went wrong" entry from the last three years, and compiled the most frequently made mistakes (usually over five times each) into this cautionary feature.

As the saying goes, those who do not learn from the past are doomed to repeat it. So here you have the top 10 "wrongs," in no particular order, over the lifecycle of the current generation so far. No one is safe!

1. Content added too late.

Getting story and features right is difficult at the best of times, but when that content comes in just under the wire, not only does that content suffer, every element of the game that relies on that content suffers.

Titan Quest (Iron Lore, Jeff Goodsill) "We struggled getting 25 detailed design documents to the technical department on time. We were still working on first cut design documentation more than one year after the start of the project, so the technical team had to fly blind in the beginning and some systems had to be redone.

"There was a fairly constant issue of design documents not being detailed enough for technical implementation."

After this trouble, the company devised an approval process, which would have the producer, lead designer, technical director, and the publisher's creative manager sign off on each document before finalizing it. While this helped, it did slow down the company's process a bit, and it's worth noting that for all Iron Lore's good points, the studio is now closed. It stands to reason that a "what went wrong" from a shuttered studio would warrant extra scrutiny.

And to quote Riley Cooper, who had a similar problem with Tomb Raider: Legend, "This clearly serves to remind us that if you don't want any features in your game to under-perform, you need to do them 100 percent or not at all."

BioShock (2K Boston, Alyssa Finley) "We had many drafts of the story over the course of development, but the final draft turned out to be an almost complete rewrite.

"Competing demands for time and resources meant that, unfortunately, some of the important narrative details of the game weren't created until the final rewrite, and therefore required quite a bit of work to retrofit into an existing game."

In an RPG-like game, story is the guiding star of the project. While ideally the systems would be fully integrated into the story to the degree that each can affect the other, at the very least the scenario needs to be nailed down before full-on production. In a game like BioShock, which features loads of spoken dialog, implementation and pipeline problems can crop up at the last moment, if you don't deal with them early on. This is exactly what happened to the 2K Boston team.

2. Communication.

When deadlines get fierce, and everyone's chugging away, communication is most likely the second thing to go. The first, of course, is quality of life, leading to crunch, but you can bet that we'll get to that.

Age of Booty (Certain Affinity, Max Hoberman) "Over the course of the project there were numerous disconnects between the perceived state of the game and the actual state of the game.

"The hardest hit were the designers, who continued fine-tuning plans for sophisticated features like matchmaking and party support long after the programmers had already made huge simplifications (and often cuts) to these systems. A combination of lack of attention to the project, poor communication, and wishful thinking led to the design team believing that several features were far more advanced than we were actually able to implement, and they did not find out the reality until very late in the project."

This depressing quote is made moreso by the fact that this was an Xbox Live Arcade title, and thus had a relatively small team. The company was working on multiple projects at once, which widened the communication gap. But that's another element we'll get to in due time.


Square Enix's Final Fantasy XII

Final Fantasy XII (Square Enix, Taku Murata) "During the development of Final Fantasy XII, the pressure to succeed was at such a high point that we were on the brink of losing control during even the slightest misunderstanding. What happened was our team was given the freedom to make changes at various stages of development, but the adverse affect of this freedom was miscommunication, confusion, and disorder. How work was to be distributed was also often ambiguous, which contributed to the problem."

Management overhead is a particularly large problem in Japan, but that's not to say it can't happen at home, too. Often there are too many managers, but not enough actual management going on. Sound familiar?


Article Start Page 1 of 5 Next

Related Jobs

Zynga
Zynga — Chicago, Illinois, United States
[10.21.14]

Senior Software Engineer (Front End)
Harmonix Music Systems
Harmonix Music Systems — Cambridge, Massachusetts, United States
[10.21.14]

Senior Product Manager
Harmonix Music Systems
Harmonix Music Systems — Cambridge, Massachusetts, United States
[10.21.14]

Web Developer
Cloud Imperium Games
Cloud Imperium Games — Santa Monica, California, United States
[10.21.14]

Marketing Director






Comments


Mark Harris
profile image
I hate to sound like a broken record, because I've been harping on this for a while. Game development needs to be a more managed process, just like other forms of software development.



Too often producers come straight from other positions (programmer, artist, QA, whatever) with little to no project management experience or education. Are we surprised that the major issues in game development stem from a lack of effective project management when there can be literally NO ONE on the entire team who is trained in for such a role?



To all game developers, please please please get your producers some serious project management training, or hire effective project managers as your producers. Your return on investment will more than pay for either.

David Roberts
profile image
@Mark: Whilst I agree with the training + necessity for project management skills, I think getting game dev. in line with other software dev. will be somewhat difficult: Unless the concept is just re-manufacturing a previous one, a serious amount of time will need to left 'slack' in order to ensure the game is any good - after all, unlike functional software, you can't plan stringently for the main outcome of games, which is the fun factor.

Mark Harris
profile image
The inherent experimental and artistic nature of game development means it will never follow the same exact paradigm as purely functional software development.



However, we can clearly see that the most common problems in game development have to do with the proper application of project management techniques. That's the beauty of project management, it is cross discipline. The same management techniques apply in IT, game development, advertising, manufacturing, etc. Any project-driven industry can benefit from effective project management, regardless of content.

Cedric Hauteville
profile image
@Mark: You say : "Too often producers come straight from other positions (programmer, artist, QA, whatever) with little to no project management experience or education."



The lack of PM education is indeed a problem. But, project managers who have never "dirtied their hands" during a previous production are not a good choice either (I mean, people who weren't artist/programmer/designer before taking on the project manager role).



These people are less likely to know how much time does take a task, nor the consequences of feature addition/modification.



I have already worked with project managers who don't even think about delaying the deadlines when the team is asked to add unplanned features, or when a human resource is suddenly affected to another project.

Mark Harris
profile image
@Cedric



Excellent point. I completely agree. There needs to be some education one way or the other, either give your internal promotions PM training (which I prefer) or find competent PMs and take the time to really educate them about the specific process. I'm not arguing just to have PMs in place, I want GOOD project managers in the industry.



A good PM can make up for a lack of specific knowledge through detailed communication with the real knowledge holders.



PM principles are cross discipline, but effective application demands some knowledge of your industry and effective communication with your experts.

Ian Fisch
profile image
My biggest issues with PM's that come from other industries is they get the processes wrong. We had a PM from a telecom company take over a project. We were behind schedule so he tried to streamline the level creation process. The level designer would make a bsp and the art team would immediately begin creating and placing final assets. This was before any of the enemies or scripting was added.



Had he been in the trenches before, he'd have realized what a bad idea that was for the game's quality.

Jacek Wesolowski
profile image
I agree with Mark, but in my experience the "experts" have usually contributed to the problem. Typical development process goes like this:

- we have an idea for a feature, I(n), and that feature has a certain goodness G(n)

- we spend X(n) days implementing feature,

- we review the feature, and have a new idea for the same feature, I(n+1), with new goodness G(n+1)

- we spend X(n+1) days implementing the new version,

- we review the feature, and have a new idea for the same feature, I(n+2), with goodness G(n+2)

- we spend X(n+2) days implementing the new version,

- etc.



This is all well, good, useful and, frankly, hardly avoidable, as long as X(n) >> X(n+1) >> X(n+2) >> ... while G(n) < G (n+1) < G (n+2) < ...



However, when we have the idea I(n+1), we usually don't ask ourselves how much better the new idea is. Obviously, if it's only a bit better but takes long to implement, it's not feasible to implement it, unless a) the current idea is Not Good Enough, b) we have an infinite amount of time and money. It's a PM's job to ask that awkward question, but:

- feature owner will usually overestimate G(n+1),

- feature owner will usually underestimate X(n+1),

- feature owner will insist it is critical that G(n) be maximised, no matter what,

- feature owner will claim the idea I(n) is Not Good Enough, even though they just like the idea I(n+1) at the moment (however, when they have the idea I(n+2), the I(n+1) will become Not Good Enough, too).



The fallacy is twofold. Firstly, the nature of the beast is such that, given a working implementation of idea I(n), feature owner will *always* come up with something that *seems* better at the time. Creativity is addictive. Secondly, the player's (and author's!) "resolution" (that is - ability to say which of any two ideas is better) is limited. In other words, it's only easy to say which idea is better when one is good and the other is bad. When both are "great", or both are "good", or both are "poor", it's hard to compare them.



The inherent problem is that in many work environments a PM can either

- trample feature owner's creativity by simply not letting them have a new idea, or

- rely on feature owner's feedback.



The former leads to unpolished games, because PMs tend to force stop development at inappropriate times. The latter leads to messing up the budget, the deadline, and the 40-hour work week, because most feature owners don't know when to stop.



Complete mayhem and chaos ensue when the management has a creator's complex and starts adding their own I(n) into the mix.

Mark Harris
profile image
Had he been a good PM he would have consulted his team leads and possibly even the entire teams involved to get feedback on the implications of that streamlining.



Again, I think it is a very valid point to assert that PMs from outside the industry can't just step in and start making changes without adequate knowledge of the processes. That's why I prefer to give internal promotions the PM training.



Anywho, the point is that PM techniques can be universal and very helpful, but the application of the techniques is specific to each and every project. That's pretty much line one in the PMBOK guide.

Mark Harris
profile image
@Jacek

I'd sell my soul for a quantitative formula for Goodness. There are several ways to concoct a somewhat quantitative measure, but those aren't the concrete metrics I'd like in my corner.



The situation you describe is a recipe for disaster, mayhem is imminent. Leads with a personal attachment to features, no pre-determined framework for determining relative "goodness", no iterative analysis on ROI for feature versioning, constrained PMs without a reliable information network, the same PMs adding I(n) with their own max(G) delusions.... gives me a headache just thinking about it.

Eric Scharf
profile image
Problems with game development have as much to do with successful leadership as they do with how we, as an industry, advertise to and recruit from the workforce masses (both established and brand new personnel). The core component around which leadership and recruitment revolve is infrastructure; how well we establish and maintain it for our businesses, our productions, and the success of the personnel working within.



If we advertise life-altering excitement to freshly-graduated, incredibly eager newcomers, without telling them "what else they have won," we are doing neither ourselves, nor them, any good favors.



If we attempt to recruit successful project managers, from the general software development sector (artificial intelligence / mainframe / office productivity / networking / security, etc.), without sharing with them how unique the production pathways are within entertainment software development (or asking those candidates to explain their true grasp of game development processes during an interview), then, we are, once again, poking ourselves in the eyes.



The success of game development personnel, from all rungs on the ladder, and the products resulting from their own blood, sweat, and tears, even then, rely less on their talent and far more on a solid business approach and infrastructure. If your infrastructure involves a short-term foundation that washes away in the face of hardcore, long-term planning, then, it had better be “by design,” where your preferred business model is a highly mobile one, with shallow overhead. Otherwise, you will have simultaneously ruined one or more products and, potentially, the livelihood of one or more people.



There is a phrase used in the National Football League to describe coaches, players, and teams who have achieved some success and “look great on paper” but not so great between the hash marks, and that phrase is “paper champion.”



Even a quick profit business, these days, requires a good infrastructure. In fact, you want a solid infrastructure which, regardless of your business goals, encourages and supports forward-thinking leaders and resourceful production personnel who have been in the trenches, have talked the talk, and can walk the walk as a result. A paper champion, without a legitimate infrastructure to limit the fallout, will only get soggy and wilt when the blood, sweat, and tears begin to flow during production.



Studio longevity, project quality, and new-and-existing personnel (without the support of proper infrastructure, proper leadership, and proper recruiting) all pay the price, in large and small chunks, until only crumbs and a failed effort remain.



There is only one scenario, in acquiring the right personnel, where there may be little-to-no operating room: your 800-pound publisher, or heavy-handed private financier, has explained to you that the project must begin pre-production by X date, full-production by Y date, and reach gold master by Z date. This explanation has been deemed non-negotiable, with a subtle suggestion that dates will prove more important than quality.



Being forced to choose from a field of candidates who do not meet even the baseline position requirements (with the given that such standards are not Herculean), for either leadership or production positions, is like being asked to choose between swallowing a glass of "100 proof" hydrochloric acid or decapitating yourself, objectively speaking, of course.



This "do it or else" scenario is not rare, and, unless you have the righteous, big picture nerve and / or financial comfort to tell your publisher / financier to "take this short-sighted process and shove it," you must tie on the darkest-and-thickest blindfold you own, throw your sharpest dart in the direction of heavy breathing and dancing hoof noises, and, then, just listen intently for the loudest HEE-HAW. The candidate who suddenly asks for gauze and some ointment is your new employee, like it or lump it.



Just imagine all of the times game development studios have furnished incomplete job descriptions to HR managers and external personnel. The chances of removing that blindfold and seeing an actual donkey have suddenly improved 100 fold.



We can all do better than this, and many of us have acknowledged as much at one time or another. There will have to be a few pioneering companies (that comprehend but are not tied to current games industry business standards), with non-traditional funding sources and a well-planned long-term infrastructures, as well as several bright, proven industry veterans who have that righteous, big picture nerve, in order to solve the production and game development problems of today for a more pleasant, productive tomorrow and well beyond.



There is delicious irony, however, in that this day of April 22nd, 2009 is Earth Day, which represents the annual opportunity for most people on the planet to attempt to adopt a more energy efficient way of living lives and operating businesses.



We, the people of the games industry, have our own Earth Day, as it were, and it just happens to be every day. We have the opportunity, every day, to enhance the infrastructure of our companies, plan our projects better, and, in turn, remove many of the job stability limitations and performance roadblocks placed on us and our products. Every day that we delay such achievable and necessary improvements is just one more day that we prevent our “world” of game development from moving forward as a leading industry, and moving away from its history as a step-child off-shoot of children’s toys.



If you are up to the challenge, I offer a few infrastructure-based suggestions and potential solutions, in support of "Games Industry Day," in a 6-part article from 2008, entitled "Transforming the Games Industry Into A Well-Oiled Machine" and located at http://www.emscharf.com/blogosphere/genuinearticle/genuinearticle
_2009/genuinearticle_2009_0017.htm.

Priscilla Elfrey
profile image
A thought. Consider project managers with an entertainment background, training and experience as film or video producers, for example, who have a disciplined approach to planning within a more creative framework than many software developer project managers. A short film or documentary producer on a tight-budget but with lots of talent to keep both energized and in check may be the answer. For one thing, that background is strong on rigorous early multidisciplinary planning to clarify criteria, explore options and prioritize. This reduces a lot of late and expensive changes. It fosters innovation. Software engineering--especially in engineering titlted organizations, tend to start with requirements and give insufficient attention to the getting the story sketched out, discussed and framed. Pays off...

Sean Farrell
profile image
I am a professional software engineer and a hobby game developer. When I hear game developer talk about their games, you often hear things like "games can't compared to traditional software" and such (see above). Although end product is largely different the actual project management involved is similar; traditional software projects also suffer from the exact same problems outlined in this article. The difference is that the somewhat faulty software runs on the nuclear power plant and oil refinery you drive by every day to work. (No kidding...) Game developers call it fun, traditional software developers call it usability and robustness.

Eric Scharf
profile image
"The difference is that the somewhat faulty software runs on the nuclear power plant and oil refinery you drive by every day to work. (No kidding...) Game developers call it fun, traditional software developers call it usability and robustness."



Good point, Mr. Farrell.

[User Banned]
profile image
This user violated Gamasutra’s Comment Guidelines and has been banned.

Stephen Etheridge
profile image
@Eric:



You have made some very interesting comments here, but your website is a navigational nightmare. You might feel you have something crucial to say, but unless you work on presenting them in a more accessible manner, nobody is going to be inclined to hear it.



Please for the good of the Industry, redesign your site. How is anyone supposed to know that all your games items are blogged under 'The Genuine Article'? Not to mention the bugs.



And the DCMTLRA stuff in your portfolio may be functional, but who wants to waste time referring to your rubric when they could read a straightforward portfolio and know exactly what the person's talking about?



I don't mean to be harsh, but sometimes directness is the only way to get the message through. Your self-titled 'visionary' blog looks as if it could benefit from an outsider perspective. Fix the layout if you expect people to start taking you seriously. I want to read what you have to say, but I'm not going to wade through your esoteric layout to take tips from a creative expert who can't get his own website right.

Eric Scharf
profile image
@Stephen:



The Microsoft FrontPage "disaster" that is my site has officially been exposed! I admit it, and I humbly fall on my HTML sword. Yes, yes - it is the user who is at fault, not the tool.



I have been putting off a site redesign for about four years now, Stephen. In fact, I am, right now, staring down a beautiful copy of Adobe DreamWeaver 8 that has just been dying for attention. Slightly higher priorities have, unfortunately, gotten in the way. Things tend to slip when you have a family to distract you with life . . . but that's life.



Thanks to your outing of my maniacal rubric, however, the wheels of change have begun to slowly trudge into motion, with some amazing torque, too.



Ever since my site was first established, sometime around 1999, I have received relatively-and-proportionately few complaints about my site's complex structure.



I have also learned, however, that several ADHD sufferers have not had an enjoyable time with the site, one close family member included. I do not suppose . . . ? No, that question would be in poor taste.



In any event, for those who have difficulty navigating my wicked spider web, the following alternatives have been available:



http://emscharf-the-genuine-article.blogspot.com

http://emscharf-the-media-magnate.blogspot.com

http://emscharf-the-tortured-cowboys-fan.blogspot.com



And, if you are into something more like books-on-tape, you can hear, quite literally, a few of my articles over at the up-and-coming http://www.industrybroadcast.com, where site owner, Ryan Wiancko, has made audio recordings of every one of the 100+ articles on display.



Now, Stephen - unless you reside outside of the U.S., I will tell you, from personal experience, that navigating through my digital swamp is not the best thing to be doing at 4:00 AM, PST, unless your body is processing a six-pack of Mountain Dew or Jolt Cola.



LOL! Digital Swamp - get it? After all, I do reside in Florida.



In any event, for anyone who has been following this thread or feels the urge to perform another flame job on how painful it is to navigate my site, feel free to contact me directly at emscharf@emscharf.com or 443-797-2236. I am all eyes and ears.



Otherwise, I am left wondering from what kind of person I am receiving such passionate love letters: An overtired and angry young person with an axe to grind? A well-established adult who's specific navigational requirements must be met before being furnished with any reading material? Both?



Nonetheless, Stephen, let there be no doubt that you are correct in picking on my "hooptie" site design. It is time for an infrastructure update - my favorite kind, of course.



Regarding my project credit displays, or DCMTLRA, anyone who has spent significant time interviewing game development candidates, and building quality teams, wants that kind of information, up front. Gravy-training on someone else's coat tails remains as prominent today as pirated software, and you can leave no stone unturned.



While I am known to be a bit of a wise guy, I have no ego to bruise, and, thus, I have no problem making such project credit information available without question, or having to be asked. Could the delivery method be improved? Certainly. Otherwise, we can agree to disagree on this detail.



Moving forward, I encourage you to leave your cheese out in the wind as well, so that I can review your background, too. After all, Stephen, I know so little about you, and I am intrigued to learn whether or not you can eat what you have served.



And, Stephen, while I genuinely appreciate the opportunity to steal some thunder away from Brandon's article (and the original focus of this discussion thread), let us get back on target out of respect for Brandon, shall we?

Raoul Duke
profile image
hey, actually, i kinda hope everybody continues to think that game software is some unique thing that can't be managed and gosh we could never learn anything from other software engineering domains, so they can continue to cock it up, so i can continue to have a chance to beat them in the business.

Aaron Casillas
profile image
From my experience, there are many different experiences out there believe me, it seems that the culprits of problems are really entrenched in the Vision and Execution of the game. There are many methods for checking a game design before going wide and I implore any publisher/dev to get the following done first before green lighting horizontal hiring. Besides choosing the right tech, here are some general guidelines:



1) Have a strong Leader with a driving Vision that can judge/razor and id the conceits of this IP in game form! Committee’s might be great, but even a Jury has a Foreman.



2) Cost of Feature Entry= what are the core features of this game versus its competitors? Do not re-invent the wheel, just straight copy these core features. How much time are you going to need just to be competitive? Some other team spent a lot of money getting their game to X point, use their experience!

A) What new features do we want to add? What is innovative and creative?

B) Use a Cake production plan (we have a core lets build it) versus the Onion peel dev plan (we don’t know what it is until we peel away all the layers).



3) Strong Narrative= get your narrative straight at least in a first pass form, this helps inform what types of spaces and factions you will need to create. Offline prepare a plan for your vertical slice, collect ideas for scenarios. Big Read Art Concept work that supports the vertical slice should follow! Collect images from other forms, IP’s and licenses.



4) Identify your Compulsion loops= why is this game “sticky?” What is the player going to be doing, meta, core and micro?



5) Reverse Design, create a GCD and or a Game Manual. Yes a Game Manual, the only things that are going to ship with the game is the box, the dvd and the game manual. Start to razor your vertical slice and create a “packet GDD.” Do not determine the quality of your game by the weight of your GDD, that’s just surreally (sp?) silly.



6) Create your vertical slice and get informed. Is this what we want? Get your metrics done! Anyone joining the team should understand your game by playing the slice! Be honest.

A) Another great step before creating a vertical slice is to create a tone video. 10-15k its much cheaper than spending months on a slice that’s for sure!



In general, most production go bad when:

a) The team doesn’t know the game because a vertical slice was never created or no one is honest about the resulting game or its metrics.



b) the Science Project…the technology from the previous game is going to be re-done, this costs a lot of money, time at the cost of a better production. Re-inventing the wheel or the deadly “Science Project” should be identified very early on and placed in a high risk category.

If your licensed engine is a 3rd person hallway shooter, then weigh the risks of changing it.

If your licensed engine lacks any tools, then get worried and nervous

If you are making an mmo, um choose wisely, very wisely.



C) “Making games is a discovery Process through the unknown miasma of features” puhleaze! if this occurs, you needed a Design Leader who understands spatial relationships, numerical relationships, rule relationships a blood and guts person who has been in the trenches. If this is also occurring, I’d take a strong look at your Creative Leads, from EP to LD. Money is burning here…(I went to visit a dev a while back that went from working a FPS to changing their game to a “FPS with no first person shooting” because the writer wanted to establish a more emotional bond with the player and other actors…ok, stop, put the plume in the inkwell and run.)



D) The game is the object, your team’s feelings should not be hurt when you critique it, but always be respectful of the time people have put into their work and ask them were they plan to take it; make sure their execution is in line with the vision of the game.


none
 
Comment: