|
[In this Game Developer magazine article, designer Ara Shirinian (The Red Star, uDraw) takes a look at two years' worth of magazine postmortems to mine them for common data points -- examining both what went right and what went wrong how often, and why.]
After having participated in numerous game projects throughout my career as a designer, including many failures and successes, I noticed that certain types of development mistakes appeared to recur with surprising frequency. Was this just another Twilight Zone-inspired idiosyncrasy of my career? Or is there something more going on here?
I imagined a development hell where throngs of teams all ran into the same pitfalls over and over, without knowledge of what they were doing wrong, and without the realization that anyone else might be making similar mistakes.
As developers, we have our own career histories to depend on for knowledge and experience about the rights and wrongs in game development. Beyond that, we can only rely on other developers' willingness to relate their own experiences, mistakes, and solutions to us.
To that end, things like postmortems, conferences, and just plain open discussion amongst developers are great tools.
But these are only vignettes in a sense, and often highly contextually dependent. For example a team's troubles with Lua integration are not helpful if you never use Lua.
Beyond that, I was curious whether there was a bigger picture, what it looked like, and if the results could help us learn something new about game development in general. That, in a nutshell, is the motivation behind this analysis.
Collecting Data about Game Development
Without any expectations about whether I would find any interesting trends or commonalities across game development projects (or whether I'd even find anything coherent in the first place), I decided that Game Developer's extensive history of postmortems was the best and most consistent source of data.
Conveniently, postmortems written for Game Developer all share a similar structure: the developer writing the postmortem is required to select and illustrate five things that went right during the project, and five things that went wrong. Despite the enormous range of project types and sizes, this made it relatively straightforward to collect and organize data about the good and bad things that were reported about projects.
The data set for this analysis consists of 24 successive postmortems published in Game Developer, covering a period of two years, from articles that were published from February 2008 to January 2010.
Collecting Data about Common Issues
Before data collection began, I defined a set of various issues or situations that I thought would be interesting to track across all postmortems. For example, whether the postmortem mentioned using scrum or some agile process, whether a successful experience with outsourcing was reported, and so on.
For each of these items, each postmortem was scoured for mention of that issue or situation. I didn't just search for specific words; collecting the data required a lot of extensive re-reading to ensure that if the postmortem was counted in a certain category, there would be no question about its inclusion.
Results Part 1: Postmortem Metrics
Here are some metrics that characterize the projects whose postmortems were included in this analysis.
Team Sizes Reported
The largest project was Far Cry 2, boasting a team of 265 members. The smallest projects were Age of Booty and n+, both reporting just five team members. Four postmortems reported a range of team size; of those, the maximum was recorded.
Two postmortems did not report a team size and were excluded from this section. Team sizes tended to fall into four clumps, with the smallest teams consisting of five to 10 members. The next five largest teams consisted of 30 to 40 members. There was another clump representing 68 to 90 members, and six projects reported over 100 team members.
Multiplatform vs. Single-platform
Slightly more than one third of the projects were released on multiple platforms.
Platforms Represented
Multiplatform projects are counted once for each platform they were released on (there were no PSN games represented in this selection of postmortems).
Development Time
The average development time for projects was 2.4 years. Two projects did not report a development time and were excluded from this section. Interestingly, development time also seems to clump at yearly or half-year marks, although this is probably an artifact of how durations were reported. There is also a large jump between the two-year mark and the three and a half-year mark.
Use of Contractors or Outsourcing
Eleven of 24 postmortems reported utilizing contractors in the project. Only 7, or about 29 percent, reported outsourcing work to separate companies.
|
Printer-Friendly Version
Also, keep in mind that anything related to the actual sound engine or rendering tech was most likely collated as programming.
I think the article very clearly states the *importance* of process / production.
http://www.gamasutra.com/blogs/EthanMollick/20110706/7916/Why_Producers_Matter_S
tatistically_More_than_Designers.php
My conclusion would be that producers (and managers) are perhaps "more significant" than any other position because most of them are woefully underqualified for the job and a less-than-competent producer has a disproportionate negative effect in the whole process. This is usually not the fault of the individuals themselves; inadequate training and staffing is pervasive to the management discipline in all fields.
Portal and Farmville both took design seriously from a production perspective. Portal had the famous continual iteration and play-testing while Farmville (also famously) does their "try it on a million people and see which profits more" approach. Product design (in all aspects) is important, and production is an important facilitator. It can also screw these things over terribly.
Did you do any analysis of correlations between what went right and what went wrong in projects ? I.e. was there a trend that if production went well, design went wrong or some other correlations ?
Also: I would intuitively guess that production/process is more important for larger projects, but did you see any correlations to support or deny that ?
With regards to your second question, you may want to look at my earlier reply above in the comments.
Processes and planning are always important, but there do appear to be unique kinds of planning and process challenges in 20+ person teams that do not really surface when teams are very small, say 6 people and under. I'm not saying this necessarily based on anything in my analysis, although the results do appear to be consistent with this idea.
A couple of thoughts:
* Production management is probably disproportionate from the other areas, because all areas are affected by it. If there is a design vs art conflict down the road for instance, who gets the blame - design, art or production? Interestingly, if you assume that production is just half of any part of development, the numbers actually work out pretty well... though I wouldn't read the numbers like this.
* The role of production is in large part to avoid problems. It is very difficult to track the absence of problems, so this may put some extra perspective on the "what went right" percentage of production.
If a good design decision is made early, it will be evident later in that the game is fun, etc. If a good production decision is made early, it will not be evident later (ie. "we saved a month of programming because of this" - how can we tell?.
* About late changes and extended deadlines - I find that this is often (at least in my experience) a function of the developer - publisher relationship. The developer has to pitch a great game to the publisher to secure the necessary funding, and so tends to promise everything they can. Of course, people are optimistic, and when unforeseen problems arise, features have to be changed or deadlines have to be extended.
On the other hand, the publisher wants the best possible game out of the developer, and has a lot of financial power to push extra features. This usually happens late in the process, when the game is presentable enough for greenlight presentations with executives... and so, features get introduced late in the process.