Gamasutra: The Art & Business of Making Gamesspacer
Waterfall Game Development Done Right
arrowPress Releases
July 22, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
Waterfall Game Development Done Right

November 21, 2012 Article Start Previous Page 2 of 3 Next
 

Build a Rough Order Magnitude Estimate (ROM)

If we feel comfortable moving forward under a fixed priced contract, it's time to do the second analysis of how, and how much. We will also do the first analysis of when. We use Excel.

First take a look at your set of features and implementations. It's time to define some milestones. Milestones are waypoints towards your end goal of delivering the product. A good milestone is measurable and defined by the outcome of your work. Write these down as eloquently as possible. Hopefully, you can describe 90 percent of the milestone in one sentence. A detailed description of the milestone is best described as a test plan written by QA. If a developer can test their work before it gets into the hands of QA, the less time you will need to iterate.

Milestone Definition: The first play experience of Level 1.

At each milestone, think about the steps you will need to take to reach that outcome. Look at your implementations and segment them in a way that puts risker items earlier in the development process. Why? Because if an implementation goes awry, you can tell your customers as soon as possible so that you can work together to resolve it. A lack of time is your biggest enemy when things go wrong.

Many project managers forget the following tasks. Set aside time for the following:

  • Shims (scaffolding code required to stand-up an implementation or increase testability and efficiency)
  • Refactoring before milestones build on new code foundations
  • QA
  • Reactions to QA (why find a bug if you aren't going to fix it)
  • Usability
  • Customer Feedback and correction time

Now that you have your milestones, write them down. Add in the implementation tasks. Use a hierarchy to help you organize work and decompose as necessary. A simplified snipped of a ROM might look like the following:

Milestone

Estimate

Confidence

     

Milestone 2 -- User Kills a Simple Enemy

   

Art & Design

   

Build Level

32

9

Build Initial Player Character

80

8

Build Initial Weapon

40

8

Programming

   

Implement Player Damage

32

7

Implement Basic Enemy AI

16

6

Implement Enemy Damage

8

8

Implement Enemy Death

32

7

A snippet from a ROM. We prefer to use estimates in hours, but sometimes days makes more sense.

For each line, develop an estimate. But before you start, we need to think more carefully about what an estimate really is. At GarageGames, we believe an estimate is a time and risk estimate. Let's take a second to really understand how GarageGames determines risk adjusted estimates, it's critically important.

Note: When estimating, we always use work effort which is the total work required to complete the job. Duration measures how much calendar time it will take from start to finish. For example a 16 work effort hours can have a duration of 8 hours if two people are working on the task.

Risk Adjusted Estimates

Let examine two estimates: Task A is a non-risky job you've done before. Task B is a risky job you've done before. Assume for a second that I've asked the engineer to give me an estimate for each and in both cases, they say two weeks.

Beware: In this case, two weeks does not equal two weeks.

More often than not, undiscovered work will cause job B to take longer than expected. It's likely that two weeks REALLY means: "two weeks if everything goes the way I hope it will." It's probably more accurate to assume that the job will take two to four weeks. Since we don't really understand all ramifications of a risky task until we do it, we really can't be very accurate. Some people pad the estimate; we prefer something a bit more pragmatic, descriptive, and standardized.

With every estimate, GarageGames uses confidence to determine a risk adjusted estimate. So if we include confidence numbers, the two estimates might look more like this:

Task A: 2 Weeks, Confidence: 9

Task B: 2 Weeks, Confidence: 4

To calculate the risk adjusted estimate, GarageGames uses the following function:

Risk Adjusted Estimate = Estimate + (Estimate*(10-Confidence)/9)

Task A: Risk Adjusted Estimate: 2.2 Weeks

Task B: Risk Adjusted Estimate: 3.3 Weeks

Having the confidence value represented is useful for more than just determining risk adjusted estimates. As stated earlier, finding ways to reduce cost for a project is something that we openly embrace. Low confidence numbers are expensive and clients are motivated to help you increase confidence numbers to reduce cost. Before writing a single line of code, you can reduce risk the following ways:

Cutting a Feature -- "We didn't need it anyway... It's not that important."

Constraining a Feature -- "What if it did half of what we were hoping for."

Defining a Feature Better -- "Oh, I didn't understand what that meant... That's not so risky now."

Rapid Prototyping - Allowing time for rapid prototyping early in the project (and then updating the estimate ex post facto) --"Eureka!"

This is a great time for leads to share estimates with those who need to actually implement the task. It's hard to have developers who are vested in delivering a project if you didn't consult them initially. They are usually quite motivated to help. It's during this phase that implementers will ask you, "What does this task actually mean?" You will need to work with them and the customer to define what the implementation will or won't do. Save this definition, it will be important later.

Now that you have all the estimates, confidence numbers, and risk adjusted estimates, you should be able to provide your first estimate of when the project will be complete. Add up the risk adjusted estimates and divide it by full-time employee team size (two half-time employees equal one).

This isn't a very accurate estimate because it assumes a perfect world where everyone can do every job perfectly parallel -- hardly the case. But at least you will have some data to help you understand the very best case very high-level ball park figures. If someone wants a task done in 6 months and your estimate says you'll be done in seven months, you'll know early that something needs to change.

Same goes for the risk level. Average the confidence numbers. If the confidence numbers are too low (below, say, a 6), then maybe it's time to suggest either 1) moving to a time and material contract or 2) doing a small fixed bid contract up front to rapid prototype in an effort to bring up confidence numbers.

Keep in mind that while you are working on estimates, you will be defining the project or asking questions that will lead to the definition of the product. Write them down in small one or two page descriptions and save them. You will use them to develop the Statement of Work.

The final step is to share it with your customer. Show them where the risk is high and tell them why. Discuss opportunities to save money by reducing risk.


Article Start Previous Page 2 of 3 Next

Related Jobs

Vicarious Visions / Activision
Vicarious Visions / Activision — Albany, New York, United States
[07.22.14]

Concept Artist (Temporary)-Vicarious Visions
Treyarch / Activision
Treyarch / Activision — Santa Monica, California, United States
[07.22.14]

Cinematic Animator (temporary) - Treyarch
Treyarch / Activision
Treyarch / Activision — Santa Monica, California, United States
[07.22.14]

Associate Cinematic Animator (temporary) - Treyarch
Activision
Activision — Seattle, Washington, United States
[07.22.14]

Software Engineer - Activision Seattle






Comments


Eric Preisz
profile image
I don't think I would advocate getting paid per day (perhaps there is some confusing text in the article). We build estimates on the granularity of a day. Payments tend to be on delivery/acceptance of milestones.

Clinton Keith
profile image
I fundamentally agree with the Eric on the gist of this: why employ purely iterative and incremental development approaches when there is little uncertainty.

However, a problem common with comparing agile and waterfall is the labeling. Few, if any, studios are purely "agile" or "waterfall". They are more mindsets that encompass a wide variety of practices and approaches to development. Labels are convenient for helping make an argument, often with cute little straw-man statements to help reinforce preconceived notions.

In reality, its more of a scale from pure iterative, inspect and adapt practices to assembly-line approaches that need little inspection. Studios have *always* been somewhere in the middle of this scale. Individual practices are employed based on uncertainty. To make the argument that "waterfall works" fails here. Sure, *some* waterfall practices work when we know more up front. You might say that you don't need any agile practices, but that's like saying you'll never need to inspect what your making and how your making it...not a state that any game developer has been in for very long, in my experience.

Anyways, I love the confidence scale. I'd love to try doing this on a project with some leads using planning poker! Thanks for writing this Eric.

Eric Preisz
profile image
"However, a problem common with comparing agile and waterfall is the labeling. Few, if any, studios are purely "agile" or "waterfall". They are more mindsets that encompass a wide variety of practices and approaches to development."

Well said.

However, I do think that the popularity of Agile methods has somehow colored developers and diminished the discipline and acceptance of prior planning when possible. The pendulum has swung too far in the belief that all development require a reactive only project management process.

Clinton Keith
profile image
Agreed Eric. I call those projects "Incremental and iterative death marches" ;)

Chris Toepker
profile image
Eric, the confidence scale is a thing of beauty, putting into words things I think every day. Thanks for that!

As for Waterfall, what I think is interesting is how we (as the 'new industry'?) seem to want to invent new words all the time. I've found working in manufacturing the approach is just called "project management." Not as sexy a name perhaps, but when done fully and professionally its about about as waterfally as you can get, covering the scoping and quoting and resourcing and etc. questions quite well.

And when an "agile" approach is needed, a design phase is added. In other words, you don't commit all the resources until you really know what you want to make...whether it is some fun (in our case) or stylish (every bit as squishy, no?!) or ergonomic or what-have-you. Don't know what I mean? Check out: www.pmi.org/, especially: http://www.pmi.org/PMBOK-Guide-and-Standards.aspx

James Hofmann
profile image
I think the essential difference between manufacturing and software is that hardware's physical constraints and lead times produce a lot of known-quantity requirements and deadlines that aren't negotiable, while software allows everything to be negotiated, so experienced and well-managed practitioners can preserve "any" amount of desired flexibility into the end of the project, at the cost of weighing down development of individual features.

Tom Kofoed
profile image
Dont blame Water Fall or Agile/Scrum
Blame the the person who use a method incorrectly or to religiously (Rigid).....
:)

Eric Preisz
profile image
@Chris - Originally, I used a 1-10 scale of "risk" instead of "confidence". I found over time, it was easier to get people to give more consistent results using the later. Collecting confidence is all about managing and quantifying risk. An estimate without a risk assessment is missing a lot of valuable detail.

Jeff Murray
profile image
Great article! Nice to see an alternative view on this subject, for a change.

Speaking from my experience, I've worked with both Agile and Waterfall with games programming teams between 4 and 13 programmers. Although we saw better engineered products with Agile, we ended up with better (more fun) games when we used a more Waterfall-based approach. Was that the methodology, the team or the game design? It's hard to say- these are organic products at the end of the day! Given the choice, I'd try to aim somewhere between the two and see if it is possible to balance them out.

We did risk assessment in a similar 1-10 scale, but found that without a strong PM on the project, they were difficult for clients to understand / deal with. Clients would still ignore them, turn tail later in the project and go ahead regardless!

Christopher Lambert
profile image
Over 18 years software development experience here, my feeling regarding process is quite simple - there is NO process until you understand the company, the project, the people and technology at play. Agility is about selling books for the most part, as is Waterfall. In my opinion you should be examining the situation and developing a strategy on a case by case basis and using aspects of all processes as a tool chest from which you piece together something to suit the needs of a given project/company/team/technology.

In my case, I find Boehm Spiral Model to still be the 'best general fit' after all these years and frankly see no advantage of waterfall or agile over this approach 'in the general case'.

Svein-Gunnar Johansen
profile image
I would argue the following: If there is a place for waterfall, it is in building the first iteration of your project: The "walking skeleton". And only if this can be built with high levels of confidence.

Robert Riedl
profile image
A third scenario to use 'waterfall' would be when you've created something very similar. In this case, you can use your actual time from the previous project to set up your schedule.

Speaking of actuals, I'm curious how accurate your risk modifiers were. Sometimes estimates can be too conservative and adding even more padding can unnecessarily increase your project cost estimate.

Richard Ranft
profile image
Having actually worked for Eric I'd like to say that as long as everyone is honest in their assessments of their tasks this seemed to me to be fairly close on the average. As always, we have to remember we're "estimating" and probably won't have a number that lands within minutes of the actual task duration.

Nickolas Anderson
profile image
This was a good read and very insightful. Thank you, Eric. The commentary surrounding it has been poignant as well.

In my opinion it is best to be pragmatic about methodology and to not become wedded to a single approach.


@Eric: The Confidence Measurement is fantastic! I'm going to have to try that.


@Christopher: The Spiral Model is an interesting visualization of the development process. I hadn't encountered it before. Thanks for sharing that.

Brian Dreyer
profile image
I’d defer to the “Agile Manifesto” or the 12 Agile Principles:

1. Does what we’re doing at this moment support the early and continuous delivery of valuable software?
2. Does our process welcome change and take advantage of change?
3. Does our process lead to and support the delivery of working software?
4. Are the developers and the product owner (stake holders) working together daily? Are customer and business stakeholders working closely with the project team?
5. Does our environment give the development team the support it needs to get the job done?
6. Are we communicating face-to-face more than through phone call and email?
7. Are we measuring progress by the amount of working software produced?
8. Can we maintain this pace indefinitely?
9. Do we support technical excellence and good design that allows for future changes?
10. Are we maximizing the amount of work not done- namely, doing as little as necessary to fulfill the goal of the project?
11. Is this development team self-organizing and self-managing? Does it have the freedom to succeed?
12. Are we reflecting at regular intervals and adjusting our behavior accordingly?

The big issue with Waterfall is change – the team/people, features, tech, etc., true Agile deals with this better, quicker, cheaper and allows you to kill what needs to be killed faster.

Agile, in my humble opinion will never work without a dedicated ScrumMaster and that’s not (always) a traditional Producer/PM – especially when there's Jr. people on the team.

Eric Preisz
profile image
@Brian - I don't see anything in the Agile Manifesto that we don't apply to our practices. Although, from my experience, I've seen #9 overrule #10 more often than it should...but that's another article.

Dan the gaming Guy
profile image
I like having a macro waterfall schedule that breaks project features / efforts up into buckets (this ensures your project is not a run away train of endless itteration), within each of those time buckets use an agile and fast itteration process to accomplish desired outcome of each part.

Also ensure to schedule a very healthy polish (around 25-35%) phase at the end of the project to:
1) Reduce risk of over scoping your product by having a shorter production phase
2) Create some buffer if youre budget is reduced, your asked to bring your dates in or if something else goes wrong with staffing/hiring/planning, etc
3) If you avoid the above scenarios, you should be able to create a highly polished product

"If you can make every aspect of your product good, your result will be great".


none
 
Comment: