|
Features

Manager In A Strange Land:
Milestones Considered Harmful?
We didn't have milestones for Spider-Man 1.
Well, we did
we had a schedule of month-by-month deliverables
with lists of items that were supposed to be delivered by when.
But we rarely looked at that document during the project. And nobody
from upper management ever said, "Why isn't item A, which was
supposed to be done this month, not done?" And yet we still
managed to ship on time.
In Critical Chain, Eliyahu M. Goldratt suggests
that one of the things that makes projects go south is the padding
put in between milestones. Because when you pad a deliverable, and
you finish that deliverable early, you don't move on to the next
deliverable. Instead, you polish that deliverable beyond what it
needed. Gold-plating. Tom DeMarco says the same thing in Slack.
DeMarco and Goldratt say that instead of having deadlines,
we should have optimistic goals. We will almost always miss these
optimistic goals. But we won't lose any time to gold-plating.
But what if something goes wrong? (Which happens over
and over and over on a game project.) If you make optimistic goals,
you'll probably miss them. That's why, instead of putting the padding
between the milestones, you put the padding at the end.
Most game development contracts have this padding
built in, in the phase most of us call "alpha". We try
to finish the game early, so it can get into testing, so we can
fix the bugs.
I hadn't actually read these books when we were working
on Spider-Man, but I think they help explain the phenomenon
we saw, which is that you don't actually need milestones per se
to ship a product.
So how big should that pad at the end be?
I've frequently seen milestone schedules that factor
in one month for "alpha" and one month for "beta".
(The difference between "alpha" and "beta" was
never really clear.) I don't believe this is enough. This might
be enough time to fix the bugs in a twelve-month-long project, but
not for anything longer. This padding isn't just to fix a few lingering
bugs; this padding also gives you some slack so you can miss your
target and still ship on time.
Also, the number of undiscovered bugs that are introduced
during the lifecycle of a project is proportional to the length
of the project. With Spider-Man, an 18-month project, we
tried to be content-complete four months before the ship date. (Although
we slipped; the last in-game cutscene limped in with only a month
left.) Still, the bulk of gameplay content was finished within a
month of that target date, so an early alpha served its purpose:
giving us room to slip. We even had enough of a comfort zone in
those last few months that we could add a few small features.
What if we had a cast-in-stone milestone schedule?
When that first set of levels was supposed to come online, we would
have missed our planned date by a great deal. In addition, we probably
would have decided to cut the amount of game content in half --
or more -- to get back on schedule. So we would have created a smaller
game. (And it was pretty small to begin with - some reviewers said
it was too short.)
Also, we would have lost flexibility in our development:
the opportunity to find better, faster ways to do things can be
lost if you don't have some slack time.
There are downsides to the "goals, not promises"
method of milestones. First, it hurts morale when you keep missing
your goals, so you need other methods to keep morale up. These morale-boosting
exercises include telling everybody how awesome they are all the
time, and "love ins" where you show the game to the team
and show the progress that's been made lately. The other downside
is you almost never punt. When you're approaching a cast-in-stone
milestone, and you're about to slip, you can alter the deliverable
to achieve the milestone goal. For example, your publisher says
you have to deliver two levels, but you only have one, so you cut
that level into two parts and hand it over. That's an example of
bad punting, but sometimes punting can be good. You reduce the scope
and deliver something to spec, even though it's not what
you originally wanted to deliver.
Finally, "goals, not promises" milestones
encourage a phenomenon I call the "Dead Zone", which I'll
get into next article.
Still, let's say you're on board: you want to do "goals
not promises" in your milestones, but your publisher does not.
So if you don't hit your milestones to the letter, you're not getting
paid. What do you do?
Greg John, producer on Spider-Man 1, has this
message taped to the wall of his office: Minus Promitte Nivus
Perfice. That means "Under promise, over produce."
If your publisher is going to punish you for missing milestones,
you have to pad the crap out of them. But then you'll probably
fall into the gold-plating trap - you'll work on that first milestone
deliverable until it's a highly polished gem, and when something
goes wrong on the next milestone, you won't have any slack.
To make that padded milestone schedule work, you have
to have internal goals that keep you well ahead of the external
milestone schedule. Greg John and I did this when we worked on Tony
Hawk ports. Early in the project, we were weeks ahead of our
milestones. As the end of the project drew near, that gap narrowed.
These were six-month projects, so for a longer project, you have
to have a huge gap-I'd say try to be two to three months ahead-in
front of the first milestone.
It would look something like this:
|
Deliverable
|
Internal
Schedule
|
External
Schedule
|
|
Design
Document
|
1
months
|
3
months
|
|
Proof
Of Concept
|
3
months
|
6
months
|
|
First
Shippable Level
|
5
months
|
8
months
|
|
Complete
Play Through With Most Levels Stubbed Out
|
7
months
|
9
months
|
|
One
Third Of Levels In + E3 Demo
|
9
months
|
11
months
|
|
Code
Complete, Two Thirds Of Levels In
|
11
months
|
13
months
|
|
Content
Complete: All Levels In
|
13
months
|
14
months
|
|
Game
Balanced
|
15
months
|
16
months
|
|
Ship
|
17
months
|
18
months
|
If you don't hit the internal schedule, it's not the
end of the world. But it does mean you have to cut features and
struggle to catch up, because you have to try to get that padding
back in case something really goes wrong. Because it will.
If you haven't felt what it's like to have the burned
disc in your hand for a milestone that's not due for weeks, I highly
recommend it.
A side note: if you do actually manage to hit your
content-complete goal, this gives you an option that game developers
usually don't get. You can add wish-list items and polish. Gold
plate all you want
at the end. Enjoy.
One last thing: by the time you're reading this, you've
probably already given an optimistic milestone schedule to your
publisher. You may be tempted to put padding in by shrinking the
internal schedule. You can't. Your best recourse would be to cut
features and renegotiate the milestone schedule.
______________________________________________________
|