Milestones and Glass Houses

10.  "That's funny. It works on our programmer's computer."

There is some mystical law of programming that ensures that developers complete milestones twelve minutes before Federal Express makes its last pick up of the day. In an age when most developers use a local network for developing games, "completed" often means that they burned the work-in-progress onto a CD-ROM for the first time since submitting the last milestone. Now, common sense suggests that the developer should wait a day to verify that the milestone runs properly, but I've lost count of the number of times I've received milestone submissions that failed to even load into my computer. (Oddly enough, they usually come from the developers who send their invoices in advance of the milestone.)

Even when developers are diligent about testing milestones prior to submission, they may discover that the milestone still doesn't work at the publisher's office. Hardware incompatibility is usually the culprit here. Since compatibility issues are often best addressed in the latter phases of development, the ideal solution is for the developer and publisher to set up identical computer systems at each of their offices. These systems should be alike in every way - right down to the installed software - to ensure identical results at both sites.

9.  "Come on, don't be a stuffed shirt! Who cares if the contract says that the joystick interface should have been implemented? Look at these cool explosions we put in instead!"

There are those people who say that once an agreement is signed, you should lock it away in a filing cabinet and not bring it out again unless there is a problem. So what if the milestone doesn't do exactly what the contract says? Well, I am not one of those people. Should a problem ever arise during the game's development, it may be too late deal with it effectively if either party has not been meeting the terms of the agreement along the way - and that includes the milestone definitions. If a milestone submission fails to meet the terms of the agreement, assurances and promises that the deficiencies will be corrected later are not acceptable. From a business and legal standpoint, the only appropriate thing to do is to reject the milestone, or, if the developer makes a compelling argument for doing so, revise the contract.

The developer can circumvent this problem sending the publisher a progress report on the milestone well in advance of submission. One developer with whom I have worked always sent me a report explaining in great detail how each milestone would be implemented in the next submission. Most importantly, the reported listed features that would not be implemented with the milestone delivery. Consequently, any disagreements about what the milestone should accomplish could be resolved before a resubmission was necessary. However, such precautions work only when the people involved really do have approval authority over the game. I've known developers who exerted a lot of energy attempting to persuade me to send them a letter approving changes to an upcoming milestone, even though many other people in my company were involved in the approval process.

8.  "You say 'partial,' and I say 'perfected.'"
"Partial!"
"Perfected! Let's call the whole thing off!"

Consider a milestone description that says, "Ability to select and complete missions, allowing player to fight anywhere from two to two thousand Ninja armed to the teeth." Does that mean that the player should be able to fight two thousand Ninjas in this milestone or in some future milestone? Sometimes interpreting a milestone is as contentious as interpreting the Bill of Rights. As a result, the developer can submit a milestone fully believing that he has satisfied the requirements, only to have the publisher throw it back to him as incomplete.

Problems involving interpretation of milestones can be prevented by having the people responsible to interpreting the milestone involved in the creation of the milestone descriptions. It is not practical to create milestone descriptions that cover every conceivable detail of a game's creation - especially since such schedules are usually created before serious planning of the game's development can be done. Therefore, people reviewing the milestone must be aware of the intent of the milestone.

As the game progresses, it's important for the producer to keep their finger on the pulse of the project. It generally makes the developer uncomfortable to work for long periods of time without the producer stopping by to look at the project. Also, if the producer isn't around when the milestone is ready to be submitted, it leaves a bigger gap in the approval process.

Regrettably, not everyone who reviews a milestone is guaranteed to have an informed opinion. While the management of many publishers don't even look at works-in-progress between the Design and Alpha milestones, I've encountered Big Wigs at several companies who look into projects at random intervals. Without knowing the details of what lead up to the milestone and what is yet to come, they sometimes demand changes before allowing the milestone to be approved. Usually this sort of surprise participation in the approval process throws everyone involved in a tizzy and can even throw the entire project off schedule.

Publishers need to understand that very large and complex games always have "just one more bug" and could use a "bit more fine-tuning." Thus, there are degrees of meeting specifications and evaluation of milestones must take this kind of imperfection into account. Unfortunately, this is - and often cannot be - done scientifically. It often amounts to who shouts the loudest, but since micromanaging Big Wigs usually exercise a great deal of influence over the release of funds, they often win shouting matches. Very little can be done except to remain calm, be patient, and try to avoid sharp objects.

7.  "But my engine is rendering 10,000 polygons. There're just, um, all being plotted on top of each other, so you can't see them."

I have a confession to make. I once worked for a developer who had grossly underbid a project before I joined the team. To make ends meet, I was occasionally required to submit milestones that were not as complete as the contract required. Fortunately for me, the publisher was not technically hip, and with a little smoke and mirrors trickery from the programmer and a little song and dance by myself, I was able to convince the publisher that the milestone represented more progress than we had actually accomplished.

I do not recommend this practice to other developers. Ethical and legal considerations aside, it usually only serves to delay the inevitable (in fact, the developer to whom I just referred went out of business not long after finishing the game). If you can not stick to the schedule the early milestones of a project, chances are you won't be able to catch up at the end.

As for publishers, I advise keeping a technical eye on the project and not allowing an unscrupulous developer to pull the wool over their eyes. There have been cases where a publisher has paid top dollar for a developer to create a state-of-the-art game engine, only to discover that an off-the-shelf third-party engine was substituted instead, with the rest of the cash winding up in the hands of a Ferrari dealer.

6.  "Well, yes, Milestone 12 does do what it's supposed to, but when are you going to actually start converting your BASIC code into assembly?"

Sometimes the developer satisfies every milestone requirement to the letter of the contract, but the publisher still has a nagging suspicion that something is wrong. Perhaps a key feature that was implemented early on has stopped working in the past three milestones. Perhaps some of the more difficult milestones to program are turning out to be the later ones. Perhaps the list of bug fixes, tweaks, polishing and finalizing for Beta is growing uncomfortably large. Whatever the reason, the publisher is losing confidence in the developer's technical ability to complete the game.

While it would be prohibitively expensive and time-consuming to deliver milestones that were final and perfect in every way, a developer does need to ensure that each new milestone submission does indeed demonstrate progress. In an ideal world, a milestone submission should stand securely on its own without an accompaniment of excuses or conditions, eliciting an enthusiastic response from the publisher.

Admittedly, we don't live in an ideal world, but a succession of milestones that provoke only a ho-hum response from people may be a sign that the project is in trouble. If the developer is indeed following the letter of the contract but does not appear to be inclined or capable of correcting the problem, is there any recourse the publisher can take?

Should friendly persuasion fail to work, there is a clause in most contracts that overrides milestone descriptions: the publisher has a right to cancel the project for any reason. Now, if it actually becomes necessary to make such threats then the project may be in great trouble indeed, and I do not recommend any publisher to make such threats lightly. However, I do wish to emphasize that a publisher should never feel that he has to accept substandard work just because the letter of the contract is being met. Sometimes it is necessary to take a firm stand rather than surrender the project to mediocrity.

5.  "So, when are you actually going to begin that video shoot... hey, isn't that an eviction notice posted on your office door?"

Consider the publisher who has advanced half of the million dollar budget, and then realizing that the project is less than a third complete, begins to get a sinking feeling that the project is turning into a money pit. In theory, the milestone deliverables and advance payments should have been arranged so that much of the project's risk is overcome before too much money is invested. But once again, we don't live in that ideal world in which the most difficult milestones are always scheduled first, the developer has the financial wherewithal to take on projects with the advance payments weighted at the end, and the publisher has been carefully watching where the money is being spent.

When the publisher finds himself in a situation in which the budget is out of control, the prudent thing to do is to stop all milestone payments. If the developer and the publisher have an honest relationship with each other, they can then get together and assess the financial well-being of the budget. But money has a funny effect on people, and not everyone is open to discussing their finances. Therefore, publishers have been known to find excuses for rejecting milestones, just to see if the developer can make due with less money. Of course, such experiments may only accelerate the demise of a cash-strapped developer.

4.  "I've loved the game from Day One - you know that. But the guys over in Sales… they just don't think they'll be able to sell it."

One of the most important milestones is Alpha. That's when all the game's elements usually come together for the first time, and people finally get a feeling for what it's like to play. In fact, looking at a milestone developed prior to Alpha can be a letdown to the untrained eye. Often when I've shown pre-Alpha milestones to people who are not in product development, their reaction is, "That's terrible! That's not a game!"

"So what," you say? "What matters is what the game plays like when it's finished! Developers don't create sales and marketing materials, you know!"

Actually, developers do creating sales and marketing materials with their milestone submissions, whether they are aware of it or not. Many people outside of development look at milestones: the publisher's sales and marketing staff, the publisher's management team and investors, potential distributors, foreign localization companies, trade show attendees, the press - everyone who has a future interest in the game, since the milestone is usually the best indicator of what the game is going to be like. On occasion, a milestone that was difficult to run, full of crash bugs, and featured half-finished artwork has so underwhelmed preview audiences that the publisher had to reconsider their investment in the game.

As I stated above, milestone submissions should elicit an enthusiastic response from the publisher, but here I am broadening the definition of "publisher" to include people apart from product development. In crafting the milestone schedule at the project's start, thought should be given to including a "wow" feature within each new milestone - if only to continually build everyone's excitement for the game. The developer should complete all features to his own satisfaction before including them in a milestone. He might lose some time if the work needs to be redone, but he will more than make up for it by earning the publisher's confidence.

Another point worth repeating is that milestones should be easy to install and run so that they can be put onto a conference room computer or a salesman's laptop. It is in everyone's interest to include an installer and all necessary third-party software onto each milestone submission and to ensure that crash bugs do not interfere with one's appreciation of the game. When submitting or reviewing a milestone, consider whether it is worthy of showing at a trade show, because there is always a chance that it may need to serve that purpose. Since all games inevitably wind up being developed late, any milestone that includes a trade show demo will likely be developed late, too.

3.  "Your milestone's all right, but I think it can be done better. In fact, I know for a fact that someone else can do it better."

Some publishers have been know to contract two developers work independently on the same game. The publisher's intent is to hedge his bet: once one developer proves himself to be creating the superior version, the contract with the other developer is canceled. This is capitalism at its best if everyone enters the deal knowing what they're getting into, but in cases where the two developers are unaware of the other's involvement, it does not speak well of the publisher's trustworthiness.

Similarly, a developer's game may be competing against another game created by that same developer. In a case of putting too many eggs in one basket, a publisher and developer may be working on more than one project together. When one project is suffering, the other project may be used as leverage for getting the first one back on track. This stratagem is also used with ports and localizations that are being developed concurrently.

Thus, approval on one milestone may be held up because of another game either doing better or more poorly than the one supposedly being reviewed.

2.  "What? This is the last day of the milestone review period? Oh, uh, well, the milestone's no good 'cause there's a sentence in the README file that ends with a preposition."

Many development contracts specify a limited time period by which a milestone must be reviewed and either accepted or rejected. The intent, of course, is to force the publisher to provide feedback on the milestone within a reasonable time period. But what is reasonable? In some contracts I've seen, the review period is so short that it fails to account for when people are at trade shows or on vacation. In other contracts, it is so long that it just encourages review of the milestone to be put off as a matter of little urgency.

Not that it matters. A publisher can always find reasons to reject a milestone if the review period is close to expiring, and failure to adhere to the limitations of a review period is not generally considered to be a breech of contract. With this issue, the personal relationship between publisher and developer is more binding than the legal obligation.

Prior to submitting a milestone, the developer should contact the publisher and find out if someone will be available to review it right away. If not, it might be worth delaying the milestone submission a few days to polish it a little more. After submitting it, the developer should follow up with a call to get at least the publisher's initial impression of the milestone.

1.  "Don't worry, we're in good financial shape. We'll send you your money just as soon as your milestone is approved."

Even large companies get into cash-flow problems; in fact, they can be harder to get money out of than small companies. Besides the red tape that must be gone through in order to cut a check, some corporate behemoths set aside a specific amount of money each month for development expenses. If the money runs out that month, little can be done about it. Some publishers I've dealt with have been up front with me when the cash is running low, but others find excuses to reject milestones just push off the development payment.