|
5. Isolate key variables and assumptions
In this step, we isolate all of the assumption variables from the previous calculations -- that is, all of the variables with an orange background, which had assumption ID numbers listed next to them -- and list them together in one table.
Since we have a full 27 assumptions, we'll show only the first 3 here for the sake of brevity:
For each assumption, we also now estimate upper and lower bound values, along with a distribution type (the cells in blue). These allow us to specify the potential range of uncertainty around each assumption. You should pick the widest possible range for each assumption that can reasonably be correct based on the best available data.
All of the variables in this example use a normal distribution. However, depending on the optimization tool you decide to use, you could also use other distributions, such as a triangular distribution, which would allow you to skew the most likely value toward the upper or lower bound as appropriate. For example, you might estimate conversion rate between 0-10%, but with lower values between 1-3% being much more likely than higher values, and an expected value of 2%. In this case, a triangular distribution would allow you to simulate the range for the conversion rate so that it was appropriately skewed toward lower values.
6. Perform a sensitivity analysis on your assumptions
In this step, we run a simulation on the assumption ranges built in step 5 to build a sensitivity analysis indicating the effect of each individual assumption on net profits. This requires the use of Oracle Crystal Ball or an equivalent simulation tool.
At the end of this step, we should be able to make a tornado graph such as the one below, indicating that some assumptions have a vastly greater effect on profitability than others. These are the assumptions we should work to test as early as possible. In this case, retention rate, conversion rate, and new users per month are the top three unknowns with the greatest effect on profitability, so we should work to narrow these down as quickly as possible.
|
Printer-Friendly Version
I'm sure it could be an article in itself, but could you speak to the issues of bridging the gaps between discovery-driven planning and development methodologies over the SDLC? I understand the connection between driving your development cycles with input from the discovery-driven checkpoints, but it seems to me that this is more geared toward agile methodologies than more front-loaded design schemes. Are there any key factors to integrating DDP with specific design methods that we should be aware of?
DDP is really a very broad project planning and business planning framework that can be used for almost *any* kind of planning, regardless the scale or even what industry it's in.
It's much more of a high-level, big-picture, ten-thousand-foot-view type of planning framework, and you would generally use other software development and planning methodologies inside of the DDP plan you create.
You're right that DDP does lend itself more toward agile methodologies. Non-agile methologies like waterfall tend to assume correctness from the outset (from http://en.wikipedia.org/wiki/Systems_development_life-cycle : "Sequential or big-design-up-front (BDUF) models, such as Waterfall, focus on complete and correct planning to guide large projects and risks to successful and predictable results").
And we all know that that certainty doesn't really exist in software, much less in game development, where you combine all the unpredictability of software development with the added unpredictability of creatively-rooted projects.
DDP, on the other hand, is basically taking the opposite approach. It says to the planning team: "Start by admitting how much you don't know and how much you can't predict. Then identify all of those parameters as best you can, figure out the boundaries on the uncertainty associated with each one, and use the DDP system to figure out which uncertainties represent real risks and sort them by risk. Then you can gradually corral those risks over time as you go from one Checkpoint to the next and gradually reduce uncertainty by converting your assumptions into knowledge."
The uncertainty can arise either because we're missing some amount of a priori knowledge (say, we might not know a certain parameter in advance, even though the parameter has a definite value -- for example, we might not yet have done the marketing research to figure out cost of acquiring a user on mobile, even though it is a definite dollar figure that many companies know precisely).
Or it could be because it's something that will happen in the future and there's inherent uncertainty involved (say, we hire 10 programmers and expect them to be done in 2 years, but we don't know if they'll actually get it done, if our design changes will push their schedule to 3 years, if our lead coder will have a heart attack, etc, etc).
In either case, DDP can help you get a handle on where the biggest risks are really hiding and help you build a plan to identify them, quantify them, and manage them.
I've always been pretty agnostic about it, but I definitely think Scrum has its limitations.
There's really nothing inherently wrong with it as a day-to-day planning methodology, but it seems really incomplete to me. Scrum tends to be totally focused on the short term, and there's very little focus on long-term planning.
And that just doesn't work.
Even if you don't have a fixed schedule or feature set, or you're building your game under a huge cloud of uncertainty for whatever reason, you HAVE to do long-term planning at least at some level.
I've worked on a few projects that used Scrum and/or ScrumWorks software at some level, but were actually totally ineffective teams because they ended up chasing their tails rebuilding their own games more or less from scratch every six months.
So an artist would come in one morning and say, "Hey, guys, I just had this amazing new creative idea! We gotta do this!" And the team would immediately throw out the last six months' worth of work to focus on this awesome new idea ... which would then be thrown out itself six months later in favor of something else.
Their planning was only looking at the here and now, and no one was looking at the big picture or holding the team accountable for it.
That really drove home for me that for all its benefits, Scrum by itself is no guarantee that all that task tracking necessarily translates into any actual progress toward a goal.
So, bottom line, I think that combining Scrum with DDP could be a good approach, if you were to use DDP for the overall, long-term planning aspect, and then use Scrum (or some good adaptation of Scrum, or another good agile methodology) for the short-term, day-to-day and milestone-to-milestone planning.