Improving Quality

The focus on quality is a principle of Lean Production. Lean Production minimizes the inventory of unfinished work between each stage of a value stream. This allows change to be introduced much more frequency because the debt of unused components is keep low.

This is key to how companies like Toyota have improved the quality of their cars and dominate the market. If you find that you have a defect in a part of your car in production, it becomes a lot easier to introduce a new improved part when you don't have a million of the old parts sitting in a warehouse.

This philosophy translates to the assembly line as well. On the Toyota factory floor, if any assembly line worker sees a defect in the cars being produced they hit a big button nearby. If that problem isn't fixed in the next 20 minutes, the entire factory assembly line comes to a halt until the problem is fixed. This dedication to quality is unmatched in any other company and it is made possible by using Lean Production principles.

Additionally, as we reduce the cycle time by finishing batches of product, we improve our iteration cycle. Using our example, we can play finished zones every week as they "roll off the line".

We don't have to wait 16 weeks for entire level to be finished to play it. This one week cycle allows us try the level much sooner and introduce change before we have spent much time creating the remaining zones of the level.

If we build all of our levels in parallel and don't discover problems until 90% of the work is done (such as rendering or memory budgets or gameplay quality) we are faced with having to throw out a great deal of work or ship the lower quality work to meet the deadline. It's the same problem that car companies face when they decide to throw out the million potentially defective parts in the warehouse.

Outsourcing

Outsourcing has its benefits and place in asset production. However many studios have found that outsourcing limits the amount of iteration that can take place in the creation of large assets such as key characters or levels. This limited iteration can impact quality or introduce expensive rework that limits the cost benefits of outsourcing.

Lean Production evolved to work with external suppliers. These are indispensable for manufacturing industries such as car production. Suppliers to Lean Production companies have to become Lean themselves. The key difference with Lean suppliers is that they deliver smaller batches of parts to the main production line. This is done to allow quality improvements to be introduced frequently and at lower cost.

How can this translate to game production? With our example, we don't want to outsource the entire value stream. They key is to outsource parts of the value stream that don't require the high skill level that you want to retain in house. Generally in level production, you want to be able to keep large the layout tasks within the studio and outsource the parts that are used in layout.

Examples of this are environment sets or collections of assets that are common throughout your level. If you were creating a large city level, you would outsource all the props such as light posts, mailboxes, vehicles, building components, ambient sounds, etc. These environmental sets are brought into the layout stages (hi-res art and audio layout). This allows for continued iteration of the layouts in the studio where they belong.

The value stream for our outsourced level production would now look like this:

The outsourced assets can be identified in early level concept development to allow for sufficient lead time for outsourcing. Many layout tools support asynchronous introduction of outsourced components. An example is the Unreal Engine 3 editor. The packaging system allows for proxy assets that can be replaced one at a time which automatically replace all instances of the asset throughout the game.

What if the Rest of the Team is Still Using Scrum?

During production, not all iteration is useless. The team is still innovating the game in areas that don't affect production. Sprints are still valuable for these teams. How do these teams work with the teams using Kanban?

Scrum teams can still use Kanban to drive production. If we have one week cycle times inside of four week Sprints, the production teams can still show the result of four cycles every Sprint review. The production team won't have to do Sprint planning, but they should still conduct regular retrospectives. Additionally Daily Scrums are still a useful practice for the production teams. Impediments will still arise that need to be addressed by the team.

Conclusion

In using this step-by-step process for creating level assets we had taken the cost of producing a single game level from 16 weeks to seven weeks (seven zones times one week per zone). This represents a 56% improvement in the cost of creating a level and required very little new tools or technology to achieve.

This is just a start. Lean production demonstrates improvements that can replace or supplement the need for outsourcing large key assets that a studio may prefer to keep in house.

Like Scrum, the adoption and practices of Lean are subject to change based on your own environment. The key is to achieve visibility on the process and to create metrics that are meaningful. Time-boxing asset creation can be a challenge. Most artists don't consider quality a variable in their work. It's a key role of the product owner to communicate their vision for not only the final product but the responsibility for ROI that they control.

References

1 http://leansoftwareengineering.com/2008/05/21/coffee-cup-kanban/

I consider the two books by Tom and Mary Poppendieck to be the bibles of Lean Development: Lean Software Development: An Agile Toolkit and Implementing Lean Software Development: From Concept to Cash

I also pay close attention the to writings of Corey Ladas and David Anderson of Modus Cooperandi for inspiration and knowledge about Kanban: http://www.moduscooperandi.com/

---

(c) 2008 Clinton Keith

Title photo by Gregg, used under Creative Commons license.