In my last post, I discussed the value of PMP (Project Management Professional) certification in game development. On a scale from Worthless to Don't Leave Home Without It, I'd give my PMP cert a 2 out of 10. Today's initials are CSM, or Certified Scrum Master, and the executive summary is that they get an 8. Or possibly a 5. Before I explain the meaning behind those two scores, I'll give a brief explanation of what Scrum means.
For the uninitiated, Scrum is a framework for project development. Without getting into its origin, artifacts, and the subtle nuances of its verbiage, I'll simply say that the main cycle around which Scrum revolves is the idea that you start with a rough project plan, perform an amount of work over a set period of time, examine the results of your work, and refine your project plan accordingly.
Then you repeat the cycle until the project is done or – as is much more likely in game development – you're out of time. For a detailed description of all things Scrum, please go here.
What the CSM cert means to the recipient is that you have verified to this field's supervisory organization, the Scrum Alliance, that you have been trained by an appropriately knowledgeable individual, and you have demonstrated in a brief test a proper grasp of the Scrum process and its fundamental ideals.
You are capable of facilitating a team in the use of Scrum to complete a project. You understand what problems this framework is designed to solve (obtaining knowledge about the project and the process, ensuring completion of highest priority work) and what areas of development it doesn't address (engineering practices, performing miracles).
You know to help the team on a daily basis by removing their impediments. You can assist in organizing their work with user stories and Scrum boards. You can help them track their progress with burndown charts. You can even help with project predictability by creating release plans and tracking team progress with velocity numbers. And perhaps most importantly, you know the value of improving the team's productivity by serving them rather than managing them. This is, ideally, what the CSM initials represent.
The value of all of this knowledge is particularly applicable to the chaotic and complex field of game development, hence the 8 out of 10 value I gave this cert earlier. As has been long borne out by studies in multiple fields, creating a design up front and sticking to a single plan from start to finish is a recipe for failure. The iterative approach outlined by Scrum is almost ideally tailored to the creation of games and is much more likely to result in success, which explains the higher appraisal I give the CSM – it can be valuable to an organization, and it's definitely valuable to you personally.
Why then the lower score? Why might this certification only garner a 5 out of 10 for value in game dev? There are two main reasons. The first of which is simply that a tool is only of value in a practical sense if you're allowed to use it. If you have a shovel but aren't allowed to use it, but you are allowed to use a football, the football would actually be more valuable when digging a hole – despite the fact that it's really an inappropriate choice for the job.
While you could possibly apply this same value-lowering assessment to any piece of knowledge, it's particularly worth noting in the realm of production methodologies.
Making a shift from, say, standard waterfall tactics ("Let's make a huge game design doc and then build the game from it without ever revisiting our plan") to Scrum is a huge undertaking for an organization. It's a cultural change that requires buy-in and support from all levels to ensure success. The local trough of the J curve associated with methodological adoption is noteworthy here, and you'll find that it engenders more than enough fear in many leaders to prevent a valuable outcome. So if you're looking to become a CSM keep this in mind: if you won't be allowed to apply your knowledge within your organization, the value of that knowledge is reduced – with respect to your organization.
The second reason why Scrum certification isn't of greater value is this: it is necessary but insufficient. I'll explain that statement with a bit of my own background. I read quite a bit about Scrum years ago and worked long and hard to implement as many aspects of it as possible on a floundering project. Even though I was not certified, and even though I wasn't properly applying some of the underlying principles of the framework, our two Scrum teams on the project averaged about a 700 percent increase in productivity over the waterfall methods previously employed.
I was impressed with these early results, so when the opportunity came up years later to obtain CSM credentials, I thought I would finally get a chance to truly understand and master the intricacies of the material. Although the training was superb (I strongly recommend Clinton Keith as a Scrum trainer for game developers), I probably could have passed the final certification test prior to taking the class. In other words, before obtaining the credentials I already had most of the knowledge that those initials represent.
I would call the CSM knowledge necessary because it clearly has great capacity to improve the production of a game, and the certification at the very least represents a common understanding of the Scrum material. The certification is insufficient, however, because it connotes neither project experience nor the cultural awareness and social engineering ability required to implement Scrum in a non-Scrum environment. The organizational value of the knowledge is lowered if you can't apply it.
Case in point – the game I mentioned earlier, the one with the 700 percent productivity gains? The project leaders reverted to waterfall after two iterations because they feared the lack of traditional control, and I didn't know how to convince them otherwise. (That team, by the way, ran way over time, way over budget, and eventually saw the project handed to another team to be completed.)
Would the CSM be more meaningful if it came with a prerequisite of project experience, like PMP? Perhaps. Would it be better if it also provided training in the anthropology of corporate application? Probably. But now you're talking about a package that it is not the Scrum Alliance's goal to deliver (at least, not with CSM…but that's not the only Scrum cert available).
If, however, there were another body specific to game development, kind of a cross between the IGDA and the Scrum Alliance, and that body provided project management certification that included all of the above points, I think we'd all benefit from having those IGDASACSM's on our teams. But they'd need better abbreviations.
Sidebar: Allow me to interject that I am not a Scrum zealot. I won't lambaste you for not keeping a product backlog. If you say "team" when you should have said "Scrum team", I will not hold you down while Ken Schwaber and Jeff Sutherland take turns kicking you. And I do not promise that Scrum is the be-all, end-all solution to game development. It will not magically ship your game for you any more than it will do your laundry. But what it says are good ideas, are – by and large – good ideas. And if you haven't tried them, you probably should.
To summarize my view of the CSM credentials, I recommend them. They represent critical and applicable knowledge for game development. And they do have value outside of the industry, should you desire such a fallback. Every team should have a producer or project manager who possesses what this certification entails, but be aware that they won't necessarily have the skills or knowledge to implement what they've learned.
Post Script
There is a new certification being offered by PMI later this year called Agile Certified Practitioner, or (PMI-ACP)SM. It's worth keeping an eye on, but it has yet to be fully developed. My appraisal at this stage is that this cert will eventually coalesce into something more valuable to game development than PMP and will possibly be more valuable than a straight CSM, but I think some time is needed to see how things shape up. Not unlike the fanciful IGDASACSM I mentioned earlier, this cert is in need of a better abbreviation. (PMI-ACP)SM? Ugh.
[This piece was reprinted from #AltDevBlogADay, a shared blog initiative started by @mike_acton devoted to giving game developers of all disciplines a place to motivate each other to write regularly about their personal game development passions.]
"Our two Scrum teams on the project averaged about a 700 percent increase in productivity over the waterfall methods previously employed."
If you were using waterfall before, what metrics did you use for this comparison?
"I'll simply say that the main cycle around which Scrum revolves is the idea that you start with a rough project plan, perform an amount of work over a set period of time, examine the results of your work, and refine your project plan accordingly."
That's really just iterative and incremental development (read: the basis of most agile development). Really, Scrum's main contributions are its very specific roles.
Sorry, the more I work on "scrum teams" (as it's often termed: Scrum-but), the more I think it's a red herring. Getting certified is a shield a lot of producers put up when they really don't understand software development methodologies. It's a farce. Now, that's not to say some scant few individuals do not actually deeply absorb the material and become passionate supporters of I&I methodologies, ravenously devouring material related to alternative methods and constantly testing and trying; but, those PM's or producers are few and far between.
Good article, though, and interesting review of the certification process.
The metrics were admittedly pretty rough. I took every task assigned to team members over a set length of time and compared the Waterfall completion rate and the Scrum completion rate. After X weeks of Waterfall, the team had completed less than 10% of what the project leaders had agreed their departments could accomplish. Under Scrum (or my attempt at Scrum, anyway) the teams accomplished an average of about 70% of what they had committed to. Don't bother picking apart the numbers, it was by no means a scientifically rigorous analysis.
:}
Fair points re: red herrings. That's the main reason I'm not a religious devotee of Scrum or anything else. It's a methodology with some good approaches to some common problems. So's PMI. So's Kanban, etc. I much prefer to aspire to be one of the "few and far between" that you described. Absorb as much material as you can, try applying methods where you think they're a reasonable fit, and see what works.
Agreed. I continue to be surprised by the number of people that I run into who don't fully understand the concept of 'tailoring.' The only constant, especially in the gaming industry, is change. No single methodology will work for all teams on all projects.
That having been said, Scrum (or at least, Scrum-like) processes seem to fit game development very well, at least insofar as developing and testing core gameplay features. There's always the risk of realizing halfway through a game project and realizing that your primary gameplay hook isn't as much fun as you thought it would be. But that's why you have to plan to iterate and get constant feedback from the get-go. For quick vertical slicing, getting non-partisan test feedback, and re-iterating game mechanics, Scrum seems to work just fine.
I've been doing most of my game project management in Hansoft lately. The folks at Hansoft suggest (and I agree) that more Agile approaches work well for iterating on game mechanics and more traditional Waterfall-like methods lend themselves more towards the development of art assets. Thesis, antithesis, synthesis. With as little zealotry as possible =D
Awesome. That's exactly what's needed! Part of being agile is being agile in mind, as well as in practice. Constantly evaluation of the processes being used and the system they're being used in are critical.
Also, again, I wanted to say thank you for posting this. We need more conversations about this and related topics!
@Joey:
"Tailoring" is what Lean and systems thinking call "right fit." I'm a huge proponent of right fit. It's good stuff.
I disagree with the waterfall approach to content creation -- especially art assets. Iteration is just as needed here. Now, whether that iteration is more like spiral or not? that's questionable. The exact processes should definitely be custom to the culture. Kanbans are great for content creation.
Spot on for both points. One of the biggest hurdles of agile adoption is the mindset that your game dev process has to be continually iterated as well. Scrum isn't a methodology or process (as it is often mistakenly said), but a framework for a studio's own I&I process that embraces respect for individuals.
Kanban is a good toolset for content creation that has been applied for years. It embraces iteration and respect as well, but ditches the practice of time-boxing, which doesn't wok well with longer value chains that are more predictable:
One word on the CSM. The value of the certification is in the standards and education of trainers established by the Scrum Alliance. This prevents the problem of the Scrum-butt variations and dilution of the framework that has impacted other PM areas.
But as you point out, it's not sufficient in itself. I compare it to driving school. A licensed graduate of a driving school is not an expert driver yet, but you are far better off sharing the road with someone who had they merely "read a book" about driving. Same with the CSM. It's a starting place, but PM excellence comes from a lifetime's passion about helping people do their best work together making the best products.
Clint
Certified Scrum trainer and agile coach
Author of Agile Game Development with Scrum
clint@clintonkeith.com | www.clintonkeith.com
Join us at the first agile game development gathering!
I understand the pricipal of Scum appication to manage work flow, task managment and prioritizing what is possible in relation to what is desired.
Some of the issues I seem to have is with the creative aspects of game development. By limiting some of the scope of desired featurs some developers never get far enough down the list to what may be a core element to the fanbase. They are making a list of mechanics, level design, asset creation, user interaction and social connectivity features.
What we end up with is the short list of completed backlog and a patch list for post release of the bug back log. Where is the game? IMO some developers are looking at this backlog backwards and holding on to feature sets and game mechanics simply because the task is acheivable when in fact it may not be what the fans want or what will make the game most profitable.
Scott, that's why it's important to really understand what sprints are for and how they work. Ideally, there should be representatives for stakeholder interests (I like to have one for the players). During sprint planning, priorities aren't assigned because something is easy; rather, a high priority is given to an item that adds the most value to the product.
Lean is adept at removing waste from a system of development and focusing solely on value-added items.
Of course, this all relies on a system (culture, management, etc.) built around iteration and value-added activities.
Scott the problem you are highlighting is due to lack of an experienced Product Owner or an imbalance in experience and "strength" between the Scrum Master and the Product Owner.
In all honesty the value of CSM o Scrum in general is limited by the understanding, availability and character of the Product Owner more than any other single factor.
For example I`ve recently joined an IT consultancy (non gaing) that supposedly utilises Scrum yet their main product owner insists on doing a comprehensive waterfall style analysis and scoping before committing to anything - it defeats the entire purpose.
So folks make sure your lead designers, creative directors or even marketing folks (depending on where you work) understand their role in the process.
If you were using waterfall before, what metrics did you use for this comparison?
"I'll simply say that the main cycle around which Scrum revolves is the idea that you start with a rough project plan, perform an amount of work over a set period of time, examine the results of your work, and refine your project plan accordingly."
That's really just iterative and incremental development (read: the basis of most agile development). Really, Scrum's main contributions are its very specific roles.
Sorry, the more I work on "scrum teams" (as it's often termed: Scrum-but), the more I think it's a red herring. Getting certified is a shield a lot of producers put up when they really don't understand software development methodologies. It's a farce. Now, that's not to say some scant few individuals do not actually deeply absorb the material and become passionate supporters of I&I methodologies, ravenously devouring material related to alternative methods and constantly testing and trying; but, those PM's or producers are few and far between.
Good article, though, and interesting review of the certification process.
:}
Fair points re: red herrings. That's the main reason I'm not a religious devotee of Scrum or anything else. It's a methodology with some good approaches to some common problems. So's PMI. So's Kanban, etc. I much prefer to aspire to be one of the "few and far between" that you described. Absorb as much material as you can, try applying methods where you think they're a reasonable fit, and see what works.
That having been said, Scrum (or at least, Scrum-like) processes seem to fit game development very well, at least insofar as developing and testing core gameplay features. There's always the risk of realizing halfway through a game project and realizing that your primary gameplay hook isn't as much fun as you thought it would be. But that's why you have to plan to iterate and get constant feedback from the get-go. For quick vertical slicing, getting non-partisan test feedback, and re-iterating game mechanics, Scrum seems to work just fine.
I've been doing most of my game project management in Hansoft lately. The folks at Hansoft suggest (and I agree) that more Agile approaches work well for iterating on game mechanics and more traditional Waterfall-like methods lend themselves more towards the development of art assets. Thesis, antithesis, synthesis. With as little zealotry as possible =D
Awesome. That's exactly what's needed! Part of being agile is being agile in mind, as well as in practice. Constantly evaluation of the processes being used and the system they're being used in are critical.
Also, again, I wanted to say thank you for posting this. We need more conversations about this and related topics!
@Joey:
"Tailoring" is what Lean and systems thinking call "right fit." I'm a huge proponent of right fit. It's good stuff.
I disagree with the waterfall approach to content creation -- especially art assets. Iteration is just as needed here. Now, whether that iteration is more like spiral or not? that's questionable. The exact processes should definitely be custom to the culture. Kanbans are great for content creation.
Spot on for both points. One of the biggest hurdles of agile adoption is the mindset that your game dev process has to be continually iterated as well. Scrum isn't a methodology or process (as it is often mistakenly said), but a framework for a studio's own I&I process that embraces respect for individuals.
Kanban is a good toolset for content creation that has been applied for years. It embraces iteration and respect as well, but ditches the practice of time-boxing, which doesn't wok well with longer value chains that are more predictable:
http://www.gamasutra.com/view/feature/3847/beyond_scrum_lean_and_kanb an_for_.php
One word on the CSM. The value of the certification is in the standards and education of trainers established by the Scrum Alliance. This prevents the problem of the Scrum-butt variations and dilution of the framework that has impacted other PM areas.
But as you point out, it's not sufficient in itself. I compare it to driving school. A licensed graduate of a driving school is not an expert driver yet, but you are far better off sharing the road with someone who had they merely "read a book" about driving. Same with the CSM. It's a starting place, but PM excellence comes from a lifetime's passion about helping people do their best work together making the best products.
Clint
Certified Scrum trainer and agile coach
Author of Agile Game Development with Scrum
clint@clintonkeith.com | www.clintonkeith.com
Join us at the first agile game development gathering!
http://tinyurl.com/3r46afc
Some of the issues I seem to have is with the creative aspects of game development. By limiting some of the scope of desired featurs some developers never get far enough down the list to what may be a core element to the fanbase. They are making a list of mechanics, level design, asset creation, user interaction and social connectivity features.
What we end up with is the short list of completed backlog and a patch list for post release of the bug back log. Where is the game? IMO some developers are looking at this backlog backwards and holding on to feature sets and game mechanics simply because the task is acheivable when in fact it may not be what the fans want or what will make the game most profitable.
Lean is adept at removing waste from a system of development and focusing solely on value-added items.
Of course, this all relies on a system (culture, management, etc.) built around iteration and value-added activities.
In all honesty the value of CSM o Scrum in general is limited by the understanding, availability and character of the Product Owner more than any other single factor.
For example I`ve recently joined an IT consultancy (non gaing) that supposedly utilises Scrum yet their main product owner insists on doing a comprehensive waterfall style analysis and scoping before committing to anything - it defeats the entire purpose.
So folks make sure your lead designers, creative directors or even marketing folks (depending on where you work) understand their role in the process.