Gamasutra: The Art & Business of Making Gamesspacer
Measuring a Designer's Value
Printer-Friendly VersionPrinter-Friendly Version
View All     RSS
April 23, 2014
arrowPress Releases
April 23, 2014
PR Newswire
View All

If you enjoy reading this site, you might also want to check out these UBM TechWeb sites:

Measuring a Designer's Value
by Kain Shin on 07/19/11 07:32:00 pm   Expert Blogs   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.


Out of all the disciplines, game designers definitely have it the hardest when it comes to evaluation during interviews and getting respect from other disciplines.  Their "soft" skill sets are difficult to quantify in a standardized tangible manner because they are, in essence, professional thought architects.

In discussions with friends who work on collaborative multi-disciplinary software projects in and outside of the games industry, one question came to mind:
Can a QUALITY product be completed WITHOUT a dedicated full time designer?

The conclusion we came to was a conditional "yes", but only if either of these two things were true:

  1. Each member of the team was fully capable of cohesively visualizing the ramifications of their component before actually implementing it
  2. There was infinite time to iterate on the implementation of bad ideas until they become good ideas

Time is rarely infinite and trusting everyone on the team to maintain a cohesive vision of the product can be a gamble. For that reason, most projects will involve people whose role it is to supplement the team's ability to visualize the details before time and resources are spent on the implementation of those details.

Depending on the industry, the people that are tasked to perform these functions hold titles such as Producers, Directors, Project Managers, Product Owners, Designers, or Planners. The idea behind these roles is two-fold:

  • Thinking about ramifications and managing expectations from all angles warrants a fulltime job
  • The cost of thinking is cheaper than the cost of doing implementation work that will be discarded

So... what are the metrics of this "product owner" position?

I believe that every industry can distill the evaluation of this pivotal role into three distinct categories:

1. Pre-Visualization and Target Synthesis
How good is the designer at running simulations in the brain before committing resources to an implementation of the design?  Nobody will be perfect at this... iterations will be a fact of life in any industry involving collaborative creation of product no matter how good any designer truly is. There is real value from being able to fail as quickly as possible and extract useful information from those failures.

If design can be thought of as iterative problem solving, then there are two ways to evolve a prototype into a final product:

  • One way is to throw random darts at the problem until a solution hits... and hopefully that designer will recognize a solution when they see it; Or perhaps do variations based on each possibility and outsource decision-making to a group in order to know what is best for the game
  • The other way is to incorporate the problem into a mental model with an effort to define success and failure conditions so that experiments get mapped within an intentionally designed philosophical framework

Experiments are neccesary, but they do cost time and money. The empowered "method" designer who believes in a governing philosophy will cost the company less money than the "random darts" designer who might be prone to seeking idea validation from external sources to compensate for a lack of internalized values for the game.

I pair mental simulation with philosophy because one without the other is useless, and the evaluation of this pair is often strongly coupled as one provides context for the other.

Sample Pre-Visualization and Target Synthesis Interview Questions:

  1. Design a product about X with constraints Y
  2. Why did you design product X that way?
  3. What would happen if you do Z to product X?

2. Converting Thoughts to/from Language
Pre-Visualizion takes place inside the designer's head, but that vision needs a reliable channel in order to result in effective output that can be used by other developers.

Team members (including the designer) may not have all of the information needed to perform their function. A designer will need to be able to understand the input they are getting as much as others will need to understand the output they are giving.

Specifications (the "what") are critical to the conversion of ideas into code, but being able to convey the reasons (the "why") behind those specifications is what separates a creative director from a worker bee designer.

The best creative directors I have worked with have a way of communicating not just the specifics of their ideas, but also the philosophy behind their ideas to the team. These mentors infect the rest of the team with a galvanizing self-perpetuating aesthetic compass that flows in one unified direction. This transferrence of philosophy in addition to the visualization of tangible details gives implementors an opportunity to internalize cohesive overall design values into their belief system for making the game great.

Put another way, the best designers I know have a way of making everyone around them a better designer through vocabulary.

Sample Communication Interview Questions:

  1. Describe a product you like/hate and explain why it is good/bad
  2. Deconstruct a product that you did not make, and explain some of the design decisions behind that product
  3. Teach us how to do something complex that you are an expert at

3. Managing Relationships
Even though communication may be clear, debates may still occur. People will not neccesarily follow blindly.  Unless the designer is going to work alone, this person will need to deal with human factors.

I have seen many highly skilled designers felled by a fatal flaw in this particular category.  The pattern was often the same:

  1. Person is highly skilled and competent (or just overly confident)...
  2. Person expects everybody to be like them (i.e. agrees with their mentality)...
  3. Person antagonizes people who are not like them (i.e. disagrees with their mentality)...
  4. An authority figure eventually realizes that the cost outweighs the benefit and fires that highly skilled powerful member of the team for the overall good of the project

... or in the other direction, people quit the team or become passive-aggressively dysfunctional because of a breakdown in relationships with other team members.

Like othello, the science of getting along is said to take a minute to learn and a lifetime to master. If games were to one day suddenly not exist in this world or become banned as an occupation, I imagine that the best game designers out there would eventually go into the world of politics as a systems architect.

Sample Diplomacy Interview Questions:

  1. Describe a product design that you do not agree with
  2. Explain why the makers may have chosen to go that direction
  3. How would you get the makers to change their mind about that direction?

The Problem:
Few developers ever say that they are a bad designer. Game developers who call themselves good game designers will rarely ever get tested for that skill unless they are actually interviewing for a game design position.

This can lead to a situation that is potentially hostile to game designers who are surrounded by unchallenged creative and frustrated individuals who believe that they can do the job of the designer if they weren't so busy making tangible game code and assets.

So how do you combat this mentality?

The Solution:
A design department can be framed as a service provider just like any other department.

If code is unstable or tools are difficult to make a game with, then that is a failing on the programming department. If art is not aesthetically suitable for the game or won't fit on that particular hardware, then that is a failing on the art department. If the design department is getting the support it needs from other disciplines, but the game is flailing in the formation of an identity or the solidification of a coherent set of systems mechanics, then that is a measurable failure of the design department.

Inside or outside of games, the design discipline is responsible for managing target aesthetics of the final product. If there is a good idea to be mined from somebody who is not in a design position, then great; that line of accountability still needs to be managed by somebody connected to the ecosystem of the game's intentions. Design, Planning, and/or Product Owning is a skill-based discipline with three categories of metrics that serve as qualifiers of competency within that discipline.

If you find yourself in a position to develop an interview process for a design position, then your process would ideally incorporate at least three RPG stat trees:

  • Predictive Mental Simulation + Philosophy
  • Thought Communication
  • Diplomacy
... and if you are preparing to interview for the position of "designer" in any industry, then consider this a suggested harness for self-evaluation.

Related Jobs

Linden Lab
Linden Lab — San Francisco, California, United States

Lead Engineer
2K — Novato, California, United States

Lead Mission Designer
Gameloft — New Orleans, Louisiana, United States

R&D Game Designer
SOAR Inc. — Mountain View, California, United States

Game Designer/Narrative Writer


Timur Anoshechkin
profile image
Interesting article, but misleading title. Measuring leads to believe that some metrics are involved.

Jitesh Panchal
profile image
Good article with interesting information. Time to check my attributes! :)

Kain Shin
profile image
Thank you

Laura Bularca
profile image
Kain, this is a great article because it is clear, concise and well presented. If there are any other details you can share, please do so because I would be quite interested in finding out how you evaluate a designer/ design task and what you consider to be a well done job. You are a programmer and therefore I would like to get an example on what you would like to receive from a designer in terms of design specs. And if it is not too personal, do you consider yourself able to be the designer you describe or not?

I ask this because, while I fully agree on clarity of information and concise, structured specs, I met developers who did tended to simply ask for too many details. I want to know where do you draw the line, because presenting a vision, a philosophy, a concept is no easy task to do (and therein lies the value of any creative person, in my opinion). Beauty is in the eye of the beholder, in the end, even if it seems like everything is clear.

As for metrics, it would be really cool to conceptualize something to use in this regard, but impossible to standardize them. They can at best be designed per project and per team in strict correlation, so I didn't find your title misleading.

Kain Shin
profile image
I apologize for my abuse of the word "metrics". My intention was to convey a sense that there are tangible success and failure conditions to look for when evaluating a designer as opposed to going with gut feelings. After looking "metrics" up in the dictionary, I realize that it sounds like there are numbers involved that can be compared as if the interview were a standardized test, which was not my intention at all. If anything, a truly standardized test would be an invitation for designers to "game" the system :)

I agree with Laura that the designer evaluation exists per project and per team. A test for a street fighter designer is probably not appropriate for an MMO designer.

@Laura: Thank you for the kind words.

I have seen the detail problem as well. In the other direction, I have witnessed the 300+ page design doc that nobody reads. On this, I believe that the relationship between designer and implementer is a two-way collaboration that works best when channels of feedback and response are maximized. Different people have different communication styles and needs for information. Some programmers tend to ask for too many details because they may feel disempowered to "design" (would rather somebody else be wrong on design than them). I may be biased, but I like the idea of gameplay programmers (such as myself) who are empowered to fill in the blanks left by design while engaging in constant feedback (and iteration) loops with a designer from conception all the way to post-execution. The feedback loop not only reinforces the quality of the design and its implementation, but it also gives both sides repeated practice in working together within this context.

About where to draw the line... I like the idea of the designer taking ownership of the WHAT (big picture) and WHY, which gives the programmer/artist/level builder the direction needed to own and become an expert at the WHAT (minute details) and HOW.

To answer your question ("do you consider yourself able to be the designer you describe or not?"), I developed these values after being mentored by some very good designers, and I have found that striving to maintain these values makes it very easy to work with designers... So I would like to believe that I might do well on an evaluation if the project and team were like mine (emergent sims), but I will probably do very badly if I were being evaluated for a design position on business application software :)

Mikhail Mukin
profile image
There are different types of designers, and IMHO the "value" should be based on what they are needed for, it takes different set of skills, different "mind set".

Systems/technical designers.

Senior level people. Responsible for implementing systems (like inventory, save/load game, mission objectives etc) on a "design side". Create mission "templates" and other re-usable parts. Main point of contact for engineers. For them, good knowledge of whatever scripting language is used in a game (lua, unreal, C#, Python) is a must. "Systems", logical thinking and ability to take ownership of several systems is required. Basic knowledge of C++ and game debugging is a good plus. Knowledge of visual scripting systems, how systems are done in popular games is greate too. I think people who started in engineering "long ago" but wanted to work more on a game/design side are the best for such role. My favorite type of designers ;)

Mission designers.

Well - they make missions. Place mission objects, hook triggers, assign enemy AI types, create combat encounters and tweak them till they are fun. Ideally - they would know how to use a set ot "tools" that systems designers/engineers prepared for them best. Some mid/basic-level scripting knowledge IMHO is very valuable (on some games - required). Knowing a lot of games, what made particular levels, particular encounters in them "fun" is a must. They usually need to be very good players too.

World builders.

Not all games have them - but for games with "big worlds" they are good to have. They populate the world - create terrain, foliage, rivers, bushes etc. Position main cities and structures. Basically - build the foundation - and mission designers add mission specific objects/scripts on top of those. I saw them also actually making some objects - bushes, trees, environmental peices, terrain textures. They are half artists - half designers. Usually - do not need technical knowledge.


Create dialogs, overall story, setting. Might not be technical, but need to know how to use (typically limited) game/engine means to express character moods, personalities, fill in "ambients" etc. Have to be able to come up with a few (the more the better) good jokes.

Design leads.

The guy who thinks he know what this game is about :) and trying to make people around him to implement this vision, and people above him to pay for that :) Well, together with director/producer. Can come from anywhere, but I think typically ex mission designer with good logical/organizational skills or systems designer with a good eye for fun gameplay. There might be examples of writers or world builders becoming good design leads - but I think not in my experience.

Well, and sometimes there are specific designer(s) for "social aspects" of the game, for game balancing, for cutscenes, for "multiplayer" etc.

Kain Shin
profile image
@Mikhail - True. The values of this article are no substitute for being able to execute on the specifics of what is being designed.

This article originally started out as a conversation between myself and some accomplished tech-professionals outside the games industry. We were feeling out the commonalities of our fields as a means of breaking down some common patterns that arise from industries where thoughts get turned into 1's and 0's through a collaborative process.

Whether you are designing gameplay, level geometry, narrative, production schedules, or code architecture, I believe that these three axes are an effective galvanizing compass towards the development of your screening process.

Anonymous Designer
profile image
As I see it, 1 is knowing the magic, 2 is using the magic, and 3 is controlling the magic. I really enjoyed your perspective, so much that I would have liked to read more about it. Like you said 3 is the downfall of many - if we were all experts at diplomacy we may not have entered this industry in the first place :)

Kain Shin
profile image


Alan Jack
profile image
Brilliant, and a very professional and academic assessment of what I was planning for my next article!

As a student studying at Masters level, I've had a lot of practical experience in both design and production in a safe, protected environment, and its been very insightful.

I've seen pretty much everything you've said here - I began assuming that design meant producing heaps of documentation that nobody would ever end up reading, and I've seen the chaos that erupts from a project in which nobody holds the vision of the final product.

I think the last point simply can't be overstated enough - a designer needs to be able to work closely with everyone on the team, and needs to be approachable at all times. I can't imagine doing my job in my current position (as a designer on a game) if I couldn't happily interrupt someone else to talk about a feature or vice versa.

Kain Shin
profile image

Joe Cooper
profile image

No comment except that this is a very smart article.

Kain Shin
profile image

Luis Guimaraes
profile image
Great article.