Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
July 31, 2014
arrowPress Releases
July 31, 2014
PR Newswire
View All
View All     Submit Event





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
Hiring Managers: Vetting Game Programmers
by Marc Mencher on 03/01/13 04:58:00 pm   Expert Blogs   Featured Blogs

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

So you're in that blue sky brain-storming session and there are a ton of awesome ideas up on the board, and then someone says, "This is GREAT! Now we need someone who can make it work like that." Yep, you need a programmer. And not just any programmer. You need someone who can push the envelope, work with all the teams and fit into and grow with your company's culture. That's the moment your Quest begins . . .

There's a lot of "ebb 'n flow" these days and a lot of open jobs. A lot of people are getting hands-on experience, thanks to the ease of making small social games. Their work is easier to see but harder to vet. There are programmers at big companies whose projects are either ending or whose companies are downsizing (or closing), but there may be relocation issues that have to be considered in addition to salaries.

To increase the odds of making the best hire for your company, you want to develop a basic "vetting" system that looks at traditional and non-traditional aspects of your potential new programmer.

Step 1: The Job Description

Before you write the job description, be sure you have a good understanding of what various job titles mean. Terms like developer, programmer and engineer aren't always interchangeable. A QA person might be called an engineer at one company, while developer might refer to someone other than a programmer. Some programmers prefer engineer because it sounds less like a drudge position. If need be, include a sentence or two that describes what the title means at your company.

Figure out what you want your new hire to do, then write a job description that clearly states you’re required and preferred criteria. Aim for something between "as long as you're breathing" and "must have a Masters Degree in everything." (Hint: It's pretty easy to spot a job req that's been customized for someone you've already decided to hire.)

Here are some things to look for:

  • Deep interest in gaming, both as a programmer AND a player. This used to be designated as "avid gamer" but what does "avid gamer" mean any more? Of course candidates are going to tell you they're avid gamers!
  • Wide variety of gaming genres (ok, at least two!) If you make MMOs or FPS games only, specify that you want someone whose interest and passion is in your genre.
  • Want someone with corporate-culture experience? These days, that's different from "must have shipped an AAA game" because small companies can ship AAA titles too.

Let candidates know if some kind of testing will be administered. If you have questions about the legality of testing, check with your HR department, and if they don't know, get the info from someone who does, like your state's employment agency. Surprising an interviewee with an on-the-spot test (the formal kind) can be grounds for action, and not the good kind.

Be clear about your interview agenda. Will candidates be asked to undergo both "personal" and "technical" interviews?

Do you want to see a demo of the candidate's work? Do you want to see it online by yourself? Ask for a CD or URL. Because of proprietary software and NDAs, be willing to look at open source work and/or game modding.

Does your formal application allow for an attached resume, or will the candidate be required to fill it out? Will the candidate be asked to provide a salary history? Letting interviewees know this in advance saves you time and helps reduce the normal anxiety that comes with interviewing. (It also tells you if the candidate can read and follow directions.)

Step 2: The Resume

Read the resume before the interview. This may sound sort of "duh" but looking at the resume in front of the candidate sends a message (intentional or otherwise) that this interview is either inconvenient or a pro forma exercise so the company can hire the person
it REALLY wants. Reading the resume in advance gives you a chance to come up with specific questions.

Check out the companies listed on the resume. How long has the candidate worked at the past few companies? Are any of them start-ups? Do you really want someone who has moved quickly up the ranks and name-drops like crazy but has no real experience?

Step 3a: The Interview

If someone in HR is doing the initial interview (other than having the candidate fill out paperwork,) be sure you've reviewed the job reqs with that staff member, and provided some initial questions you'd like them to ask the candidate. Letting someone with no technical background and/or real industry knowledge conduct the initial interview might let a fast-talking candidate in the door and a good one get away.

Everyone who's participating in the interview should be prepared (i.e., has a copy of the resume and has seen the website or CD). If you're including "peer" interviews for the candidate, be sure your staff knows that their job is NOT to scare the candidate way.

You know the drill: be prepared, be prompt (and if you might get called away to an emergency, let the candidate know before you start the interview), don't take calls (unless there's a pending emergency). Be focused and ready to listen. As a courtesy, if you're deathly ill, stay home and let a colleague conduct the interview, or make arrangements to do it via Skype or some kind of video conferencing.

Having a member of the team with whom the potential hire will be working, preferably the team lead, as part of the interview process is always a good move. While there are many skilled programmers who can fill your job, getting one whose personality meshes well with your other programmers is always a bonus and in some cases, a must.) Remember that these people will be working very closely with each other, often in frustrating circumstances (the dreaded crunch time, for instance), and an argumentative or disruptive team member can cause a hit to productivity. Personality and work ethic is just as important as skill set, especially when you've got a small team, a tight schedule and no money to spare.

Step 3b: Interview Testing

Assuming you're cleared for testing, use simple programming or logic tests. Asking very specific questions, like hex and terminology definitions, isn't the most effective way to evaluate a programmer’s ability to code because rote answers don't tell you how the candidate programmer THINKS.

Recent grads will probably be able to answer "arcane" questions because the info is fresh in their heads but is that what you really need? Good programmers like to solve puzzles, riddles and mysteries rather than apply canned solutions. The right candidate will have some basic problem-solving skills in addition to specific programming knowledge.

Propose a simple programming issue and ask the candidate how s/he would handle it. Maybe a brain teaser or a suggestion for a modification to your product, which has the added benefit of showing you how well the candidate prepared for the interview. A good type
of coding question is one that has several answers; ask your prospective hire to give you the most optimized solution. You can quickly gauge how well he/she thinks and solves problems based on the answer. Someone who consistently picks the most obvious but less optimized answers is good for entry level positions, but if you're hiring for senior positions, you'll want people who can think on their feet, understand the need for optimization and have good reasoning behind their choices.

Here's an example of a good question:

Every number between 1 and 100 has been inserted into an array of 99 integers in random order, with a random number missing. What's the most optimized and memory-efficient way of figuring out which number is missing?

A weak answer would be to create 100 flags, then loop through the array and log each number, and subsequently loop through the flags to find the missing one. A stronger answer would be to loop through the array and add them all up into a single integer, then subtract the answer from 5050 (the sum of all the numbers between 1 and 100). An even stronger answer would be to sort the array, then loop through until a number gets skipped.

If you want to ask technical questions, avoid asking hypotheticals like what types of inheritance or global variables appear in a CPP file or polymorphisms or singletons in C++. Instead, present actual situations that are relevant to your product or project (unless, of course, any of those other examples ARE relevant. Bear in mind that the simplest code is, more often than not, the best code. A programmer who loves to pepper the code with
unnecessary methods like 'mutable's and 'goto's might not be the best candidate. Likewise, don't ask questions that are overly complicated for your code base. Unless you regularly need inline assembly code, don't ask the hire to describe how to unwrap loops or other overly complicated questions. While it's a great indication of general knowledge, it won't tell you if they'll do a good job.

In some circumstances, you may be looking for someone who can not only move forward with a project's code but also knows how to deal with legacy issues in a manner that doesn't involve stopping the entire project and starting over from scratch. It's great when the candidate sees this situation as an interesting challenge but you want to avoid the candidate who claims to be able "fix anything."

Step 3c: Interview Questions

Ask the RIGHT questions. Hopefully, the combination of a well-crafted job description and vetted resume has weeded out candidates who aren't right for the job.

Does the candidate use/play your product regularly? If you make MMOs, you'll be able to determine the level of immersion and investment pretty quickly. If the candidate gets that glazed look and launches into a Very Detailed Account of her avatar and the last raid, that tells you something, too.

"Beware the lone programmer in a room" is an old industry adage that still rings true. You want someone who will fit into your company's culture and actually likes working with other people. Does the candidate seem like someone who will thrive in a high stress team environment (if that's what you've got) or does the candidate seem like someone who's more interested in showing others "how it's done."

Consider asking some off-the-wall questions like, Do you prefer cats or dogs? Cake or pie? Summer or winter? And why? An industry veteran interviewing a programmer candidate said, "Tell me a joke." The stunned candidate replied, "Oh. Do you want a funny one? I didn't really prepare anything." That told the interviewer what he needed to know about
the candidate's ability to think on his feet.

Ask about a "hot" programming topic that's making the rounds on industry boards and at conventions. Is the candidate passionate about one side or the other? Dismissive? Baffled? (Hopefully the candidate will not say, "Well, does that really have anything to do with this job?") Having a sense of humor is actually pretty important in our industry because it
reduces the possibility of melt-down at the worse possible moment.

Ask candidates what they love about programming. (Hint: "An easy way to earn a living" probably isn't what you want to hear. "I love to solve problems" or "I like to make great games" is better.)

What's the biggest thing the candidate worked on that didn't ship? Do they have any idea why it didn't ship? Watch out for indications that the candidate thinks failure was other people's fault. One of the tenets of Agile Development is that failure by one unit is failure by all, so you don't want to hire someone who's more adept at finger-pointing that accepting
responsibility and proposing positive solutions.

Has the candidate worked in an Agile Development environment and if so, how was it? If it seems that the candidate felt it was intrusive, see if you can determine whether the system was poorly administered or the candidate just doesn't like to be interrupted or told what to do.

What was the biggest challenge the candidate has faced as a programmer so far, and how did s/he solve it? (For female programmers, gender bias may be the biggest issue so be sure you stay within the boundaries of what can and can't be asked legally.)

Ask question(s) that give you a sense of how flexible the candidate is, how willing to try new approaches, take suggestions and explain solutions. Odds are you probably won't be happy with some hot shot who thinks that everyone else is too simple-minded to understand what he does. (In fact, sometimes this is a red flag that the programmer might not be so good at
building strong foundations or accepting responsibility when fixes don't hold together.)

If you decide to review the candidate's demo in person, ask what specific portions s/he handled. Obviously, with a small app, it's probably "all the programming" but if it's a big game, the programmer was probably part of a team. Ask about how collaborations worked and whether the programmer's suggestions for game play improvements were considered. Avoid the programmer who says, "Oh, I write the stuff but I don't play the game."

Here are some questions on the "lighter" side:

  • What do you like about gaming?
  • What was the first computer or console game you played?
  • What was your first computer?
  • What's your favorite game and why?
  • What's your favorite book? Movie? TV show?
  • Do you prefer open worlds or well-defined quest lines? Do you think a game should/can have both?
  • What's your favorite character class?
  • How would you briefly describe the mechanics of your favorite game to a non-programmer?
  • Do you usually play games to the end?
  • What's your Beta test experience? (No, you're not looking for a QA person BUT it doesn't hurt to hire a programmer who thinks like a QA person at least a little, as in being able to vet their own work before they hand off a fix as "done.")
  • What's your favorite game of ours and why? (If you've only published one game, they better have played it! And listen for their own words—if they sound like they're parroting what they read about your game, it's entirely possible they haven't actually played it.)
  • If you could work in any other area of our industry, what would it be and why?
  • What makes a game fun for you? (No, you're not hiring a game designer BUT the programmer's job is to make the designer's vision work.)
  • If time and money was no object, give me a quick pitch for a game idea. (No, you're not hiring a marketing person but you want your employees to be well-rounded and be able to communicate with each other.)

Although there's no single magic formula for hiring the best programmer, you've got a lot of tools at your command that will give you a pretty good sense of which candidate has the right skills and is the best fit for your company.


Related Jobs

Raven Software / Activision
Raven Software / Activision — Madison, Wisconsin, United States
[07.31.14]

Senior UI Engineer
Disney Consumer Products
Disney Consumer Products — Glendale, California, United States
[07.30.14]

Contract Game Programmer
Zindagi Games
Zindagi Games — Camarillo, California, United States
[07.30.14]

Software Engineer
Telltale Games
Telltale Games — San Rafael, California, United States
[07.30.14]

Core Technology – Client Network Engineer






Comments


Cordero W
profile image
I always wondered how programming interviews go, mainly cause although I have a lot of programming problem solving, I'm not a wiz when it comes to knowing all the specific terms and types of algoritims. That's just something you build over the years, not something I should know off the bat. I mean, yeah, it would be a bonus, but even MIT students don't stick to those kinds of standard. It's just the natural way we computer programmers think. We care more about making it work than about doing fancy tricks. I respect the very high tier programmers who can code a specific line to make it 50% faster, but they're the exception, not the norm.

Whenever I do go to a programming interview, I hope I'm not overwhelmed.

Casey Labine
profile image
Based on your comments here, there's a very good chance you'll be overwhelmed. That said, it depends on the position and the company. It sounds like "scripting," i.e. things like game logic and content implementation (quests, items, etc.), would be a better fit for you than an engineering job. But it's all programming, and the requirements really depend on the shop.

Michael Dawe
profile image
Make sure that technical people are doing the evaluation of technical questions! The missing integer question is a great question, but the answers given in this article are out of order in terms of strength. Know your questions well.

Amir Barak
profile image
"Every number between 1 and 100 has been inserted into an array of 99 integers in random order, with a random number missing. What's the most optimized and memory-efficient way of figuring out which number is missing?"

calling the guy who wrote the algorithm and ask?
Look at the source code and see?

Jonathan Jennings
profile image
lol a friend of mine took a test like this a few weeks ago it made me laugh because considering the things he told me about the company and the way they designed their custom game engine this was beyond an irrelevant question. it gets into the logical side of programming but had nothing to do with the type of code they would expect him to write( according to what he told me)

Jonathan Jennings
profile image
double post my bad

Jonathan Jennings
profile image
triple post my bad

Paul Tozour
profile image
Most of the "lighter side" questions seem really irrelevant to me. The suggestion of asking "off the wall" questions is also very unhelpful and much more likely than not to make you look really clueless and turn an engineering candidate off of your company completely.

It would be far better to fill the non-technical parts of the interview with questions relating to teamwork and organizational issues. I've found that it's much better to give candidates simple case studies to find out how the engineer would actually react.

Write up small case studies of difficult teamwork situations you've encountered in the past -- situations with interpersonal conflict, teamwork issues, or places where employees were being uncooperative or appearing to be uncooperative -- and where you have a good understanding of what went wrong on the team and what SHOULD have actually happened.

Then ask the candidate, "What should the engineer do in this situation to resolve the problem? What should the producer do to try to resolve the situation? Was the artist's reaction to the programmer's comments reasonable, or not?" Etc.

Paul Tozour
profile image
Also, this:

> What makes a game fun for you? (No, you're not hiring a game
> designer BUT the programmer's job is to make the designer's
> vision work.)

seems a bit off the mark to me.

The engineer's job is to make the designer's vision "work" in the sense of getting it to run smoothly and bug-free on your target hardware, yes -- but it should never be to make it "work" in the sense of having to change existing designs, or fixing fundamental game design flaws that your designers shouldn't be making in the first place.

That's still the designer's job.

Otherwise, you can very easily end up with situations where hand-wavy designers throw intrinsically flawed or inadequate designs at the engineering team, and then turn around and try to blame them when those flawed ideas can't be made to "work."

If you happen to find engineers who are good at fixing design flaws and improving existing designs, great ... but you shouldn't have to hire for that for most engineering roles, especially those outside of pure AI and gameplay programmer positions.

I'm a gamer, and I play tons of games and do a ton of design, and I think it helps me.

But your organization *ought* to be able to hire talented engineers who don't play any games, ever, and who come from Wall Street or Google or any other industry outside of games, and still be productive with them, especially if you're hiring them for something not directly related to gameplay (tools, rendering, etc).

Hire engineers for their engineering ability and their teamwork skills. If you find yourself having to hire engineers who are half designers just to make the design work in the first place, it's probably a symptom of something else going on that you should look at more closely.

Amir Barak
profile image
weird, double post I think...

David Cook
profile image
I would disagree with the assertion that a programming hire must be an avid gamer. For certain roles, such as implementing core gameplay, yes. But for audio, graphics, tools, ... not necessary.

Josiah Manson
profile image
The given answer to "What's the most optimized and memory-efficient way of figuring out which number is missing?" seemed strange to me. "An even stronger answer would be to sort the array, then loop through until a number gets skipped."

Why would this be the stronger answer compared to subtracting from the sum? Subtracting takes O(n) time, whereas sorting takes O(N log N) time. In fact, this "strongest answer" seems like to worst solution, as even the flag answer that was first given takes O(N) time. Although using flags takes a little bit more space it is O(N) in space and time and is robust to multiple missing integers.

Cordero W
profile image
I knew something seemed off about that. I was like "but computers have lots of space these days. 100 isn't bad. Maybe 10000 or so."

Matt Walker
profile image
Ya, I noticed that as well. Although my first instinct was to sort and then compare, when I read his "strong answer" I thought that was much stronger actually. Plus it's storing 1 unsigned integer as opposed to the space that would be required to efficiently sort the array.

Jonathan Jou
profile image
I'm so glad you brought this up! I hope I never wind up in an interview where the interviewer's best answer isn't the one I believe in. As an ivory tower graduate student, reading that part of the otherwise highly informed article made me fidget a little.

More interestingly, the sum subtraction method is robust to any continuous sequence of integers, but once you move to letters, colors, or otherwise unordered sets, then sorting makes no sense at all. If you allow duplicate items in the array, or for more than one item to be missing from the set, the 100 flags method comes a lot closer to being the right solution! The optimal solution would be to impose some sort of order, after which you'd create a process closer to union-find such that a contiguous sequence of items could be stored as one flag and not have on flag for each item.

...Computer Science is fun! That is all.

Eric Raboin
profile image
Ouch, I noticed the same thing. I'd hate to think someone gets dinged on an interview for giving the better answer =p

Amir Barak
profile image
@Jonathan
Actually the optimal solution is one that cost you the least in the currency you are willing to spend; developer time, performance time or maintenance time. Over-engineering a solution to be robust against all future cases is a rabbit hole...

And this is exactly what's wrong with technical questions like that in interviews; they lack the application/social context to tell interviewers and interviewees whether the position in question is suitable for the applicant.

Jonathan Jou
profile image
Agreed, Amir. The first question to ask, even while in the rabbit hole, is "what are the requirements?" Once you have real-world constraints the problem is a different beast entirely, which is both more interesting and less obviously about theoretical limits. I wasn't championing union-find so much as I was emphasizing your point: the "best" answer depends on the problem.

On the other hand, "over-engineering a problem" is no different a gamble than any other business decision. I've been in the thankless situation where I've been asked to write a less robust solution that would take longer, be less robust, and eventually be rewritten. There's a fine balance to be had between doing it right and doing it twice, which isn't always as easy to recognize.

Michael Joseph
profile image
And if you are the programmer and you make it through the gauntlet, when they finally get around to asking you to join the company, demand double the industry rate for the position and your years of experience. If they want "the best" and they're convinced they found it, then they're just going to have to pay for it.

Then you'll know if they really want the best or if it was all alot of power play.

Mathieu MarquisBolduc
profile image
I'll throw my 2 cents. First, it takes *months* to properly evaluate the technical skill of a programmer. A lot of the very best guys (or gals) can't answer technical questions under pressure, and some people who are good at it can still be messy coders or terrible at software engineering. The point is, its not a good interviewer im looking for, its a good programmer. Thinking on your feet is not all that important in day to day work. So, I usually focus on the basics, and on methodology, and I avoid any tricky questions or questions with a catch.

Since you can't really interview for technical skills, the most important thing is to know if he is going to fit with the team he is meant to work in, so he should be interviewed by people of the team. Skills can be taught, but fixing attitude is almost impossible.

Marc Mencher
profile image
Awesome feedback and comments!


none
 
Comment: