What's the secret to hiring great programmers? Do you have to have a Google-style cafeteria, or Microsoft-style interviews, or Scrum or Agile? All of those things help, but keep in mind that it's also possible to find great talent the major companies let slip through their fingers.
The club, the Oakland As, used clever statistics to discover certain kinds of players who were not being efficiently targeted by the market. They worked out an estimate of how much these players should be valued, and ruthlessly targeted them. Because these players weren't in demand by the other clubs, the Oakland As could hire them inexpensively and give them a chance to play great baseball.
Think this would work with engineers? It's gotten some press recently. In fact, IEEE Spectrum recently interviewed author Stephen Cherry about this topic. Cherry wrote a book about the disconnect between job applicants who are clearly qualified, and companies who claim there are no qualified applicants. TL:DR; he claims that companies write their job descriptions to be "exclusionary," thus filtering down their list of applicants but yet failing to find a candidate they like.
This certainly seems like an opportunity for Moneyball.
I'll share with you another secret I discovered: in the 1990s, I briefly worked for a company that wrote resume processing software. During that era, companies would receive thousands of paper and fax resumes for an attractive job listing in a newspaper, and they had to somehow filter a huge pile down to a smaller, meaningful list. The software I wrote helped them filter candidates based on keywords that appeared in the resume. We wrote lots of search tools that helped people find complex and unusual skills…
But do you know what happened? Basically, every company filtered on "bachelor's/master's degree" and "years of experience". Those two filters proved to be quick, simple, easy to find, and helped them narrow down their lists considerably. By excluding anyone without these magic items, the busy HR manager could focus on only the key 10-20 percent of the available resumes.
This clearly happens to this day. Take a look on any job listing website and you're virtually guaranteed to find:
"Bachelor's degree in computer science or a related field required" – this is exceedingly common today; ten years ago most job listings said "… or equivalent experience."
"3-5 years of experience with [current fad technology]"
"3-5 years of experience with [development methodology]"
And, in the gaming field, "Must have shipped at least 3 AAA games."
You might be forgiven for not knowing this, but C. Northcote Parkinson wrote about this strategy in his 1958 book, "Parkinson's Law". He described how you want to create a job listing so fiendishly specific and devilishly challenging that you only receive one applicant, whom you can hire right away without wasting your time evaluating the rabble.
When I read through a two-page long job listing that specifies dozens of "requirements" for a candidate, I can clearly see that the HR person is really trying to cut down the number of resumes they have to read.
So now that we've covered a bit of the field, let's get down to business and create a theory about programmers.
Who is being undervalued?
Given what you've read so far, I would argue that the software engineering industry tends to overvalue things that are easy to measure. First of all, if you want value, you can't focus on people who have advanced degrees, those are already in high enough demand that they'll be snapped up right away.
Find programmers who have innate talent but for some reason didn't want to go through a university; find open source programmers or hackers or people who write mod scripts in their basement while getting a degree in English Literature.
Second, we tend to overvalue experience with a specific current hot-button technology. The technology industry changes frequently, so look for programmers with experience in something that was hot five years ago but isn't now.
Remember, good programmers understand serious algorithms and can often switch languages without breaking a sweat. So if we see, according to TIOBE, that C++ and Perl are declining in popularity, maybe that's a good indication to hire experienced C++ and Perl programmers while their skills are "out of fashion".
Third, we overvalue people who have exactly the right number of years working in a specific field. Maybe the hiring manager has an idea that they want someone with 5 years or 10; but instead, we could look for people who have an opportunity to surprise.
One example is a junior person who is chomping at the bit to get a chance to prove themselves; another is a senior person who has been seeking a specific opportunity. In either case, the degree of motivation or hunger this person has for your job opening can drive them to deliver much more value than if you narrowed it down to a limited time range.
All these criteria have one thing in common: measurable text on a resume is being overvalued. What's being undervalued is the stuff that doesn't show up in measurable "certificates", "degrees", or "years of experience."
How can we take advantage of this?
It may not be possible to hire every programmer the "Moneyball" way, but an interesting feature of the way teams work is that they have a maximum size. When a team grows beyond 6-8 team members, new dynamics start to form, and the group tends to split up into lots of separate subgroups even if they are nominally all together.
This means that you can safely build teams with a few senior members who are well trusted and capable of mentoring and guiding junior members, and use moneyball strategies to attempt to fill out the teams to full productivity. So let's give this a try.
Clearly, we're going to have to attract different candidates, and review them differently. First, write your job listings to be very open-ended. Don't write a gigantic list of requirements; instead, describe what you are doing, and make the job description drip with excitement (of course your work is exciting!). Describe your office culture and the kind of work you do, but leave out all the details of the specific tools:
Searching for talented programmer who wants to write user interface script modules for a multiplayer app for phone and web customers. The programmer will work in a small group of 2-3 front end team members, frequently prototyping new ideas and implementing new graphical features for A/B testing.
The goal of your job listing is to get lots of resumes – probably more than you would get the traditional way. Then, don't filter the resumes by experience; instead, read the cover letters! Call the people whose cover letters sound most exciting and relevant. Make sure they know how to communicate verbally and describe their desire to work in your field.
Do a phone screening first. Get an idea how much these people want this opportunity, and then gauge how suitable they seem for the work. Quiz them on whether they're motivated enough to have done some research themselves.
Does it work?
Of course it works! The dirty secret of the recruiting industry is that recruiters live and die by exactly this strategy every day. They take piles of resumes and piles of job listings, then contact promising candidates and ask if they can make "just a few changes" to the wording of their resume to emphasize a particular feature or highlight something that makes them just right for an opportunity.
Here's a common scenario:
Recruiter: Hi there, I read your resume and think it's great for a job opening I have. First question, though, do you have any experience with Java?
Candidate: Well, I wrote a program in Java last year to organize my home media file server.
Recruiter: Great! Can I ask you to send me an updated resume with that project listed?
This kind of thing exposes how trivial the difference can be between a resume that gets dropped and one that wins the job. Companies that focus too much on resume details are missing out on candidates who happen to be very talented but lack a magic keyword.
At EEDAR, I decided to use a hiring process that brought junior team members into our group regularly as interns, and gave them each a chance to demonstrate their skills and grow into a full-time programming position. I search for candidates who specifically had an interest in programming but had no experience – and many of them came through with flying colors.
So clearly it can work, but will it, in your case? You'll need:
Persistence and resilience to keep trying experiments to find programmers the market may have missed.
Time and attention to pay to them to see if they really are worth their time. People overlook this frequently: in the book Moneyball, not every candidate was an instant success! Some of them needed mentoring, some needed time, and some failed and had to be traded away.
The ability to acknowledge when you make a mistake. If a programmer seems promising but doesn't excel within your company, you have to be honest and forthright about it, and give them a chance to move on somewhere that's right for them.
Above all, a desire to reach out to people who may have a hard time getting their foot in the door elsewhere.
If you do your part, we can all find new ways to hunt down talent the market may have overlooked.
Postscript
I'd also like to mention one idea that may be relevant. Recently, "quirky interviews" have become the norm in the industry, such as the Microsoft Interview or Guerilla Interviews or Google Interview.
These interview strategies are extremely challenging, and tend to weed out people who can't think on their feet or who get overwhelmed with certain types of interpersonal pressure. It's entirely possible that these interviews systematically exclude a certain class of person, but I don't yet have a specific suggestion about how to leverage that idea. I'd welcome your thoughts and feedback.
[This piece was reprinted from #AltDevBlogADay, a shared blog initiative started by @mike_acton devoted to giving game developers of all disciplines a place to motivate each other to write regularly about their personal game development passions.]
In particular, the point about measuring against reality needs to be emphasized. It's an iron law that as organizations grow and age, there is a tendency to rely more on fixed process than on personal judgement. That's not entirely a bad thing (for maintaining a fixed level of quality in outputs) as long as going too far down that road is resisted constantly. Otherwise you get what John Gall (in his wonderful book on systems design, _Systemantics_), following Parkinson, called the Fundamental Law of Administrative Workings, or FLAW: the real world is what gets reported to the system. As processes replace judgement, what gets reported is whatever is easiest to measure.
Eventually reality and what the system thinks is reality are no longer functionally similar. Decision-makers become insulated from the real world. And that's when you start getting products and services -- including games -- that fail to give customers what they actually want.
To defend against that, the hiring process needs to be controlled ruthlessly to assess applicants against what a team actually needs to succeed, not just what's easy to measure. By that standard, the "guerrilla interview" is almost completely nuts.
If you're planning on living in high-stress crunch mode with irrational management, then pelting an applicant for an intellectually creative job with rapid-fire questions might be a realistic assessment of survival abilities. Otherwise, demanding that interviewees for contemplative technical jobs demonstrate an ability to "think on their feet" and be verbally facile bears too little relationship to what those jobs actually require.
If you must do panel interviews, then instead of staffing interviewer panels with extroverts who thrive on stress -- and who naturally give higher marks to interviewees like themselves -- make sure there are some introverts on the panel as well. They will best understand the quiet contemplation that doing creative technical work often requires, and thus can assess applicants for such roles most accurately.
_Quiet_ by Susan Cain explains how much good work is lost by extroverts undervaluing those whose competence is internal. Of course communication skills and being able to work with people in groups matter... but those abilities shouldn't be overvalued just because they're easier to see than the logical, creative, abstract mental work that technical staff actually perform inside their heads.
In short, I agree strongly that hiring specialized creative-technical talent is too critical to team success to let it become mechanized to filtering on easily-observable keywords or fast-talking.
Your point about undervalued quiet employees is very appropriate. I've been guilty of this myself, and it really requires that we give ourselves the ability to take risks and acknowledge mistakes. We need to be constantly re-evaluating ourselves and others in order to see if we've missed something.
I think I may have omitted a link to Joel Spolsky's Guerilla Marketing article, so I'll post it here:
http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html
The hiring manager could allow the interviewee to choose which type of interview he would receive. I'm not sure how practical that is (it seems like it would take an additional degree of planning), but it would allow each candidate to emphasize their strengths.
Good point. This reminds me of my "hippy" college professor that offered everyone a choice of exam format. You could write a paper, take a standard format exam, give a speech, or be interviewed by the professor to determine if you understood the material.
He warned that many people often take the interview because they think it's easier, but it's the hardest of the options. Yet, the people who struggled with written exams or public speaking quite often excelled at the oral exam option anyway.
Great idea to translate it to the interview process, though. I wish I had choices in an interview!
In particular, the point about measuring against reality needs to be emphasized. It's an iron law that as organizations grow and age, there is a tendency to rely more on fixed process than on personal judgement. That's not entirely a bad thing (for maintaining a fixed level of quality in outputs) as long as going too far down that road is resisted constantly. Otherwise you get what John Gall (in his wonderful book on systems design, _Systemantics_), following Parkinson, called the Fundamental Law of Administrative Workings, or FLAW: the real world is what gets reported to the system. As processes replace judgement, what gets reported is whatever is easiest to measure.
Eventually reality and what the system thinks is reality are no longer functionally similar. Decision-makers become insulated from the real world. And that's when you start getting products and services -- including games -- that fail to give customers what they actually want.
To defend against that, the hiring process needs to be controlled ruthlessly to assess applicants against what a team actually needs to succeed, not just what's easy to measure. By that standard, the "guerrilla interview" is almost completely nuts.
If you're planning on living in high-stress crunch mode with irrational management, then pelting an applicant for an intellectually creative job with rapid-fire questions might be a realistic assessment of survival abilities. Otherwise, demanding that interviewees for contemplative technical jobs demonstrate an ability to "think on their feet" and be verbally facile bears too little relationship to what those jobs actually require.
If you must do panel interviews, then instead of staffing interviewer panels with extroverts who thrive on stress -- and who naturally give higher marks to interviewees like themselves -- make sure there are some introverts on the panel as well. They will best understand the quiet contemplation that doing creative technical work often requires, and thus can assess applicants for such roles most accurately.
_Quiet_ by Susan Cain explains how much good work is lost by extroverts undervaluing those whose competence is internal. Of course communication skills and being able to work with people in groups matter... but those abilities shouldn't be overvalued just because they're easier to see than the logical, creative, abstract mental work that technical staff actually perform inside their heads.
In short, I agree strongly that hiring specialized creative-technical talent is too critical to team success to let it become mechanized to filtering on easily-observable keywords or fast-talking.
I think I may have omitted a link to Joel Spolsky's Guerilla Marketing article, so I'll post it here:
http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html
He warned that many people often take the interview because they think it's easier, but it's the hardest of the options. Yet, the people who struggled with written exams or public speaking quite often excelled at the oral exam option anyway.
Great idea to translate it to the interview process, though. I wish I had choices in an interview!