Hiring: false positives and negatives

There’s a view of hiring that I think is becoming almost a commonplace in the Valley. It’s stated well by Joe Kraus, and also in this entertaining book on Microsoft; the basic idea is that you should screen really rigorously and not be afraid to turn people away, because the cost of a bad hire is so much greater than the cost of not hiring some particular good person.

I don’t feel certain that this view is wrong, but I’ve always vaguely mistrusted it, and I think I can finally articulate why. The logic is fine, given the assumptions about costs and benefits, but those assumptions are really questionable. Which hiring strategy makes sense really depends on the particular employment world you’re living in.

Here’s a sketch of a world (A) in which the “No false positives!” rule about avoiding bad hires makes maximum sense: In this world, there are “good” candidates (i.e. people who will turn out to be able to do the job), and “bad” candidates (people who, whether or not they’re competent in other ways, won’t be able to do the particular job). And let’s add the following assumptions:

  • The pool of good candidates is very large
  • “Good” candidates are roughly interchangeably good
  • “Bad” candidates are completely useless, and even harmful
  • Once hired, a “bad” candidate will be with you forever

Clearly, in this world the only way to win is to minimize the number of bad candidates you hire, and so it’s incumbent upon you to be extremely careful to never hire one.

Now, let’s try an alternative world (B). In this one there are “bad” candidates, “good” candidates, and a third category of “game changers” or “secret weapons” — people that are so good that they make a serious qualitative difference in what happens with the group. (This could be someone who’s amazingly productive in writing code, or someone with great ideas, or even someone whose personal qualities makes the whole thing gell.) And for extra assumptions, try these:

  • The number of good candidates is unknown but probably not large
  • The number of “game-changer” candidates is unknown but most likely very small
  • Bad candidates are still (in this world) useless, even harmful, but
  • Candidates who turn out to be “bad” can be fired, or otherwise rendered harmless

And one more assumption (and this is the kicker): by ruthless screening you can maximize the chances that the candidate is “good”. This cuts both ways: you reduce the chance of a “bad” candidate, but you also reduce the chances of a “game changer”.

This last assumption may seem unlikely, but I don’t think it is. As an example, take a candidate who’s really smart and motivated, but who recently changed fields. The chances of a “bad” hire are higher, since you don’t have the usual creds to rely on; on the other hand, you’re hiring someone who a) had the guts to change fields, and b) may now know what they really want to do when they grow up…

Clearly in this second world, the only way to win is to take a chance occasionally. Also, by doing so, you may get the extra benefit of some diversity of view and experience that you lose by insisting that everyone pass the same set of “good” tests. It _is_ risky, especially since I agree with Kraus that if hiring mistakes get to hire other hiring mistakes who can hire themselves, then all is lost. But there’s more than one way to lose.

So it all comes down to which kind of world you believe yourself to be living in, A or B? And note that there are really two parts to the belief: 1) what you believe about individuality, game changers, ability to evaluate people, etc, and 2) what you believe the policy on firing is. :/

The other interesting thing about false negatives (people you should have hired and didn’t) is that you cannot learn from them — people you don’t hire usually just disappear into the pool, and you never hear from them again. So, if your belief that false negatives didn’t matter was wrong, how would you ever learn that it was? This alone makes me suspicious of the “No false positives!” view — I wonder if it’s partly just due to that kind of bias where your hiring mistakes are in your face, and your non-hiring mistakes are invisible.

(A final confession: I obviously lean toward world B, but that may when it comes down to it be the purest touchy-feely wishful thinking. I would like to believe that “good” employees are not interchangeable, that taking risks on people can sometimes pay off, that our abilities to rate candidates on unidimensional scales are not as good as we think they are, and so on. So take this all with a grain of salt. 🙂 )

5 thoughts on “Hiring: false positives and negatives”

  1. What you say makes sense especially if ‘good’ means ‘just like me.’ You make think you’re optimizing for a small number of desirable traits–intelligence, creativity, for example. But if you’re hiring ‘just like me,’ you’ll tend to hire people with your negative traits, too. Some of these will be explicitly bad (you might be a procrastinator, or tend to use chi-square tests inappropriately), but others will be implicitly bad, especially in the way you discuss. That is, you lose the special knowledge or abilities you would gain if you had hired someone not ‘just like me.’

  2. This is actually very pertinent to me. I’ve just been hired by a very selective company after a rigorous and well organized interview process.

    I did horribly on the programming tests (first interview in many years, first ever whiteboard programming). But I did well on other aspects of the interviews. The day after my would-be manager called to express concern over the tests, and to ask what I thought should be done. I told her I would send her an email.

    In the email I suggested a few interesting avenues for further testing to belie worries. I also took the initiative and wrote some interesting algorithmic code for an open-source app of mine, and pointed her at it.

    I called the day after, to be told that I was hired.

    My point is that real game-changers are strong enough to be “good” candidates, but it may take some finesse. You have to understand the game well before you can change it.

  3. […] team over someone who has unanimous but lukewarm support. Tim Converse wrote a good post on “Hiring: false positive and negatives” that analyzed categories 2 and 4, noting that allowing for some number of 4’s allow […]

Comments are closed.