Since starting Lattice in 2015 we've hired about 130 people. I've come to believe that one of the most underrated secret weapons for hiring is reference checks. Reference checks done well can contain real signal that can help you avoid making costly mistakes (either false positive or false negative).
People don't do reference checks enough because 1) most people are somewhat lazy and 2) they don’t think there’s enough signal to make it worth it. But the truth is references don’t take that long (especially compared to the countless hours you’ll waste if you make the wrong hire) and, more importantly, reference checks can be an extremely valuable part of your candidate evaluation process because they can distill years of data that you simply can’t reproduce in even the longest of interview processes.
So do reference checks and consider them a core part of your evaluation process. Here are the steps I've used:
Set expectations with the candidate
Let the candidate know that the final part of the process is reference checks so it's not a surprise when you ask for them. This doesn't have to be weird; usually during the first interview or while scheduling a second you'll lay out the steps, so just include this there. E.g., "Next steps are a meeting with X and Y, and if that goes well, we'll schedule a half-day onsite. From there the final step is reference checks and role scoping with your would-be manager."
Get a mix of provided and backchannel references
When you're near the end of the interview process, ask the candidate for 3-5 references. Ask for the list to contain people they've worked with in different relationships (a manager, a direct report, a peer, etc.), and ideally in different contexts within a company or at different companies. The closer they've worked together the better. Ideally, you’re looking to speak with people who have a strong business relationship but a weak personal relationship, since those are the people with the most referenceable content and the least partiality. In practice this is hard to find, and that’s okay.
Let the candidate know that you'll also do a couple of backchannel references, but check to make sure there aren't any off-limits groups. Frequently, the candidate won't have told their employer they're looking, in which case doing a backchannel reference with a current colleague could do real damage. Don't do this. But backchannel references can be an important part of getting a full picture on what a person is like to work with, especially with more senior candidates with longer work histories.
Running the calls
Generally, I've found 10-15 minutes is enough time for these calls. You don't need much small talk or tons of questions, but rather just some clear context setting and a few direct, important questions. Here's the basic format I use:
Thank them for taking the time. Quickly explain what your business does, and give a little background on your stage and the general state of affairs.
Tell them about the role you're recruiting for. Describe what you think the job will be like, why you're hiring it, and what kind of person you think you're looking for.
I start by asking them to tell me about how they know the candidate, in what capacity did they work together, and a bit of context on the nature of their working relationship.
Then I ask specific questions based on 1-3 areas where I most want to learn more. This might be ability to thrive in a startup environment, cultural / stylistic questions, management ability, technical proficiency, or whatever else. I usually pick a small number of areas so I can ask about each topic in multiple ways, kind of like those surveys where you're asked nearly the same question over and over. It feels strange when you're asking these, but it really does help get you closer to the truth.
Wrap up by asking general questions to try and gauge how strongly they feel about the candidate. Questions like, "Is this person in the top half, quarter, or decile of everyone you've worked with?" or "Do you think we would be crazy to pass on this candidate?" or "If you were starting a new company or team would this person be your first hire for the role?" are good types of questions to help you get a sense of whether the person thinks the candidate is solid, great, or truly exceptional.
What you're looking for
On a reference call, the normal rules of interaction are slightly altered. You don't all of a sudden assume people aren't being honest, but you do discount the strength of assertions by a couple notches from what you might in a normal conversation.
This isn't because you don't trust people, it's simply because people tend to like the people they've spent a lot of time with, the stakes are high since a potential job is on the line, and the cost of embellishing is zero since the referencer frequently doesn't know you or necessarily expect to interact with you again. Think about what you'd say on a call about a colleague you knew well and liked, but who was a solid, not exceptional, performer. You'd probably be generous.
You're looking for very strong signal. If you ask them if they're in the top half, quarter, or decile of people you've worked with, you're hoping for an answer along the lines of "top 1-5%". If you ask them if they think you'd be crazy to pass on a candidate, you're hoping for something like "completely crazy. And if they do join please call me so I can apply too."
When referencing our VP of Product, one of the referencers didn’t know the candidate was looking and clearly start trying to feel out whether or not she could try to hire him. And when referencing our head of customer success, her former CEO’s answer to the “what percentile” question was “no percentile, this is the number one person I’ve worked with.”
Obviously, you won't always get such a strong answer, and that doesn't mean you shouldn't hire the person. But directionally I wanted to illustrate the type of answers you're ideally looking for.