Chapter 12 - The Importance of Reference Checking
DIGITAL COMPANION
To read:
Remembering Keith Benjamin: Brett Hurt memorializes his friend and mentor, the legendary and widely respected venture capitalist Keith Benjamin.
The invaluable co-worker: Intuit founder Scott Cook describes why there is no better way to learn about a candidate than talking to those who have actually worked with them.
You’re not born a CEO: Brett Hurt argues the principles for founders to become good CEOs in this First Round Review interview.
Chapter 12
The Importance of Reference Checking
“References get you to the truth the fastest, provided you dig deep and talk to the right people. Don't just hire slow; hire smart.”
— Max Winograd, CEO NuLabel Technologies
Akin to my counsel on recruitment in Chapter 11, The Most Proven Way to Hire Well, the prevailing approach to the related art of reference checking is outdated, broken, and hamstrings many an entrepreneur before they even come off the starting line. But unlike my experience recounted in the last chapter, my insight into reference checking was not learned through a series of early stumbles, but rather by (perhaps) naively getting it right at my first major startup, Coremetrics, beginning in 1999.
I’ll share that experience that has now served me so well from that company, through Bazaarvoice from 2005 to 2012, and now at my current firm, data.world, since 2016. That is, it has served me well when I followed my own advice, and it has sometimes gone horribly wrong when I haven’t. But first, the two key lessons I’ve learned over hundreds of conversations with references and gleaned from the wisdom of others.
The first insight I encourage you to consider is that most companies and their HR departments devote energy to checking a candidates’ reference in a fashion inverse to the importance of the position being filled. In other words, the higher the level being filled — from entry-level, to mid-level, to manager, to executive, and on to board member or investor — the greater the reluctance to check references. It’s only human nature, understandably, to be most keen on validating the background of an inexperienced person newly out of school with a thin resume. You need to be sure the candidate really did graduate with honors, or did volunteer at the non-profit, or really does read and speak fluent Mandarin. But if the candidate is a senior vice president at a brand name company, supported with glowing written testimonials, or a well-known VC who might back you and join your Board… well there’s certainly no need to check on their well-established bona fides.
Right? Absolutely wrong!
The second insight I hope you’ll internalize is that reference checking is not — or should not be — about “checking.” Widespread assumptions and practice are that candidates’ applications, references, and qualifications are simply to be validated. Sure, you need to make sure they haven’t fibbed about grad school or concealed a conviction for embezzlement. But the requisite ticking of boxes should be the least of it. Rather, the reference consulting process — my preferred term — should be about getting to know the candidate through the experiences of former colleagues, bosses, or subordinates. This is where you learn how they react to stress or criticism, what their work style is to get things done, and what their skills of collaboration look like. There’s simply no better way to get to know how a prospective employee, partner, or Board member is likely to work out than to talk with as many people as possible who have worked with them, for them, or supervised them. Think of it as dating before you get married. As with a long courtship, a rigorous reference consultation ultimately makes for a better contract.
So to the first insight, when I started Coremetrics, Accel Partners (now Accel) wanted to invest in our first major fundraising round, our Series A, alongside Highland Capital Partners which we had already chosen as our lead. We were impressed with Highland’s Keith Benjamin, a legendary investor and Wall Street visionary and he was joining our Board of Directors. (Sadly, Keith passed away in a tragic accident in 2008. He was an incredible friend, about whom I think often. I share a memorial I wrote about him on the Digital Companion).
Accel, meanwhile, put forward Arthur Patterson to join our Board alongside me, Keith, and our independent Director, technology entrepreneur Bong Suh, who remains a really terrific mentor more than two decades later. Arthur was, among other things, Accel’s Co-founder, a former senior official in the U.S. Treasury Department, a venture capitalist since 1973, and the one-time Director of the National Association of Venture Capitalists. Arthur was a towering figure in the VC world and it was an honor to have him on our team. But as I had done with Keith, I insisted on checking Arthur’s references.
Most people at Accel were surprised, and I think they thought I was naive at the time. After all, I was a 26-year-old CEO and they probably chalked it up to inexperience. And when I called his CEO references, some of them expressed a lot of surprise that I had the moxie to do so. But those references turned out to be extraordinarily helpful to me, especially on how to best work with Arthur as a business partner. For example, they told me to treat Arthur as a true partner, no matter how scared I may be about something happening inside or outside of Coremetrics, and share bad news as early as possible to get his advice. This guidance from Arthur’s peers certainly helped us quickly match our collaborative cadence. But also, rather than be somehow offended, I believe Arthur had more respect for me as a result of being one of the first entrepreneurs to check his references.
But to the second insight I shared above, of course I wasn’t checking to see if Arthur had really gotten his MBA from Harvard, or that he indeed had a dozen or so IPOs and private exits under his belt. Of course he did. What I wanted to make sure of was that the styles, outlooks, temperaments, and values would align with our leadership team. I couldn’t see any other alternative — I deeply loved Coremetrics (I still do) and I wanted to make sure that we fielded the best team possible. That included our investors and our Board of Directors.
Needless to say, I carried this practice with references on to Bazaarvoice a few years later, at least initially. As I detailed in Chapter 11, The Most Proven Way to Hire, we had learned to become broadly obsessive about both our hiring processes and I sought to be similarly so in our reference consulting approach. In discussion with candidate references, I would probe and probe until I got something useful to work with. Usually, I was able to get to some of the “areas for improvement” by saying something like, “Please level with me. I most likely am going to hire them but I could really use your advice about how to best coach them to perform at their best from day one.” That type of personal plea was hard to ignore. They had been there too, with that same person. If that didn’t work, I would ask them to walk me through the areas for improvement on their last performance evaluation. We all have them — no one is perfect.
Over the years, I’ve refined the process, and the time and diligence have certainly been justified by our high retention rates, strong corporate cultures, and commercial success at all of my companies. But as discussed in Chapter 4, The Importance of an Always Be Learning Life, one should always be acquiring new skills. And this was the case not long ago when I was invited to speak at the annual CEO Summit of First Round Capital, the seed-stage VC firm specializing in technology investments. First Round was a great partner as an early Bazaarvoice investor, and co-founder Josh Kopelman was an invaluable source of guidance. A link to First Round’s brilliant (and free) publication, Review, is on the Digital Companion and I’ll have more to say about my talk at the summit in Chapter 14, Seven Lessons Learned on the Journey from Founder to CEO.
But the highlight of that CEO summit was, in fact, less my own presentation than the talk given by Scott Cook, the Co-founder, and CEO of the financial software giant Intuit and later its executive chairman. Both talks are also available on the Digital Companion, but I want to quote here an excerpt from an account of Scott’s that was published in the Review, in which he underscores the importance of four qualities which he seeks insight from references: intelligence, learning ability, care for the quality of work and customer satisfaction, and the ability to get things done.
“Keeping these four key attributes in mind, Cook gets on the phone with the reference and begins by letting them talk about working with the candidate. However, the reality is people generally want to be nice and don’t have much to gain by being fully transparent and honest, so they generally start by saying how great the potential hire was and how valuable they were to the team. Cook has found that it’s most effective to completely ignore this opening feedback and throw it out. Once they’re done rambling on, Cook asks, “Among all of the people you’ve seen in this position, on a zero to ten scale, where would this person rank?” They go, “Seven.” Cook says, “Why isn’t this person a nine or a ten?” And then you’ll finally start learning about what this person really thinks.
Cook concludes the call by asking this person for other people who can give a reference on the candidate and then begins the process again. Getting as far away as possible from the candidate’s suggested references often leads to the most valuable data.”
The brilliance of Scott’s approach is the comparative aspect. It forces the reference to stack-rank the candidate, and the reference’s integrity will most likely force them to be candid in response to such a direct and comparative question.
Now I do want to note with candor here that while I was on the right way to do reference checks early in my career, I have taken my eye off the ball to potential detriment at least once. This is why above I qualified with the word “initially” when I wrote about Bazaarvoice being diligent in checking references. One of my scariest times in the company’s history was when I learned that our head of recruiting at the time had decided to stop checking references. I found out about this when one of the people we had hired had started to engage in activities that I choose not to disclose here. I started to probe how that happened when we were so diligent at checking references. “We stopped checking references,” I was told to my shock. I couldn’t believe it! I asked why. “Because they don’t really tell you anything — they are all provided by the candidate.” I then spent the next 30 minutes telling stories about how important it was to always check references, and specifically how to probe. I also talked about the importance of going off-script and finding references that they didn’t provide. We reverted to our original diligent reference-checking practice immediately.
I really don’t believe the dangers of failing to check references can be exaggerated. This is broadly the case, but especially so when you are interviewing Board members or raising money from new investors. And as counter-intuitive as it may seem, I’m convinced as I wrote above that the higher level you go the more important reference checking actually is, and the more like a marriage it will be.
So let me share two horror stories so that you can understand why I’m so passionate about this.
With names changed and details fudged to protect the guilty, here’s horror story #1:
In this case, the leading protagonists are two CEOs and a CEO recruit. The first CEO is leading a company we’ll call Acme Technologies, with great potential, a promising product, and several million dollars of funds already raised and in the bank. But, the Board of Directors has ultimate control, has tired of Acme Tech’s founding CEO, and is convinced he needs to go and that they need a real professional in the job at this stage. But who to recruit?
As the fates would have it, there’s another CEO of a complementary company, Balance Resources we’ll call it, who is familiar to the Board of Advisors. They know this CEO because he serves on the Advisory Board of Acme Technologies. Mind you, as an Advisory Board member he has no executive authority, or decision-making shares, but he carries out the role of a sounding board. He’s an advisor, after all, so they go to him for advice. Actually, they sort of go halfway when a new candidate for Acme Tech CEO — currently in a sub-C Suite management role at Balance Resources — applies for the soon-to-open position. This is someone whose boss has repeatedly lauded in various meetings. So assumptions are made without questions. The Advisory Board member has spoken highly of his employee now seeking to move up in the world. If he’s from the Advisory Board member’s company, he must have recommended him? He must be good? It would be offensive to the Advisory Board member if the Acme Board of Directors deigned to ask after the candidate’s bonafides. Would it not?
As you might imagine, this led to precisely the scenario I’ve warned about. The founding Acme CEO was ousted, the new Balance Resources manager was hired as the new CEO, and things promptly went to hell in a handbasket.
Naturally, the Acme board then turned on the Advisory Board member. His response was they had never consulted him on the candidate, just made a bunch of feeble assumptions. Yes, he had spoken highly of the candidate’s brilliant individual contributions. A good guy to be sure. But lots of brilliant contributors, in sales as a common example, make lousy executives. And in this, the guy who they hired as CEO had already batted out as a manager. Every one of his hires has to be let go for performance issues. Had they asked, he would have explained to them that they were making a huge mistake. But instead, they assumed.
As the truism usually credited to Steve Jobs goes: A’s hire A’s. And B’s hire C’s. This was a textbook case, just one of many in my experience. The company was eventually sold for much less than was invested and everyone lost money.
In horror story #2, I once got a reference call about a candidate for a public-company CEO position who I had worked with in the past. Their candidate had not given me a heads up that he was looking for a new job. The person on the other end was an executive recruiter and I remember thinking, “Cool, they are going off-script and I’m a very honest reference.” The recruiter started to ask me their normal “reference check” questions and I quickly became disenchanted. I started to give them some of the cons (there were many pros that I had already given them), but they would quickly change the subject back to the pros. It was clear to me that the recruiter just wanted to check the boxes and get the job done. Anything they learned from me on the cons they would have to report back, and so they chose to not hear them at all. And then it hit me: I couldn’t believe that a public-company Board of Directors had delegated the task of checking the references on a new CEO to a recruiter. Did no one on the Board have the time? What about the Chairman of the Board? What about the Governance Committee? It was a bad sign, and although the person they recruited was pretty good (at least in my opinion) — the company didn’t last for very long.
The bottom line is that checking references is a critical activity and therefore it should be a cherished, celebrated practice. You are, after all, recruiting people to join your company and there is both a real financial cost as well as an opportunity cost — and a cultural cost if they don’t work out — to carefully consider. Thorough reference checking isn’t a foolproof practice — nothing is. But it is a very important one. And don’t forget that as CEO it is a lot easier to let go of a team member who isn’t performing than it is to dispense with one of your investors or Board members. You can get star-struck with someone’s credentials and awards. The more they have accomplished, the more tempting it is to skip a check of their references. This is counterintuitive but true. An extreme example of this is the disgraced financier Bernie Madoff - how many of his prospective investors really dug into the viability of his financial results? Social signaling and intuition will bias you to spend a lot of time checking references on junior or “unknown” team members and very little time checking references on the most senior. So please follow Scott Cook’s advice and truly dig in. The higher level your candidate is, especially potential investors and Board of Director members, the more you should check. Go off-script. You’ll be very thankful that you did.
"It was a rather delicate and awkward situation. You should call her other past employers. I made the mistake of not doing that."
— Comment made to a reference checker, reported on CBS News by correspondent Suzanne Lucas