Ah, hiring.

You need something done. A new project, augment a team, replace someone who has left, get a project back on track, whatever it is. And because you are a good manager, you know how to delegate. So, you ask Susan to put together a job description and get it posted.

She then asks whoever she thinks can best help define the requirements of the role, who sighs while reading the email or chat message, then asks Chat giptity, Claude or Copilot to “flesh it out” based on a few general requirements, then replies to Susan, who dumps the list into the standard template.

She does a quick check. It’s a senior position, so she makes sure to add they need 10+ years in a program that has only existed for 4. But she doesn’t know that, how can she, it’s not her area of expertise. Then she does a final once over and submits it because she doesn’t want to hold the team up for Donna’s “Employee Excellence” celebration lunch for fixing that reporting issue last quarter.

Done.

Then the applications and resumes come in... and there is a lot of noise. A lot of people want a job, and will apply to anything within their target market and hoped salary range. Some might even read the title. One or two might even read the description.

But nobody has time to review and interview that many people, so the ATS filters out the ones that aren’t even close. You get some from one or two recruiting agencies, which have hopefully done some of that work for you. Susan then filters the rest the best she can.

Still... too many for you to interview. So you arrange rounds. Some of the team can filter out candidates before you talk to whoever is left.

Your turn, and you glance over the notes about the general impression and the attached resume while introductions are being made. You prefer to read it ahead of time, but the last meeting ran long.

A few questions in and a small breath escapes as the usual sinking feeling hits your chest.

“Are these the best we could do?”

Maybe. The best might be very happy working for your competitor. Or you might be trying to hire good talent at a bargain, and the market is currently giving you an education.

Or, sometimes, the best people got filtered out before anyone even heard them speak.

But not for the reason you probably think.

As with all selections, there is bias.

In data science, what data you choose to include often has more of an effect than what you do with it. In machine learning, the technical discipline that powers “AI,” this is massively important, because the resulting model will, by definition, be biased toward the data you give it in training. Poorly picked data, and you'll likely make a situation worse.

You may very well have seen the famous diagram of a WWII airplane covered in bullet holes. If you haven’t, congratulations, you have now, so the next time someone drops it into a PowerPoint you can feel slightly more impressed with yourself.

The military wanted to improve the survivability of planes. The idea was simple. Let’s look at what parts of planes typically get hit, then add armor.

One flaw. And it’s obvious.

All the data they had was from planes that made it back, i.e. they got shot and survived. This is survivorship bias, you are analyzing only what makes it through your filters. So adding armor to those areas wouldn’t improve outcomes much, they can already survive it.

To solve this, Abraham Wald, a statistician working with the Statistical Research Group (soldiers win battles, math wins wars), recognized the problem and modeled where the downed planes were likely being hit based on the distribution of the surviving ones.

In hiring, by the time people familiar with the role are speaking to candidates, many have already been filtered out. You are only speaking to the survivors. How the post was written, where it was published, which agencies you used, how the ATS ranked and filtered, how Susan evaluated, and who didn’t make it through the rounds. Your perception of the candidate pool is biased by all of those.

But the most interesting selection bias, and the one that leads to stagnation, is subtle.

Before that, though, there’s another problem.

Time has already changed things.

That job posting was a snapshot in time. What you needed right then, or what you thought you might need in the future.

But by the time that offer goes out to the final candidate, things have already changed. Customers have changed priorities, budgets get reallocated, direction gets shifted, the market is different. And it will continue to be different when the person is finally onboarded and productive, possibly 3-6 months after you asked Susan to write the original post.

If you are hiring a role, you are already behind. You are hiring for where you were, not where you are.

But you are never hiring a role anyway, you are hiring a person. And a person is so much more than a role.

The entire hiring process, though, isn't built around the person. It's built around the role, and that biases who you even get to interview.

The Subtle Bias

What kind of person does the hiring funnel select for?

It’s obvious, right? The person who is best qualified for the role, preferably someone who is already an expert at doing that role.

Okay.

Who applies for a job they can already do?

And there it is. Did you catch it?

It's someone looking for familiarity. Better pay, better title, better company, maybe fewer inspirational quotes on the walls, but fundamentally they already know the work. Looking for conservative change, not risky jumps.

Then there’s the person looking at a role they haven’t done before, but sounds interesting and thinks, “I can do that.”

Maybe they have the same underlying skills, critical thinking, judgment, communication, leadership, systems thinking, whatever it is, but they have applied them to a different problem set. Maybe their history is a series of jumps into unfamiliar work, figuring it out, getting good at it, then jumping again.

Those people have gaps. Of course they do. They are trying to do something they haven’t already done. And they are consistently taking risks to do it.

But your hiring funnel is designed to filter gaps. The ATS sees missing titles, tools or years of experience. Susan sees someone who doesn’t fit the role cleanly. The hiring manager sees someone who is harder to explain if the hire goes wrong.

Meanwhile, the person already doing the role moves cleanly through. After all, the hiring manager was probably hired via the same risk-averse funnel. So why take the risk?

And yes, sometimes that is exactly what you want. If I need brain surgery, I want the surgeon who has been doing the same thing with excellent results for 20 years, not someone who is excited about a learning opportunity. Some positions need deep mastery of a specialized skill. Some, if not all, businesses need some people who are conservative with risk and change.

But when you need people who are adaptable, willing to take risks, looking for a challenge, able to learn your business and adjust as the market changes... well, you might never even be talking to them.

And yes, it's hard to filter that person from the desperate person spamming every post.

But if you only ever look at risk-averse candidates, don't be surprised when your entire company is risk-averse and resistant to change.

Don't be surprised when you stagnate and the market moves on by.