Interview Best Practice Guide

This is a definitive best practice guide into running data-driven interviews. This guide provides an in-depth analysis of the pitfalls of traditional interview approaches and the upsides of progressing your hiring to data-driven hiring with Alooba Interview. Data-driven interviews are just one crucial step to moving away from gut feel, intuition based hiring and instead adopting objective, data driven hiring.

As you will see in this article, traditional interviews suffer from five fundamental problems for organizations when hiring:

Expensive. Traditional interviews are expensive for organizations to conduct, with an average cost per interview of 1000USD.
Inefficient. Traditional interviews are inefficient to set up and conduct with lots of tedious manual work.
Inaccurate. Traditional interviews are inaccurate, with the best candidate often not being selected.
Unfair. Traditional interviews are unfair, with candidates being treated inconsistently.
Opaque. Traditional interviews often involve candidates being ghosted and left in the dark.

Instead of persisting with traditional interviews, would you consider progressing to objective, data-driven hiring? With Alooba Interview, you’re giving all your candidates a fair chance by committing to structural change in your process, focusing your hiring on answering the fundamental question: 'Who is the best person for the job?'.

In this article, we delve into all the details of running data-drive interviews, answer your most frequently asked questions and hopefully cover off some areas you might not have thought much about. Got questions? Feel free to contact us here.

Looking for a blow-by-blow of Alooba Interview’s functionality? Check out a full rundown of the features and capabilities of Alooba’s various products here.

Looking to run data-driven interviews? Get started now with Alooba Interview.

What is Alooba Interview?

At Alooba, our vision is to create a world where everyone can get the job they deserve. It’s fair to say, that world is still some way off! A big part of this vision is to help our customers progress their hiring from intuitive, gut feel based decisions to data-driven hiring, based on facts rather than opinions.

Alooba has several products to help organizations make data informed decisions about their people.

Alooba Interview is a new type of product that helps organizations conduct structured, data driven interviews to select the best candidate and reduce bias.

With Alooba Interview you can:

Conduct a structured interview online or remotely. Structured interviews are proven to be more effective than unstructured interviews at selecting the best candidates and for reducing bias.
Set up and stick to a coherent interview plan. With a clear plan and checklist you can ensure you communicate everything you need to with candidates.
Ask each candidate the same questions, in the same order and under the same conditions.
Consistently and easily grade each candidate’s answer with a clear marking rubric.
Capture interview notes & share actionable feedback. Candidates’ number one complaint in hiring is getting no useful feedback. With a single place to collate, summarise and easily distribute feedback, sharing will be easy.
Leverage Alooba’s question bank of 3000+ questions compiled by experts.

Alooba Interview integrates easily with the rest of your data-driven hiring powered by Alooba Assess.

Is this an AI interview tool?

No. Alooba Interview is not an AI tool. Alooba Interview helps you to run objective, data-driven, structured interviews. While in theory, the potential for AI to create an unbiased and objective interview and hiring process is there in the future, the current state of technology makes that unfeasible.

What are structured interviews

Structured interviews basically mean that each candidate is treated the same, receiving the same opportunity and given the same interview conditions. In a structured interview, these elements should be kept the same for each candidate:

How long the interview lasts for
Where the interview is held
How notes are taken in an interview
The questions that they are asked
The order in which the questions are asked
How the questions are framed
The scoring rubric for each question
The way feedback is provided
Any other reasonable variable that you can think of

By removing as many variables as possible from the interview, you can more easily attribute any difference in interview performance to the candidate and nothing else. This helps you hire the best person for the role.

What does bias mean?

A bias is an error. It’s as simple as that. Bias is not a flaky HR buzzword. It’s real and it represents an error in your decision making. The 2011 bestseller Thinking Fast & Slow by Daniel Kahneman did a great job at laying out all the different types of biases that humans have and also why we have them.

In the book, Kahneman explains that our brain has both a fast system of decision making and a slow one. Both are essential for us. The fast one makes rapid, automatic decisions which allow us to not have to think consciously about everyday trivialities. For example, typing on the keyboard, driving to work, folding laundry or making coffee in the morning.

The slow part of our brain is required for more in-depth thinking and is much more taxing. For example, programming, doing our monthly budget or preparing a speech.

Now, while the fast part of our brain was pretty useful for not getting eaten by a tiger, it’s fair to say we’ve evolved since we lived in caves.

Hiring decisions, in particular, need to be made rationally & carefully, using the ‘slow’ part of our brain. That means more data, less gut feel and intuition. This will give you a more objective and accurate hiring decision. Intuition, as we’ll see below, is simply a heuristic rule of thumb, which is also a bias.

How do I know if I am biased?

Are you human? To be human, is to be biased. That’s what biased is. We’re sure if you review these biases below and you were honest with yourself, you’ll realize that you’re biased too. This is not to admit defeat. Quite the contrary - understanding your own limitations is crucial for your own personal growth. As they say, the first step is admitting that you have a problem. So kudos to you!

What kinds of bias exist in interviews?

There are many common types of bias that come up in interviews. What’s important to remember is what a bias means - bias means an error has been made. Your bias has clouded your better, rational (slow brain) judgement, and you’ve made an incorrect decision because of it. This is why it’s essential to understand and admit biases, so the interview process can be adjusted to correct for the errors.

Here’s some common biases that come up in interview that you might like to be aware of:

Intuition bias

Intuition bias is one of the biggest issues in legacy recruitment processes. Every decision, from sourcing, screening, interviewing and hiring is driven by intuition and gut feel, not actual data (facts).

This comes into play in interviews when you’ll make fast decisions to pass or reject a candidate without any real evidence or justification. You might find yourself getting to the end of the interview and you ‘just feel’ that the candidate didn’t do too well. This is a surefire sign that you’re resorting to intuition, not data.

Horn effect

The horn effect is where you have - for whatever reason - an initially negative impression of a candidate, and then you can’t move past this despite them then demonstrating lots of positive things. For example, they might arrive 5 minutes late for the interview, and you’re a stickler for tardiness. They might go on to then - objectively - ace the interview, but you just can’t see it.

With this initial negative impression, the rest of your evaluation is compromised. You won’t be able to notice all the positive things about a candidate’s interview performance, because of the initial negative impression. First impressions last!

Halo effect

The halo effect is basically the opposite of the horn effect, when you fixate on one particular positive/good aspect about someone, which then clouds your judgement about everything else.

For example, you might start by asking the candidate to tell you about themselves. They answer with a really powerful and emotional story explaining how they got to where they are today. It really resonates with you and you think it’s a great answer.

You then interview them and - objectively - they do not perform well. However, because of the initially strong impression they made, you will probably find ways to justify their mediocre performance. You might use phrases like ‘Well they seem like a laugh’ or they’re a ‘Good cultural fit’. These are subjective vagaries that can’t be challenged. The issue is, your initial impression of the candidate has clouded your judgement. This is the halo effect in practice.

This is why in Alooba Interview, each question has its own weighting and scoring rubric. The candidate’s overall performance is then the sum of how well they did in each question.

Similarity attraction bias

The similarity attraction bias means that you are attracted to people that look like you or in some other way similar to you and this makes you feel comfortable with them. For example, you might interview someone who is a similar age to you, grew up in the same city and might have similar interests to you. You build an affinity with them and you ‘just feel’ like they’re a strong candidate. It’s important to realize, again, that the fact you feel similar to them says absolutely nothing about whether or not they’re the best person for the role.

By the way, this is why you have to be very careful with ‘cultural fit’ interviews where you just look for someone like you. This is a great example of the similarity attraction bias in play.

Illusory correlation

The illusory correlation means that you think there is some connection between two things, when actually there is not. That is to say, the correlation (relationship) is illusory (does not exist).

This is quite common in traditional interviews, where interviewers often have preconceived ideas about how an interview correlates with actual on-the-job performance.

For example, interviewers interpret a lack of confidence from the candidate as a lack of competence. If anything, the correlation might be negative, not positive. That is, a lack of confidence is more likely to indicate competence, not incompetence. This is basically the Dunning-Kruger effect, whereby the more you know, the more you realize you don’t know.

We actually measure the Dunning Kruger effect directly on Alooba. Candidates give themselves a rating and we compare this to their actual performance to determine their self awareness. Long story short - most people are very overconfident!

Beauty bias

The beauty bias, we hope, is a really easily understandable bias, and should drive home how real all these hiring biases are.

The beauty bias means that we assume attractive people are also somehow more competent and more suitable candidates than less attractive people. Of course, this is not necessarily true. Someone’s attractiveness really doesn’t dictate how well they analyze data, write SQL or create visualizations. This is why with Alooba you can measure these skills directly, to avoid these biases.

Because interviewers almost by definition can’t be anonymized, you need to be really careful to avoid this bias.

Confirmation bias

Confirmation bias is when you make a decision about something, and then ignore all future evidence against your decision. It’s almost like, it’s easier to be wrong, than it is to admit you’re wrong and change your decision.

This is quite common when interviewing candidates. For example, you might hear them talk about their current role and think ‘Hmm, they’ve been working with some outdated tools. They probably won’t be able to adapt to our stack.’

Even if the rest of the interview objectively goes really well, and they demonstrate skills & experiences in all the areas that you’re evaluating, you’ll ignore all that because you already made your mind up. This is the confirmation bias in practice.

Why do structured interviews help to reduce bias?

Structured interviews help to reduce bias in several ways:

With a consistent structure, set of questions & scoring rubric, each candidate can be measured in an apples for apples way.
By predefining what you’re looking for in a candidate, you can avoid moving the goalposts.
By controlling for other variables, any differences in candidate performance are really down to the candidate, not interview factors.

Are structured interviews too rigid?

A common bit of pushback that people make is that they perceive structured interviews as being overly rigid, which they feel stifles their ability to build rapport and truly understand the candidate.

This perception seems to stem from poorly conducted structured interviews, rather than structured interviews themselves. When we dig deeper, normally people with bad structured interview experience found that it was more of a box ticking exercise, with the data going into a black hole of nothingness.

They’ve also sometimes been forced to ask candidates’ arbitrary questions that are of no value whatsoever. Yes, that does sound annoying!

With Alooba Interview, creating a clear interview plan actually helps you to keep on track and treat every candidate equally. You can choose to ask candidates anything you like, just with a little forethought. This small bit of planning before you start interviewing ensures you can treat all candidates equally, ultimately hiring the best person for the role.

With a centralized place to collect all interview notes and question scoring, sharing this feedback with colleagues or candidates is easy. No more candidate ghosting!

What are the problems with traditional interviews?

The way interviews are traditionally done is littered with issues. The end result of these problems are that you’ll end up wasting time, money and ultimately fail to hire the best person for the job. There are some very common mistakes that interviewers make.

Here’s the top 10 most common mistakes we see interviewers making, and how to avoid them:

No clear plan or structure

We find some interviewers who conduct traditional interviews are often quite ad hoc and unstructured. At the extreme, they’ll rock up to the interview and just have a casual, meandering chat with the candidate. While this is definitely easy and feels nice, especially if it’s very informal, it will not help you with your ultimate goal: hiring the best person for the role. This is because without a common structure, you’ll have an apples-for-oranges comparison of candidates.

With Alooba Interview, you can create your interview plan by deciding how you’re going to structure your interview, which questions you’re going to ask and how important each question is.

Relying on gut feel

Relying on gut feel and intuition in making hiring decisions is a really common problem. As we’ve seen, there’s two parts of the human brain. We need to make sure we rely on the ‘slow’, rational part of the brain to make hiring decisions, not the ‘fast’, gut feel intuitive part of the brain.

The reason for this is that the fast part of the brain relies on heuristics and is prone to making errors (biases).

With Alooba Interview, you evaluate a candidate quantitatively, by grading them for each question that you ask them. The best performing candidate is the candidate who scores the highest.

No scoring rubric

Without predefining what answer you expect for a question, you’re left to wing it when you’re evaluating candidates. Was their answer good, bad, ok? This needs to be defined ahead of time, to avoid moving the goalposts and having an apples-for-oranges comparison of candidates.

With Alooba Interview, each question has clearing marking guidelines and you can award a score based on how the candidate has done. This will help you avoid confirmation bias.

Not adjusting for biases

While recognition of biases is now commonplace, it’s one thing to recognize bias, and another to actually change your behaviour to correct for them. Changing behaviours are hard for many reasons. The 2018 bestseller Atomic Habits by James Clear gives a great run down of how good and bad habits happen, and how to change them.

With Alooba Interview, we help you set up an interview structure that helps to adjust for your inherent biases. By running a structured interview, asking candidates the same questions with the same scoring criteria, we help you overcome your own biases.

Not taking notes

We see some interviewers don’t like to take notes during an interview. It is tricky to listen carefully, talk and write notes all of the same time. But the issue with this approach is that you'll get to the end of the interview and you will have no evidence one way or another for evaluating your candidate. You’ll be likely to fallback to your intuition of how you felt they did, and just give a pass or fail. As we’ve seen, this is biased.

This will be exacerbated if you have back-to-back interviews without any reflection time afterwards. By the end of the day, you may have forgotten who the first candidate was or any of the meaningful details of the interview. Without the supporting notes to help you, you're clearly not in a position to compare that candidate against others.

With Alooba Interview, you can set up an interview structure with a question order. As you interview, you can grade candidates for each question and provide notes to support your grading decision.

No feedback for candidates

For some odd reason, providing candidates with no feedback is completely normal in hiring. Well, this has to change, obviously. When digging in to why this happens, it’s normally for a few reasons:

providing feedback is too difficult
there may be some company rules against providing feedback

The difficulty in providing feedback is often because it hasn’t been collected and recorded well in the first place. In Alooba Interview, with a single place to record notes and score candidates, sharing this feedback with your colleagues or the candidate is easy.

If there are company rules against providing feedback, we’d encourage you to level up your hiring game and take the pledge towards ethical hiring.

Overvaluing interviews

In general, we find that interviews are vastly overvalued in how useful they are in identifying the best candidate for the role. Actually, interviews are one of the weakest predictors of on-the-job-performance.

This issue is exacerbated by organizations using a ‘last click’ attribution method in hiring. That is, if the candidate bombs the last interview, it doesn’t matter how well they did in the all the previous stages. All the weighting is on the final stage. This, by the way, is another bias - recency bias.. With interviews already being a weak predictor, there is no point in reducing your sample size from 3 to 1.

With Alooba, you run a data driven hiring process, which includes screening, assessments and interviews.

Asking irrelevant questions

In the same way interviews in general are overvalued, interviewers tend to ask certain types of questions that are quite irrelevant. For example, questions like brain teasers or ‘If you were an animal, which animal would you be?’ are obviously absurd. The key to a good question is that there needs to be a clear, objective way to differentiate between a good and bad answer. Unfortunately, ‘If you were an animal, which animal would you be?’ is totally subjective and has no right (or wrong) answer.

With Alooba Interview, you can leverage our question bank created and reviewed by experts. Flesh this out with your own organization-specific questions too.

Overconfidence

We see lots of interviewers overconfident in their ability to judge candidates. This normally sounds like ‘I rely on my gut’ and ‘They just aren’t a good fit’. These interviewers tend to shy away from a fairer, more objective approach. Again, it’s important to realize this is just a type of bias.

With Alooba Interview, the structured interview where all candidates are asked the same questions, with a common scoring rubric, helps to minimize interviewer overconfidence and reliance on intuition.

Poor time management

Finally, another common problem we see is a lack of time management from the interviewers. Often this hits the candidate worst, as their question time at the end gets squeezed down and rushed. This is especially the case when there are back-to-back interviews scheduled.

With Alooba Interview, you have built in reflection time after the interview. This gives you the opportunity to finalize your notes and decision making, all while it’s fresh in your mind.

Summary

Traditional interviews have had their day, it’s fair to say. Data-driven interviews are just one crucial step to moving away from gut feel, intuition based hiring and instead adopting objective, data driven hiring.

As you have seen in this article, traditional interviews suffer from five fundamental problems for organizations when hiring: Expensive, Inefficient, Inaccurate, Unfair & Opaque.