The 2024 Ultimate Guide to Skills Testing

Absolutely everything you need to know in 2024 about assessing candidates skills when hiring

This guide is aimed at hiring managers, talent acquisition, recruiters and anyone else who's interested in implementing hiring skills assessments in their organisation.

The adoption of skills-based hiring has exploded in recent years, as companies finally begin to overthrow the shackles of traditional hiring approaches.

In this guide you will learn lots, including:

• What skills tests are, the different types, how they compare to other types of tests
• When, why and how skills tests are used in hiring
• The benefits of skills tests and what happens when you don’t use them
• AI’s impact on skills testing
• How candidates approach skills testing
• How skills tests help promote diversity and avoid bias
• How skills tests compare to other evaluation methods
• The cost of skills tests if you create your own or use a dedicated product
• And a tonne more!

OK let’s go, everything you need to know about skills tests for 2024.

What are skills tests?

Skills tests, also known as skills assessments or skills quizzes, are a way to objectively evaluate someone’s skills & knowledge in a given area.

For example, if you wanted to hire a Data Analyst, a Software Engineer or a Sales Development Rep, each of those roles will have a different set of expected skills, either required skills or nice-to-have skills.

For a Sales Development Rep, they’ll need to have some strong verbal communication skills, great written communication skills, strong data literacy and be good at presentations.

These skills will typically involve a combination of ‘soft’ skills and ‘hard’ (technical) skills. Either way, a skills test will then evaluate the candidates in these 4 areas, measuring how strong they are in each of these skills.

The result will normally come out as a simple percentage, making it easy to interpret and compare candidates. This objective, data-driven approach is one of the key reasons for the take-off of skills-based hiring over the last few years.

Note, when used internally on existing team members, skills tests tend to be called - for whatever reason - capabilities assessments.

Which types of skills do skills tests evaluate?

Skills tests can be used to evaluate any skill, be it technical skills (hard skills) or soft skills.

In practice, the skills tested depend on the role’s requirements, that’s why it’s absolutely essential to think ahead of time about what the requirements are for the position - this then informs all of the hiring process, including the coverage of the skills test.

When it comes to soft vs hard skills, it should be said, soft skills are inherently more difficult to measure, because they tend to be highly subjective.

For example, measuring someone’s verbal communication skills is actually tricky. Are you measuring how well they explain technical concepts to non-technical stakeholders? Or perhaps how succinct they are? Or how easy they were to understand? Or how charismatic they were when you spoke with them? Depending on who you ask, what ‘good’ communication is will vary significantly.

Because of this subjectivity, it makes the results of soft skills assessments less predictive of job-performance than technical skills assessments, which are inherently more objective.

Technical or hard skills tend to involve things that are factual, and either are right or wrong. This makes them easier to measure and also more predictive of job-performance than soft skills tests.

When in the hiring process are skills tests used?

Skills tests can be used at any stage of the hiring process including as a pre screening tool, screening tool, at the interview stage and as a final validation.

The main difference is normally how long and complex the skills tests will be. Pre-screening skills tests will typically be 20-40 minutes and only cover off the most fundamental basics, whereas when used further down the hiring funnel, a more in-depth skills assessment will cover a wider breadth of skills in more detail, and might be a few hours of effort for the candidate.

The best place to use skills tests is as a pre-screening tool, instead of relying on manual CV screening, which opens up your hiring process to significant bias & discrimination. It also saves a lot of time, expedites the hiring process and prevents you from missing out on the best candidates.

What’s the science behind skills tests?

The science for skills tests is crystal clear. They are an indisputably better predictor of job-performance than other selection methods like interviews, reference checks, personality tests or CVs.

The most comprehensive study of selection methods for predicting job performance was by Hunter & Schmidt. They published their substantial study in The Psychological Bulletin in 1998.

They performed a meta analysis of 85 years of research into what predicts job performance. They investigated 19 different selection methods for predicting job performance, including various types of interviews, personality tests, work sample tests, IQ tests (general mental ability), reference checks, peer reviews and more.

To cut a very long story short, they found that skills tests are the best predictors of future job performance. What they refer to in the paper as ‘work sample tests’ and ‘job knowledge tests’ are ranked 1st and 3rd most valid, respectively.

The 2nd most predictive method is what they called general mental abilities (also known as IQ tests or cognitive abilities tests) tests. A special note is that they work best for junior candidates who have not yet developed skills, like graduates and interns.

What’s the difference between skills tests and other types of assessments?

Skills tests evaluate a candidate’s current skills and knowledge.

The other main types of tests are:

• General mental abilities tests including inductive reasoning, numerical reasoning and verbal reasoning. These are sometimes called IQ tests, cognitive abilities tests or simply psychometric tests, and basically test how smart someone is.
• Personality tests, with the gold standard being the Big 5 Personality Model, which is sometimes called the OCEAN model. This model breaks down someone's personality into 5 factors, with Conscientiousness being a strong predictor of job performance.

What other methods are there for selecting candidates?

Other than tests, there are other ways to select candidates, including: referrals, peer ratings, experience, education background, unstructured interviews, structured interviews, job trials and reference checks. Typically of course a variety of methods are used when deciding who to hire, rather than just one.

It’s worth emphasising that some of these methods are completely invalid - they do not predict job performance, at all. For example, age, interests, years of education, and years of experience, in particular, do not predict job performance, as shown in Hunter & Schmidt’s comprehensive analysis.

What are the benefits of using skills tests?

Using skills tests to select candidates has a number of benefits.

Hire the right person

When you’re hiring someone, what you’re really getting is access to their skills. So without measuring that, you’re really stabbing in the dark. Anyone can put anything on a CV and can bluff their way through an interview. Hiring someone without testing their skills would be like a basketball team playing some dude who turned up and said ‘Sure I’m pretty good’. Not going to be a great outcome. The cost of a bad hire is huge, and can you guess the leading cause of a bad hire? The candidate didn’t have the right skills. That’s a true hiring own-goal!

Hire fairly

Using skills tests helps to make the hiring process much fairer. Candidates can be objectively evaluated based on their measured skills, and this helps to minimise potential bias. It also makes it really simple to compare candidates - just look at their scores. Other methods like referrals make hiring a bit of an old boy’s club, and CV screening leads to significant discrimination.

Hire legally

In most jurisdictions, it’s a requirement that hiring - at least in principle - be merit-based. There are sometimes carve outs in laws to allow for overt acts of ‘positive discrimination’ to rebalance some existing inequity, but this is normally tightly controlled. So, make sure to set up your hiring process to be objective and merit-based, to stay on the right side of the law.

Get hidden gems

By giving every candidate a fair chance with an objective screening skills test, you will identify candidates you were previously incorrectly rejecting. Minimising this hiring error is crucial, especially during periods where there are talent shortages. CV screening tends to favour candidates who have packed their CVs with buzzwords, inflate their abilities, and this approach is open to significant discrimination.

Take for example, the recent story of Nimash Paranapalliyage, who was forced to change his name on his applications in order to get any call-backs. This is an utter disgrace, and should not be happening in this day and age.

What are the most common problems implementing skills tests?

When moving to a more skills-based hiring approach, like with any change, you should expect some pushback from anyone who’s change averse. It’s important to push through this initial resistance and adopt skills-based hiring properly. The most common myths you’re likely to hear are things like ‘Will candidates do this?’, ‘But aren’t tests biased?’, ‘I can just judge someone by chatting with them’. We cover these off below.

What happens when we don’t test our candidates’ skills?

Not testing candidates’ skills is like flying blind. You can do it, but it’s certainly not recommended. By failing to properly evaluate your candidates skills you’ll be open to mis-hires, unfairness, illegal hiring and missing out on some of the best candidates who have applied to your roles.

Why have modern companies progressed to skills-based hiring?

The skills-based hiring movement of the last 5-10 years has come about from 4 big trends:

• Push to fairness and diversity: there’s been a substantial push for organisations and society in general to be fairer and more equitable, highlighting the need to avoid the discrimination of more traditional screening methods like CV screening.
• Acceptance of biases: people are increasingly aware of their substantial biases that negatively impact their decision making.
• Push to be data-driven: data is the new oil, and making data-driven decisions makes companies more successful, and this thinking has (slowly!) filtered into talent & hiring teams.
• Technology developments: until recently, any skills tests were manually created by individual teams and hiring managers, requiring manual development grading, making them very expensive and unscalable. Now there’s dedicated technologies like Alooba, providing you with this testing infrastructure.
• Changing labour force: More and more high-skilled knowledge roles exist in modern economies, so there’s a clear link between the demand for the underlying skills and a need to evaluate them, especially for fast-paced domains like data & AI, which have traditionally not had any formal certification.

What has been AI’s impact on skills tests?

Artificial Intelligence is such an incredibly fast-changing field, it’s quite hard to commit to what impact it has had or will have on hiring in general, or skills tests specifically. On one side, there are concerns over candidates’ ability to cheat, and on the other side it opens up the opportunity for companies to evaluate candidates in more clever ways.

Watch this space!

How do skills assessments help to reduce bias & promote diversity?

When used instead of CVs as a screening or pre-screening tool, skills assessments solve the systemic discrimination that CV screening allows. By moving to a skills based hiring model, you can help to ensure your hiring process is merit-based, and that the appalling discrimination cannot happen any more.

How does skills testing compare to interviewing?

Skills tests and interviews are both selection methods for deciding who to hire. They are not mutually exclusive. The comprehensive analysis of Hunter & Schmidt from 85 years of studies into predicting job performance shows that skills tests are more predictive than interviews.

Within interviews, structured interviews were found to be much more predictive than unstructured interviews.

Skills tests are an easy way to evaluate lots of things in a short period of time, making them an efficient way to understand the candidate.

Skills tests also allow you to evaluate each candidate in an apples-for-apples way, that interviews just don’t.

Finally, interviews are heavily biased towards tall, good looking and confident people.

How do you administer skills tests in-person vs online?

In-person tests are a lot more challenging to administer. You’ll need an office space, a computer for the candidate and obviously then also need to do everything in person.

Online skills tests tend to be easier to administer and can be done asynchronously. With advanced cheating protection and online proctoring, you don’t lose anything by having candidates complete the tests online.

How should we decide on building our own vs buying a dedicated product?

That’s a great discussion - here’s a comprehensive analysis of that decision.

The main reason to build use your own test is if a) you already have one set-up b) you rarely hire c) you’re sure the take-home project hasn’t been leaked online c) you work in a large bureaucracy where you simply don’t have any agency over which software you use and you have no budget and d) you or your team are experts and can evaluate the candidates’ skills properly.

Otherwise, we’d generally recommend using a dedicated skills testing product instead of hacking out your own.

What do candidates think about skills testing?

On Alooba, we’ve assessed more than 50k candidates and surveyed their opinion on testing. It’s probably one of the most extensive datasets in the world about how candidates feel about skills tests.

Candidates generally agree that skills tests are a fair and objective way of hiring. They tend to be sick of the same BS as everyone else, and they can see how it’s better than having your CV rejected because they’re Asian, and other nonsense like that.

Candidates feel most positive about skills tests when:

• They can see the relevance of the test, so make sure the test aligns to the role requirements and the job ad.
• It’s an appropriate level of difficulty for the role.
• It’s a fair level of effort, given where it’s used in the hiring process, so avoid very long screening tests.

Candidates are also dramatically more likely to complete tests when:

• They already have detailed information on the role, the team, the company etc. so make sure your run a transparent hiring process
• Make sure your role is actually attractive, that your salary in a market-based salary and that you’ve pitched the role properly

Do candidates cheat on skills tests?

Any system will have some level of expected fraud or cheating, and recruitment is no different. A relatively small percentage of candidates will try to cheat, either by gaining some kind of unfair advantage, getting someone else to take the test on their behalf, or getting help from some AI.

This is one of the most important reasons why creating & managing your own skills tests will be challenging - without any cheating protection measures at all, you’ll be opening your hiring process up to abuse.

In order to protect the integrity of the measures, Alooba does not publish details of its advanced cheating prevention system. Learn more about these from your Alooba account manager.

Are skills tests biased?

Skills tests help to create a fair, objective and merit-based hiring process. For example, by replacing manual CV screening with a skills-based screening approach, you do your bit to prevent the shocking discrimination that happens with traditional screening.

How much do skills tests cost?

It really depends on whether you create your own skills tests or use an off-the-shelf product.

Here’s a comprehensive discussion about creating your own skills tests, and here are some cost calculators to give you a realistic picture of the investment needed.

Are there any free examples of skills tests?

You’ll find some free example practice skills tests here, such as this free SQL test, this free data analysis test and this free product analyst test.