Most Common Problems with Data Science Hiring Tests

Skills-based hiring has become more and more common over the last few years, with platforms like Alooba at the bleeding edge of this development. While there’s a lot of benefits to validating your candidates’ skills using skills-based hiring, implementing it in practice does pose some potential challenges.

Not sure what skills-based hiring is? Check out our comprehensive skills based hiring guide here.

A lack of hiring maturity

These challenges typically come when companies have a desire to use skills-based hiring, but simply don’t have the testing infrastructure set-up yet to validate candidates’ skills at scale. It’s like they want to cross the Atlantic, but they’ve got a rickety old boat and can’t swim very well.

In analytics, data science, software engineering and other technical fields especially, it’s very common for each hiring manager to develop their own case study or take-home assignment for candidates.

These take-home tests might be a coding assessment, creating a dashboard, producing a piece of analysis, building a model or putting together a presentation. They typically involve real or near-real problems of the business and are typically a close reflection of the work that will need to be done, if the candidate lands the real job.

Don’t just rely on interviews

The insights about the candidate’s skills that these assignments generate are essential to understanding the candidate’s ability to do the role, and give the candidate a glimpse into what might be expected of them. When done right, take-home assignments are a great way to get a realistic picture of the candidate’s ability with something pretty close to the real work.

When hiring processes rely purely on interviews without any skills testing component, it becomes incredibly risky because you’re making a hiring decision based purely on how someone comes across. You’re judging them on how they talk, not how they walk. It’s a test of confidence more so than competence. It’s no surprise really that interviews are a very weak predictor of job-performance, and companies so often make the wrong hiring decision.

So, skills tests are essential to prevent hiring errors, however, there are several key challenges from both the company and candidate’s perspective in having these take-home assessments in place. These challenges currently limit how effective organisations are in implementing skills-based hiring themselves.

1. Managing your own skills tests is expensive

The opportunity cost of creating your own take-home test will be very high. For analytics as an example, it will involve developing the datasets, creating a basic data dictionary, creating the questions, creating answers, creating marking guidelines etc. It’s a minimum 20 hour job to complete this.

Then when it comes to actually grading the candidate’ responses it’s normally 30 mins at least per assignment to just grade it, and a lot more if you’re providing feedback.

You can calculate the cost of creating and managing your own take-home tests here.

2. Expect the time to hire to blow out

The tests normally take a candidate 5-15 hours to complete, and this is spread over 1 to 2 weeks. This directly extends the time-to-hire. Then, there’s the time to grade and provide feedback. This then normally adds another few days and starts to create a bottleneck.

3. An apples to oranges comparison

Because the take-home tests are untimed, each candidate can spend any amount of time to complete the assignment. This then makes the results hard to compare. If candidate A got 70% and candidate B got 80%, you might say candidate B has better skills. But what if it turned out candidate A spent 2 hours on it and candidate B spent 15 hours? Candidate A seems a lot more efficient than B.

4. Cheating can be rife

It is basically impossible to prevent cheating as it’s a totally uncontrolled non-proctored environment. This is another good reason for always offering a follow-up interview afterwards, which would allow you to validate if the candidate had completed the assignment themselves or not.

5. Questions get leaked on the internet

Unless you are consistently rotating your assignment questions, if you work at a well-known company, they may well have been leaked on the internet. Try checking Reddit, Quora, YouTube & Stack Overflow.

6. Not everyone gets the same chance

The effort for candidates is especially high. Because they’re untimed, there is no upper limit on how long a candidate can commit to the assignment. This then leads to accessibility issues, where there could be a bias against the candidates who simply cannot commit the required effort, for example single parents.

So what’s the takeaway here? Skills-based hiring is a big step forward in hiring and relying on interviews alone as a selection tool is a recipe for disaster, but without the proper skills-testing infrastructure available, organizations typically struggle to scale hiring successfully.

Are you currently deciding on whether or not to create your own skills test? Check out this guide guide.