Episode 68

Piermarco Burrafato on The Impact of AI on Job Applications and Effective Screening

TIM: pierre Marco. Welcome to the objective hiring show. Thank you so much for joining us.

PIERMARCO: Thanks, Tim. Thanks for having me. It's a pleasure.

TIM: It is absolutely our pleasure. And I'd love to start today's chat with probably the most talked about thing in the history of the world. You could argue AI, the hype of hype things. Maybe it deserves the hype. Maybe it doesn't, I don't know. We'll have a chat about it. And where I'd love to start is a discussion about AI in the hiring context. And so I'd love to get your thoughts on have you started to dabble with AI? Have you seen candidates use AI and what are your thoughts so far?

PIERMARCO: So I haven't, myself, necessarily used AI actively in the hiring process. Definitely I see candidates. Using it because, you spot similarities in CVs and cover letters. And I guess that's a sign that those tools have been used. I do not mind if candidates use AI at certain points in the hiring process. In other words, If they use it to make a CV or to make a cover letter as a first draft, but then they put their own personality into it and change it to make it reflect their personality, it's fine. Also, when it comes to more technical interviews should they make use of AI for getting a quick answer, polishing code, all that Why not? More and more people are using it for that purpose. So I think it's just another tool to be added to the bag of tools. So I guess I don't mind that much, but obviously I want originality and genuinity when it comes to. showing up to the interview, starting with the CV and the cover letter. I really want to see the personality reflected on those.

TIM: Yeah, and it's a common bit of feedback we're getting at the moment is the sense that Yeah, not only are hiring managers being inundated with a lot of CVs But as you say that they're looking increasingly similar to each other which Is probably not in the candidates favor. If the whole point is to try to stand out in the process, try to rise above the hundreds of other competitors you have using the same LLM as everyone else to optimize your CV for the same job description, probably is not going to do them a service. And and as you say, looking for that kind of authenticity as well. I hear that a lot. And so it's about, using these tools in the right way that helps you, as opposed to as a kind of blunt instrument, here you go, ChatGPT write me the CV and apply for this job. I feel like that's not going to get candidates very far at all. Is it.

PIERMARCO: Yes, that's right. Those are very similar to those get rich quick schemes. Don't put much effort into it. Just go with the flow, but that doesn't really work in real life.

TIM: And have you seen candidates or experienced candidates where you, where they might've been using it during the interview itself as some kind of interview partner?

PIERMARCO: It is difficult to spot, in nowadays world, we do the interviews remotely, mostly, and therefore it's really hard to spot, I would say. And also So we have a process that goes, that starts with a manager interview. So you just speak to the candidate and they speak back to you. So using cameras you don't, I don't think anyone has ever used ChatGPT to respond to my questions, quite frankly. It might happen during the technical interview where we leave the candidate alone for 30 minutes, essentially. But again, then they come back and they need to go through the responses to questions. And it's, you easily spot if someone has used chat GPT because they didn't know what to say. And then when they have to go through the results the solutions to the exercises, they don't know necessarily how to comment the results because the results were given by chat GPT. So in that case, I think it would be very easy to spot, but it hasn't happened to me yet, quite frankly.

TIM: Yeah. And I feel personally, if I. Were a candidate at the moment, I think it would be a lot more effort than it's worth to try to use Chachapiti in an interview itself. That would cause me, frankly, a lot of stress because I'd be thinking, not only am I probably cheating in the eyes of the interviewer, doing something they probably don't want, but I'm going to have to sit there and, read the interviewer, have a discussion, look at ChatGPT look at an output, put it back into my words Oh my God, I don't know What could be more cognitively difficult than that? As if an interview isn't stressful enough, like I may as well just give the interview a crack, rather than trying to leverage some other tool. At the same time, that's what I think anyway. But maybe if you're a candidate with no knowledge or skills, and you're really trying to stretch it and get a job that really on paper, you shouldn't have any chance to get. Maybe for them, they feel like it's worth it because they couldn't pass interview themselves. I don't know.

PIERMARCO: I cannot picture it in my head that someone would say, would act as if they're not typing on the keyboard to ask Chachi PT while they're speaking to me, that would come across as not very natural. So I cannot, I really do not expect people behaving that way, even in those very specific situations you're mentioning. I don't know, is it your experience as it ever happened in your case?

TIM: it hasn't. That said, I haven't interviewed anyone myself in the last six months when the tools have really gotten to a point where you think it would actually be helpful, not a hindrance. And the voice to text, I think is a big one as well, because without that, there's still a quite a big missing link. You wouldn't, as you say, you wouldn't really be able to, type away on another screen and go, hang on just wait a second. Wait it's thinking, I'm thinking, it will be yeah, it'll be a bit of a giveaway, but yeah, exactly. But I think once like the voice to text now is pretty good. So in theory, you could have the audio of your computer coming out to then an input mic into chat or something, but you still have to sit there and read its output and then put it into real human words. Oh my God, that would be so complex. That it's not worth it.

PIERMARCO: I feel you almost have to be a fraudster to be able to think about that and put it in place.

TIM: So yeah we were talking here, Marco, about candidates, use of chatty BT, and like some other guests I've spoken to, I've had slightly different philosophies. Some have said AI is the now and the future. every candidate, once they're in their jobs, should be using this technology to make themselves more efficient, to do things better. And they would be reluctant to then have anything in a hiring process that it somehow implicitly discourages that at all. And so on the most extreme end, some people have said to me, you know what, they should go for it, have at it. We know they're going to use it anyway, use as much as they want. Also, to your point, it's a tool like anything else. Use it smartly where it actually helps don't use it right. Detracts, but otherwise some people have said just do it like to create your cv to do a test to do whatever part of their argument was also that If the company and the candidate are upfront about this from the get go and they say, please, you should use AI tools if you want to, then you can almost get a layer deep with them in the conversation. So instead of pretending that they might or might not be using it, then you can start to scrutinize how they use it. Here you go. Here's my prompts that I wrote. Here's how I attack the problem. Here's how I, here's the exact model that I used. And then, Oh, cool. Did you think of using this other bottle? Did you think of changing this parameter? And then maybe they can get like a level deeper in their use of AI, which is maybe something that isn't currently being evaluated, I don't think, in most hiring processes, like how candidates use AI is just glossed over. What do you think of that perspective?

PIERMARCO: I think when it comes to the interview process, we don't go that deep and we don't encourage, we don't like, we don't ask questions around what prompt engineering you used. We give it for granted that's just the one tool in their bag of tools that they might be using during the interview process, but most definitely the example you, you gave. you brought up before of someone talking to me and then turning to AI to get the answer to my question that never happened up to this moment in time. And if it happened, it would be so weird, I think.

TIM: It would be strange. I personally haven't experienced that either from candidates in a live interview. One thing I definitely have seen, and I know a lot of people have spoken about, is CVs that have been either completely written with Chatshippee Tea or at least optimized with Chatshippee Tea. And then also that there's now tools that would allow a candidate to apply en masse to different roles. So two slightly different things. One is the CV that's been written by GPT and then the mass applications, which seems to then be causing this massive spike in applicants per role. Is that something you've seen for your role so far? And if so, any thoughts on how we can combat that?

PIERMARCO: Yes. So this is a very great question. I think it touches on two aspects. One is I've certainly seen people using generative AI for the cover letter. So here's the thing I noticed in the spike of applications that you are referring to, I receive hundreds. of applications for a role in a matter of a few hours. But what I noticed is that the vast majority of these applications do not have a cover letter. And I think, I have an opinion about it. I think it's because of the easy apply button by LinkedIn. It is so much so that in more recent job ads, I put a big text at the top saying applications without a cover letter will not be considered because I think it's too easy to mass supply and it's not fair. The work of a hiring manager is tough. Okay. So especially in my case, I'm a director. But I'm also line manager. That means I do not have managers in my team, which I would love to have, but I don't. So effectively I'm a line manager hiring individual contributors. That in itself is a tough job, like being a manager, like being a people manager, coaching spending a lot of time through one on ones team meetings also playing the role of the leader of the team and speaking to senior leaders when it's necessary and all that. Plus you have to do the hiring. I think it's not fair to bombard a hiring manager with all those irrelevant applications. I'm not sure that AI is the driver for that. I think it's LinkedIn, quite frankly. What I've seen on the AI side is that it's used for cover letters. Why? Because I've just tried it myself. I asked ChatGPT to produce a cover letter for a specific role. And it always comes back with the same language, more or less. And that's the same language I noticed in this cover letters. This cover letters are not genuine, right? They're not authentic. And they are easily spotted therefore. So that's where, I hope I'm making sense, but it, there is definitely an uptake of of applications. Most of them is, are noise. And I would say, unfortunately, not so much driven by generative AI, but driven by LinkedIn. And those that understand that they have to provide a cover letter, perhaps they don't want to make the effort to write down a cover letter, and they use generative AI in most cases, unfortunately.

TIM: Yeah, I suspect you're right. And I think for candidates, I do feel a lot of empathy for them because they're caught in this weird scenario where they're probably looking at the number of applicants of roles on LinkedIn and going, Oh my God, it's 500 already. Are you kidding me? And so in their head, maybe the last time they went for a job, they probably thought I'm going to have to apply to, I don't know, 50 jobs to get three interviews to get one offer or something. Yeah. Now they're probably thinking, Oh, that's 10 X that. So my odds of getting any particular role I apply for have been reduced by 90%. Therefore I'm going to have to do 10 times as many applications, which then is probably creating some kind of vicious cycle where every other candidate then does that. And then you as a hiring manager is getting inundated with crap. As you say, most of them, most of the applications are effectively spam. I know this as well. Anyone who's, Been in the frontline reviewing applications knows that 90 percent of them are completely irrelevant. People from another country, not in the same profession, like just total random stuff. And that's, I guess the nature of the open jobs boards, but it's not helping anyone because the candidates who are relevant to now lost in that sea of noise you're in a day with all these CVS. How do we get out of this? Like almost a bad equilibrium and economists might call this.

PIERMARCO: Yeah, good question. How did we get there? out of this. I guess trying to pressurize LinkedIn not to have easy apply. I understand that perhaps increases engagement on their platform, but if this trend continues in that fashion, quite possibly we're not going to use LinkedIn anymore for job ads moving forward. It's really too much to handle. And you want people to make a little bit of an effort to be more selective. And in what they apply to do a little bit of homework and really understanding, okay I'm really relevant for this job or I like this job very much. I should prioritize the list of jobs I'm applying to. Also to increase his chances, if you're just bombarding left and right, I'm not so sure it's the greatest strategy quite frankly. I don't think the best candidates I've ever recruited have used that tactic of applying to, Tens or hundreds of jobs. I think they've been most all of them very selective, very committed to the organization. Like they did their homework, they did their research, and they understood that there was a good match, a big match between them and the organization and, the culture of the organization as well. And I would say the mission, the vision of the organization and all that. Yeah, how do we get out of that? It's a big question. I don't know if I haven't asked her, but I guess so we see this trend in LinkedIn where the algorithm gets updated frequently as much as the Google algorithm for search engine optimization. Now the LinkedIn algorithm gets updated also quite often to, boost some influencers or some content creators and damage others. Who knows what's behind that, but clearly I think on the side of LinkedIn, their main driver is increasing engagement with the platform. So we see that on the content creation side. I think we see that on the job seeker side as well, by providing all these tools to make it easy to apply. And I think as much as this is hitting at some point LinkedIn on the content creation side, because I see people complaining about it I'm not sure if hiring managers are voicing our, ex voicing their discomfort so much as well on, on the side of job applications. That much I haven't, at least I'm not aware of, groups or for forums where you can voice your discomfort. As a hiring manager, but I would think if other hiring manager are in the same position as me, they would be not so happy about LinkedIn at this moment in time.

TIM: Yeah, it's interesting. If I think back over the past four or five years, because the hiring market has changed so radically several times during that period. So we've got this, let's say, slightly more employer sided market. At the moment, there's more candidates than there are jobs available. Three years ago, it was in, let's say, after one year of COVID, it was the exact opposite. It was like, Oh, my God, there's no candidates. Low interest rates, every company was hiring huge funding rounds. And I feel like these kinds of easy apply options, I can't remember exactly, but I feel like they would have come to prominence then because the narrative would have been, Oh, we're going to get your job in front of as many candidates as possible. We're going to reduce friction. We can make it as easy as possible for candidates to apply to your jobs. And companies were like, great, because we just need to have the most streamlined process possible. There's only three candidates available quick. We have to get them, but now the market's the opposite. So now you as you just said, you want people who have taken a little bit of effort and it's very interesting that switch in thought process because of the difference in supply and demand. We might get to the other market again in a year, who knows what happens. It would be interesting to see, but. Yeah if I think of all the hiring managers I've spoken to very consistently would be unhappy with the quantity and quality of applications. They're getting too many applications, most of which are irrelevant through jobs boards like LinkedIn for sure. So yeah, it's definitely a big problem. I think maybe part of the thing is also that candidates have the main voice on LinkedIn because there's a thousand times as many candidates as there are hiring managers. So that probably just gets drowned out. And also and also LinkedIn probably deals more with talent leaders than they would with hiring managers, even though the hiring managers are the ones who actually end up making the decision. So I think there's this weird little gap in their feedback loop probably as well.

PIERMARCO: Oh, yeah. You're touching on another very good point. I think by talent leader, you mean like a talent recruiter, a recruiter. Yes. So I see what you mean. So most definitely, I think you're pointing the finger against that first step in the process, which is the talent recruiter doing a good job at screening all these applications, which is not always the case. As

TIM: no it, indeed. And I'd love to get your thoughts on this because I feel like maybe. Part of the challenge is our expectations over talent HR recruiter teams is maybe unrealistic. Maybe we shouldn't expect people without any experience or skills in the roles they're hiring for to be effective screeners. I try to put themselves in my shoes. If I was trying to hire a lawyer, I would have no idea what I'm doing. No idea. I just be. shooting darts in the dark. Like it would be almost impossible for me to effectively screen lawyers. So maybe we just expect too much. What are your thoughts on this step?

PIERMARCO: I would be like you a hundred percent. So yeah, it's a fair point to say shouldn't expect them to be able to hire for any role. And within large organizations, you typically have to hire for all kinds of roles, which I think poses the question, shouldn't talent recruiters perhaps be specialized in a given discipline, be specialized in hiring for specific roles, instead of being generalists? I think. Not having a specialization in this case hits pretty hard. Clearly, they have a hard time sifting through the mass of applications they receive. I've had to have conversations with talent recruiters around What I was getting from them, like I'm getting data engineers instead of data analysts, sometimes which is fine. Sometimes some someone might Want to apply for a data analyst job having a data engineering background. It's also okay But it's clear that in some cases they are on a given path, which is data engineering. They're not really looking for data analysis and vice versa. So yeah I guess the best talent recruiters sit down with you, go through the job requirements, try to understand them as much as possible. But again, I think they hit a limitation. Even if they do that, which is great, it's very proactive of them to do these are really the best talent recruiters, right? But even if they do then at some point Not having that specialization and that specific job role and perhaps having to deal with multiple Job roles at the same time across a range of disciplines makes things more complicated for them. So that's fair enough. I don't blame them too much for that.

TIM: I feel like another big systemic problem we have is that the CV itself is such a weak data set to be using to decide who to interview. And I feel like it's probably going to be even worse now if these CVs have been either optimized or written completely with ChatGPT I can only assume they'll be less representative of reality. than they were before, which was already pretty bad in my experience. So is it just that like the data we have to use is not good enough? Do we need some kind of new data set? What do you think?

PIERMARCO: So in my experience, I just use the CV as a business card. I try to get the personality of the person through the way they structure and fill with content their CV. So it is just that more often than not, I go beyond the CV and go and check their LinkedIn profile. Most candidates, I must admit, they put it straight at the top, like this is my LinkedIn profile. So I just click, go there and check. With some other candidates, they don't do that. And that's where I'm, most intrigued to go and search for the candidate in LinkedIn. Why do I do that? Because I think, like you said, people are using so many tools to write up their CVs, not just generative AI, but they're using services, third party services, people that review their CVs and all that. But I think the LinkedIn profile is more genuine, more authentic. And you can get their personality through that. For instance, if you see a LinkedIn profile with no picture, that's almost a no for me. And why is that? If you don't want to show your picture, perhaps you shouldn't have applied to the job in the first place. They might have all the reasons of the world. And that's fair enough. People might not want to share that picture. That's okay. But then that obviously closes some doors if you want to work and get a job. And I've seen profiles with no pictures with as little content as, the title of the jobs that can also be fine, just the titles of the job, but not so much in the bio, not so much information. So again, speaking of making that effort. If you haven't made the effort to use that nowadays business card, because that's what it is. It's a business card of nowadays. So if you haven't made the effort to complete it, at least with some information, then it's a natural selection process in that case.

TIM: Yeah. And if you were, I think a reasonable candidate competing with hundreds of others, not having a LinkedIn profile seems like a own goal. Maybe the devil's advocate you could say would be Depending on your privacy settings in LinkedIn, that dictates what someone else can see. So they might have it just very restrictive for non connections. Whereas if you're connected with them, maybe they show you everything. But yeah, if they genuinely haven't completed the profile, but yeah, that seems like a pretty basic one. I also would find myself doing that kind of comparison you mentioned, which is CV versus LinkedIn. I also have the same perception as you that LinkedIn is probably more real because it's public. I feel like you're less likely to lie publicly. What I notice is actually Sometimes people will inflate the titles on their CV and have something more realistic on LinkedIn. Oh, something more, more realistic and more junior. Do you see that as well?

PIERMARCO: No, that's true. Yes. Yes, absolutely. I see a lot of bombastic CVs. More recently, really bombastic. Like they have achieved everything. So why are you applying for this job? If you have achieved everything,

TIM: I wonder if that's again, the Chachapiti impact, cause it has this very waffly language that always uses overly complicated words that sound like no human ever uses, except maybe an academic, I don't know.

PIERMARCO: You may be right. Yeah. Yeah. That's true. You may be right on that. Yes. True.

TIM: in this. Yes, in this CV screening step, what I'm assuming is about to happen is that companies are going to start using AI to combat the AI. So they're going to start using some kind of AI screening step because who can read a thousand CVs? No one. So that whole process seems to have broken down. But I don't feel like that's actually going to solve the core problem because you're still just going to screen a CV. Okay. Yeah, you could automate it, which is better. You could do it in a more objective way. That's definitely better, but it's still going to be, I think, such a weak signal over who's worth interviewing such that poor people like yourself will still end up interviewing lots of people who probably you shouldn't interview because they just don't have the skills needed. So yeah, I was like, I was just thinking about what's going to happen next. I feel like we need some extra data set. That's more authoritative than a CV, like maybe we'll get to a point where we can start aggregating candidates, GitHub and their LinkedIn and a conference they went to and some YouTube videos they're on we need something else about the candidate what ideal world, what would you know about the candidate at that point? Maybe that's a better way of putting it what would you want to know, such that you could make a good screening decision? Do you think about that?

PIERMARCO: Yeah, you definitely see that trend, right? Those that really want to rise to the top are starting to curate the press, personal branding a little more by adding all those portfolio on github. com. I prefer perhaps in some other place, not necessarily GitHub. GitHub, it's quite technical at the end of the day. You're showing off code without really telling any story. And what you really want to say, what you really want to tell as an analyst and show off, is your ability to tell stories. To communicate data driven insights and recommendations that speak to the problem that you solved, that you helped solving. And that's, I think that's totally missing in GitHub and, in those repositories, those kinds of repositories. I prefer like your own website perhaps, or. A different kind of website where you show off, you write down some articles to communicate how you went about solving the problem and show some charts, some, a little bit of data storytelling, going to the recommendations that you would provide. But people, you see a minority of people doing that. Which I think is great. And also you see people linking, conferences, like you mentioned videos that they have made. That happening more and more, but I would say this is still a very small minority still to this day. I think there is a saying that only 1 percent of the people on LinkedIn are content creators, the 99 percent are just readers of the content. And so that obviously applies to candidates as well. But I like what you're saying that ideally you could have tools capturing those signals and therefore giving you a broader perspective on a candidate and really screening the best through those signals. I think that would be great to have at some point. I think If you think about that as a tool you could put out there in the market. Perhaps your tool already has that. I don't know. But then I guess the question is what is the supply to me? I think the supply at this point in terms of curating personal, your personal brand and, do all that still a very small minority. So I don't think it's. I don't know how much you can get out of a tool like that since the supply is not that high.

TIM: Yeah, I think you're absolutely right. It's, there's just not enough candidates doing it at the moment to justify something like that, because if you get 500 applying and you said, okay, let's rank their GitHub profile, their sub stack and their LinkedIn presence. There might not be any that have all three out of the 500. I suspect that's going to be the problem. Even if now we're getting better at dealing with these unstructured things, like LLMs are quite good at taking unstructured stuff and making it structured, so maybe in the interim you'd still need some kind of test, a skills test, a psychometric test, an IQ test, something that could be given to every candidate rather than trying to capture these things from other places. One other idea we were playing with was there's this weird double pain point at the Candidates, when they apply to jobs, have to go through the same shit again and again. It's so predictable what questions you're going to be asked as a candidate. Every company in the world would love to know your communication skills, and your, I don't know, SQL skills, and your storytelling with data, if you're applying for an analyst role. Every company in the whole world wants to know those three things. So it seems pointless that you would apply for a hundred jobs and have to go through this same process with every company feels like Some point there should be some kind of intermediary That is validating all these skills those things for candidates and then just sharing the data with the companies there's got to be some kind of unlock there I think in the next five years But we shall see.

PIERMARCO: I like that. I like that. It's I think that's a great idea. It's Coming up with employer agnostic tests that you can be ranked over. That's a great idea. I think I like it. I like it very much. And I might just add, yes, you're right. Everybody would want to check your communication skills your scripting skills, SQL, Python, what have you et cetera. But I guess when it comes to understanding problem solving skills, that's where I think it might get complicated, right? Really trying to understand how people reason about a problem, how they come up with solutions, how they break down a problem into smaller pieces and all that. Yeah, I wish there was something like that in place. That's a great idea in itself. I'm just saying Just the problem solving parts, which is why we have the second interview, which is a technical interview But we call it problem solving because we want to understand how people go about problem solving so going about checking for problem solving skills it's It's something that requires human intervention still, I think, to this day, but I don't know, you tell me.

TIM: Yeah, I think so Yeah, maybe this intermediary service that doesn't exist yet could also be doing the technical interviewing. I guess part of it would require companies letting go and saying I trust you that you've figured all this out. Therefore we're going to massively reduce our own hiring process. Otherwise then there's no point. And so we're going to, we're going to do just one interview and hire a candidate as opposed to this three or four stage, very elaborate process that most companies would go through. So the only way this would work, I think, is if that intermediate authority was like, really they really could do this very well, better than what a company could do. Which seems tough. I guess it's also those subjective things as well, isn't it? Because it's what is good communication skills? You and I would probably have different definitions, because it's inherently gray. So I guess, Come into some sticky points like that as well, I suspect.

PIERMARCO: fair enough, yeah.

TIM: But we shall see. Maybe someone will start that business. Who knows? Pierre Marco, I've one final question for you today. And that is if you had the proverbial magic wand and you could change the hiring process somehow to make it fairer and more effective, what would you do?

PIERMARCO: I would change the most painful part of the process, which I think is that initial screening and getting a good pool of candidates. A constant pool of good candidates, I would say. Really getting the best. You don't have to get a lot, but you have to get the best. Yeah. And for me, that's really the most painful part of the process. So if I had the magic wand, I would definitely find a way to fix that and have the best recruiter that can sift through all the applications. I expect that for a while we will keep on getting this mass of applications. So that's not going away. So that's where I would be focusing on a hundred percent.

TIM: Yeah, I think that's a great place to wave our magic wand at. And. If that problem could be solved, I feel like the rest of the funnel then suddenly becomes a lot simpler. If you're only interviewing candidates, so you're like 95 percent sure are pretty good. Everything else is going to get easier after that. So yeah, I'm with you on that one. I would love that one as well.

PIERMARCO: Very good.

TIM: PIERMARCO, it's been a great chat today. Interesting conversation. We've covered off a lot of ground and thank you so much for sharing all of your thoughts and insights with our audience.

PIERMARCO: It was a pleasure for me, Tim. Thanks for having me and wish you best of luck with everything.