Episode 29

James Laidler on The Role of AI and the Importance of Communication Skills in the Hiring Process

Tim: James welcome to the Aluba Objective Hiring Podcast

James: Thanks, Tim. Great to be here.

Tim: Uh, yeah, likewise, I am delighted to be here, and I'd love to kick off by asking about everyone's favorite topic right now that everyone is talking about, AI, and, uh, in particular, I'd love to know your thoughts about AI in the hiring context. Have you started to dabble at all with AI in hiring, especially in potentially the screening stage of the hiring process? Love to get your thoughts there.

James: Yeah, so it's interesting because, um, I think for the screening process, it is potentially useful, at least in its current form with that AI. Um, I think with the company I'm working at, previously, uh, currently, I should say, um, we did use LinkedIn's kind of job posting, and I'm not sure it's really AI, but it's kind of—they try and match skills. and that you're after skills on people's LinkedIn profiles. So I think that did help somewhat to at least get it down from my hundreds to, you know, maybe 50 or something like that, but, um, I think in terms of, like, a more general sort of question, I do think it is potentially useful for that screening phase. Um, I don't think you have to be careful with, like, potential um biases in terms of the AI in terms of how it's been trained and the data that's been used to train it, um, but I would say I'm a little bit more, um, at least in the later stages of the hiring process, a bit more old school, I guess you could say, in terms of, um, I like to set some sort of take-home project for the candidates to go away and, you know, work on for, say, like, a week and then bring it back. and I think the key bit is the, uh, presentation part of it that, um, you know, you have a group ideally of, like, both non-technical and technical people in the sort of interview panel, um, and getting the candidates to, um, you know, show what they've done for that piece of work, um, in that sort of context because I think communication is, like, really often overlooked, especially in data roles. Um, and I'm, you know, um, as I'm progressing through my career, I'm sort of realizing how important that is, especially if you're in a company, um, where, you know, it does depend from company to company because you might be presenting generally to very technical people, and so maybe there it's not as much of an issue, but when you've got, like, non-technical people that you're presenting to, I think communication is, like, on par with the technical capabilities of a candidate.

Tim: Yeah, and it's always tricky to find someone who has both, um, for sure, uh, both those kinds of soft skills and those technical skills. So you mentioned you give the candidates a take-home project, and they come and present it, um, what are your thoughts on candidates then potentially using ChatGPT as part of that, that, uh, process to do the take-home assessment? Do you have a view on whether or not they should or should not use ChatGPT or tools like that?

James: Yeah, I mean, I would. It's an interesting question, I think. Um, you know, I personally don't see an issue with people using it as like an aid to their work and sort of as a soundboard as well to kind of get some ideas. Um, I would say at least currently, from what I've seen with like ChatGPT and the other chat models, is that they do tend to struggle a little bit with like data and analytics. so I don't think it would be a case of, um, at least with sort of the take on projects that I'm thinking of, where you, you know, give them maybe a data set that your company actually uses, and maybe it's anonymized, but it's, you know, it's a real-world data set, and you essentially give it to them with a very broad kind of um, a question of like, you know, analyze it and um, I don't know, pick out some key takeaways or some key actions that we can take. I think that is very hard to just feed into a model like ChatGPT and say, like, give me the answer, so um, I mean, 10 years from now it might change completely and, you know, you might be able to do that, um, but I think currently, um, you could probably see if somebody's tried to do that, um, pretty quickly because I'd imagine that the output wouldn't be wouldn't be that, um, brilliant, but in terms of, you know, um, Using it to come up with some ideas to then look at something, I personally don't see an issue with doing that.

Tim: And I think what's really important in your process is they're still going to have to come into that interview anyway, so if they have kind of outsourced the thinking to ChatGPT, they're going to come unstuck immediately once they get into that. panel interview, and you're digging down, and you're really saying, like, why did you do this? What was your thought process here? And they're like, Oh, I don't know, so I wonder if that means if candidates are not silly, they probably realize they can't just get their friend to do the assessment or get ChatGPT to do the entire thing because they know they'll inevitably fail. I assume that's part of their process or thought process.

James: Yeah, no, that's a very good point because I think, you know, if you set up a project where it was just kind of almost a test where the candidate could just give their answer and it was either right or wrong, then, yeah, that would probably be more open to abuse from ChatGPT, but I think, yeah, if you have to present back, somebody's going to ask a question about the approach that you've made, and, um, and then, yeah, they're probably going to really struggle. Um, to answer that question,

Tim: What about in the interview itself? Like, are you doing these remotely or in person? Uh, and if remotely, would you be against a candidate using ChatGPT in the interview itself? What are your thoughts there?

James: Um, yeah, so I mean, it's, um, the company I work at, Electrify, we're a fully remote company, um, so it would be, yeah, all the interviews I've done for this company have been, uh, remote. I would say that, um, you know, I guess, yeah, I would have an issue if they, if they're sort of, they have ChatGPT open in another window and they're trying to quickly kind of answer questions. I I think probably it'd be quite difficult, uh, to do, but I don't know with the voice capabilities of ChatGPT now and stuff. They could, you know, I guess I could see a scenario where they've got it, you know. Hooked up so that ChatGPT can hear the conversation and hear the questions and then, um, get results back. So, um, I guess, yeah, in an ideal world, you would probably do that interview in person, um, but I think, yeah, with a remote company, it's obviously, um, there's some difficulties with doing that, but I think for most candidates, I don't know, and it would be difficult to pull that off with having ChatGPT running in the background.

Tim: Yeah, it's probably the sort of thing that kind of sounds interesting and cool and clickbaity. I use ChatGPT to pass an interview, but the impracticality of listening to a conversation and watching a large language model write all this text out and then answer in a natural way like a real human sounds almost impossible to me rather than just answering the question, you know what I mean?

James: Yeah, yeah, and I do think, um, you know, I think most companies who are remote do this, but having your, um, video on during the interview, I guess, I would probably make that, um, a necessity for it because I think you could quickly then see if somebody's, you know, looking on another page and then sort of reading and answering the question using chat as well. so

Tim: Do you ever have candidates turn up without their video on? I'm interested in that, actually.

James: Um, yeah, I can't remember any. Um, to be honest, I think they've all come with video on, and I think there might've been one or two, but we always ask them to share it as well because I think that, you know, like, I think maybe in normal situations there might be reasons to not have your video on because you are, like, traveling somewhere or whatever, but for an interview, I think it's kind of, you should be expected to be on camera. Um, and in like a, uh, uh, quiet environment, because you would be expecting to come to an interview in person, um, in like a, uh, a normal company, I guess, for want of a better word, so yeah.

Tim: it's funny I was asking someone about interesting little red flags that they notice during the hiring process that aren't necessarily part of the evaluation criteria but just give like a Oh, this might not be the right candidate, so someone who interviewed recently said it was like single-finger typing for a technical person would be like, Oh my God, what is this person doing? The one there you just said is coming with a camera off, like expecting to have an interview with a camera off. To me, that would be quite odd from a communication perspective after four years of working remotely, by the way. Like, it's not like it's day one of COVID; we're used to this by now, so I would be quite concerned if a candidate came with that camera off, expecting that was the normal way to do it.

James: Yeah, definitely, because I think, like I say, you know, once you're in the company, there are occasions where you might have to join a meeting and you are out and about, or for whatever reason, you know, you don't have your camera on, which I think is fair, but yeah, for an interview, it's like, it's kind of like, um, turning up somewhat well-dressed as well is kind of that, you know, expectation that you want to, um, give off this air of, like, um, professionalism, um, for that. So yeah, I think that I'd agree with that.

Tim: and what I've heard a lot in the last few weeks is companies talking about Being inundated with applications so high volume of applicants for each role and then a feeling that quite a lot of the candidates have written the CVs or at least optimize the CVs with chat GPT is that something you've also noticed as well And if so, what do you think about that?

James: I haven't noticed, um, necessarily with, um, you know, specifically them using ChatGPT, um, to optimize. I would say that, um, for the roles that we hired, um, probably this time last year, um, for my team that we were inundated with CVs, I think there is, um, in terms of, like, the LinkedIn hiring, um, that we got or the candidates who were, um, uh, putting in their application on LinkedIn, I think we caught, like, 500 applicants or something like that, and, um, going through those and this was after the the filtering stage of LinkedIn as well um so you know they're on paper they were supposedly um fit for the role um and so yeah but then with that I found that a lot of the the CVs were very similar um especially in terms of like skills you know everybody says all the you know advanced Python they're advanced SQL um and you kind of have to take that with a pinch of salt because that they're somebody's um idea of advanced Python can vary quite a bit um in terms of reality So I think what I tended to look at was experience more than anything, both in terms of professional experience and where they've worked and the kinds of roles they've done. Also, what they actually put in terms of their experience, like the output of their work. So I think a lot of people just put, you know, I was a data scientist at whichever company, and I built these models, but I think the really good candidates showed Um, like, they created a model that, you know, reduced costs or something like that by 20% um, and actually, like, quantified it, because that gives you a good idea that, you know, this person is always thinking in those kinds of terms where they're trying to, like, quantify what they're doing rather than just giving an ambiguous kind of, um, answer to that. and I think uh just the final thing was as well I tended to look at like projects that they'd done and their own portfolio as well cause I think that um you know when you when you get that many candidates you you get a lot of people who are very similar you know they've they've got very similar experience they've got similar skills but then I think the people who really stand out then uh are the ones who've done projects either on their own Um, and just, you know, taking some time to even, even if it's a project that's done as part of like a course on Udemy or Coursera, um, but then others who've actually, like, gone and tried to do something new or tried to sort of do, um, something a bit outside of their normal kind of work, I think that just shows that they've got a little bit more, um, of a growth mindset and a bit more, um, Uh focus on kind of bettering themselves in the technical area as well

Tim: In your process of screening these applications and then interviewing candidates, I'm assuming you must have had examples of where someone looked amazing on paper but then just couldn't really back that up in the interview or later stages of the hiring process. Um, if so, could you kind of tell us about a few of those experiences?

James: Yeah, yeah, so I would say, um, the hiring for the roles that we did, um, we saw a lot of that where, you know, you get, um, you get people who are, like, you know, PhDs, um, who look really, really, really strong, um, you know, technically, um, and yeah, and look great, and, you know, you think they're a shoo-in; they're going to be, like, easy, just, like, straight in. Um, and then you talk to them, and even sometimes, like, we'd have, like, an initial chat just to sort of assess cultural fit, and even then they came across pretty strong as well, you know? They'd, um, they'd say all the right, like, technical things, and, um, you talk about their experience, and obviously they know a lot about what they've done. in terms of, uh, like their PhD, for example, and their work experience, but, um, what I found was, like, the real, um, the real, uh, point at which a lot of these, um, people who were strong on paper but then, uh, were challenged afterwards was those take-home projects, um, and I think that's why they're so key to do. um and even though you know the there is I guess a question that you're putting a lot of um uh pressure on them to and time as well that they have to go away and you know spend time doing these uh projects to potentially not get a job um but I think that they're just so important because a lot of people I would say probably the majority of people who came across really strong in terms of especially in terms of academic um Uh experience were then actually really struggled with like the the hands on stuff and I would say it's it's both like the in terms of applying the technical knowledge to like a real world scenario and then also the communication I would say as well but um you know they really they tended to do like some sort of very technical um Uh analysis or um apply a really like fancy algorithm and kind of forget about what they were trying to like solve in the first place Um, and so, yeah, I think I've definitely seen a lot of that.

Tim: and with the candidates who fit into that category would they typically be reasonably junior but as you say maybe with an advanced degree of some kind but not a lot of practical work experiences that are fair categorization or would it even be more senior people who, who got found out in that way?

James: It's an interesting question. I would say generally yes, um, the people who might have, like, a strong academic background but maybe not so much actual work experience probably more of those fell into that bucket. Um, I would say that there was, like, uh, I guess, like, a non-trivial number of people who were um I would say like who had some work experience but maybe not like very senior who still um struggled as well and so I do think it kind of depends where your work experiences has been as well because I you know in terms of my experience at least there's certain companies that I've worked out where communication wasn't that much of a of a um Uh a key skill or that maybe maybe that's too general a point but at least communicating to like non technical people wasn't a general skill because you were working in a team and all I should say you were working in a very technical team and all the sort of presentations and discussions you were having you Um well with very technical people Um, and so it wasn't, it didn't really like come out because I would say, um, for me personally, PayPal, which is like a company I've worked at previously, was one of those companies, or the team at least, in which I sat was one of those where there was less of that, um, sort of focus on, um, the non-technical communication and more on kind of technical expertise. That was like the big thing, whereas with electrification now, it's almost like the other way around. You know, you really need to focus on that non-technical aspect, so yeah, I would say it depended on the candidate's experience as to their communication.

Tim: You make a great point about the take-homes being really essential because, at the end of the day, you could do plenty of interviews, but the problem I find with interviews is that, um, it's still the candidate talking about stuff they've done. It's like experimental, experiential, sorry, behavioral; they're not really doing anything. Whereas you give them a task that is, by the sounds of it, closely correlated to what they'd actually do on the job if they do a great job in doing it and then can explain it in a live interview, that's surely like 95 percent of the way there in figuring out if they can actually do the job that you want them to do. And so I figured that's one of the reasons why they're so accurate. Um, have you ever come across someone who nailed the take-home test and then that final interview where they can really back it up but then still couldn't do the job? I assume that's fairly rare.

James: Yeah, I would say it is, but I do think it has happened, and I think it's more, I would say it's maybe more on, um, the responsibility lies more with me, I guess, than, um, than the candidate because, um, I would say that, you know, we've talked about it a lot. The communication aspect was maybe not as prioritized as it should have been. And so you know they came, they did the presentation, and they did a decent job, um, communicating, but it was more their technical expertise that kind of wowed us, and so we thought, You know, very strong technical candidate. Let's, um, let's go for it. And then afterwards it was, you know, you have that realization that by actually Maybe we should have focused a bit more on the communication because, yeah, the technical expertise is there, but the communication is still a bit lacking, so yeah, I would say it has happened, and it was probably more my responsibility for that than the candidates.

Tim: What about more broadly thinking about the candidates? You've interviewed those who didn't make it and those who ultimately got hired, so you've already mentioned some things that differentiate them, like those communication skills. I'd say almost the way you've described it as well as a little bit of empathy for who your audience is and who you're communicating to. Like, if you know you're speaking to someone from, I don't know, talent acquisition, don't go four levels deep into the background of your model that you built because that's not going to resonate with them. Uh, any other patterns that you saw between the successful and unsuccessful candidates?

James: Yeah, so I think that one that you mentioned is very important. It's like knowing your audience essentially, um, definitely, um, because, yeah, it's not necessarily just that you always have to communicate in a nontechnical way; you have to know who you're actually presenting to and kind of tailor it to that. Um, I would say one other thing is, and this is more, I guess, more difficult to tease out in the interview process, but it's like having a growth mindset, I would say, um, for a particular candidate. So I think, you know, if they can show that maybe they're not, I dunno, say, like, as technically knowledgeable as maybe they should be, but that they can somehow show that they're, I dunno, willing to learn or that they are in the process of learning how to do it and can somehow show that. I know that that's like, it's kind of difficult to do in an interview setting or through your CV, um, then I think that is, that's very important as well because I'd be more likely, I think, generally speaking, to hire somebody who can show that, you know, maybe they're not, um, as technically gifted as some of the candidates, but that they're really, um, passionate about what they're doing and, um, they really want to learn versus someone who's more technical, but they've kind of stuck. I don't know, like, um, they took their foot off the gas a little bit, and they're coasting, um, somewhat, and you know, like I said, I think it's more difficult to tease out, um, than those are the points, but I do think you can get, uh, a general feel of it when somebody's, you know, really into, I don't know, like, say, Python or coding or, you know, data analytics, and they really want to learn it versus somebody who's, you know, maybe just coasting and, um, not that bothered about it.

Tim: Yeah, that growth mindset is so important, I think, because, you know, if you've kind of given up on learning and you're just, as you say, coasting through, maybe you're not growing much. If you're growing at 1 percent a day from a lower rate, you're going to surpass that person pretty quickly, and I know which person I'd rather have in my team when it's six months down the track or 12 months down the track. That growth person is going to be absolutely on fire. Um, so it's so important. Uh, to I, capture

James: Yeah, and I think kind of linked to that, it's like if you then, you know, say you hire them and you have to give them a project or a task where, um, you know, nobody has, um, background in it, you know, I dunno, building some sort of tool where, you know, you need to learn, go away and learn about, um, a Python package or a certain, um, data product or something. Um, that's when you really struggle with the people who are kind of just coasting because they struggle with that, and then they'll come and ask you a bunch of questions, and you know it's more sort of on them to go and learn it rather than you to tell them how to do it, so I think, yeah, those people who have that growth mindset are very useful in those situations as well.

Tim: Uh, I'm remembering, sorry, now how we have hired our software engineers in the past, so this might be, um, an interesting piece, so we basically came to the conclusion, similar to you, that we really valued growth mindset, or what we call the ability and willingness to learn, because we thought, Well, to start up, things are changing all the time. This person's job could be completely different in two months; we might change that technology stack; we might move from AWS to GCP. We might start using a new programming language. We might change the product. Like, anything could change. It's like quite chaotic, and so we wanted people who would enjoy that and thrive in that and be interested in it. and in particular, uh, what we thought of was trying to say, is there a way in the hiring process we could get them to demonstrate that through their actions as opposed to just their words, which sometimes is all you have in interviews. What we came up with, which I think worked pretty well, was for our software engineers, our take-home assessment for them was an R project, like R, the statistical language, which No software engineers know normally because it's just stats, statisticians, and data geeks like us, uh, and so we gave it to them. It was a really simple algorithm to build; like, it wasn't a complex challenge, but they had to be willing to learn R, which itself was an interesting test. Like, would they tell us to go take a hike? Like, why are you asking them to write up I don't like writing up I'm a software engineer. I only write in Java. That would have been the first challenge for us to see whether they responded that way, and then could they actually then learn it in a reasonably quick period of time, a couple of days, to then make something that worked? didn't have to be the perfect R code; it didn't have to be production-ready; it was just solving this simple algorithm in a language they'd never seen before, um, and that worked really well because all the candidates fit into that bucket because none of them knew R, so we could at least evaluate them in a consistent way. and then yeah, it was quite clear who was actually engaged, who was interested, who wanted to come on this journey of learning new things all the time, even some things that are a little bit outside of the box of what you would normally think of doing, so I thought that was a kind of cool example.

James: No, that's a really, um, interesting way of doing it, actually. I've never thought of, like, setting a problem in, like, a completely different language from what they would use. Um, I mean, I would say it's, it's, there's some overlap, but not, um, not completely. I would say that in the past I have done set questions where it's more just like, uh, um, analytical problem-solving questions rather than, uh, you know, take-home tests, and that's sometimes done more in the interview stage rather than, uh, The take-home test is usually like after an initial interview, and I do find that at least it gives you an idea of how they approach a problem and how they think. so it's not really like you know the ability to learn and, um, you know, um, how they feel about learning as well, but it's sort of putting them outside their comfort zone a little bit and seeing how they, you know, do they just, you know, Take it and run with it, or do the kind of I'm an R and very unsure, and you know, so I think that's also a way to at least get an idea of that side of things.

Tim: and presumably the fact that these projects are unstructured, as you say, you're not giving them a list of 20 individual tasks to solve; you're giving them something bigger and broader, probably more representative of real work that must then allow candidates to really show off their true skills because they can take it anywhere they like in a way. James: Yes, yeah, yeah, and I think that that's where you also see people who've maybe got a bit more experience or think about it a little bit more versus someone who's done, like, a data analytics course and just, like, follows the steps that the course says, um, because I, yeah, I do think, you know, I've done them in the past as well, the courses where they're foundational knowledge, but Um, it's very specific. It's usually a very simple example with a very simple data set, and then they create some charts and some tables, and you know, they think Oh, that's great. That's data analytics, um, and then, yeah, you see that with some candidates where they essentially just replicate that process and produce the same charts, the same table, but there's not much actual insight or actions that you can take from it. So I think that's, yeah, a benefit as well to keeping it broad so you see how they approach a problem.

Tim: Yeah, and yeah, I again, I'm with you. I can't underestimate the value of the take-home project or some kind of project, uh, because the number of candidates that can appear very confident in an interview and can have seen competence even under, like, a reasonable level of scrutiny is, like, some pretty probing questions. It's still words, still talking about what happened and what you did at a company, which is impossible to verify until you actually see their work, their presentation, and some actual evidence of what they're doing. You would be amazed at the difference if the people listening who maybe don't have projects and they just roll on interviews. like the difference can be absolutely stark

James: Definitely, yeah, agreed.

Tim: Uh, okay, uh, what about thinking now in terms of candidates and feedback? So I'd say the leading complaint of candidates in a hiring process is getting ghosted, getting no feedback, and getting crap feedback. This has, like, got to be their top three complaints for sure. What do you feel like is a fair level of feedback that candidates should get at those different stages? Let's say from the application stage at the start to getting down to one of the final two people in a final round kind of stage.

James: yeah no very good question i I think you know being on the um the receiving end of that myself um quite often um I would say it very much does depend on the stage I would say because um you know if you say apply to a job Um on LinkedIn um and send your CV in personally I would say that you shouldn't really expect much in in terms of feedback from from that at that point um you know I still think you should get some sort of email saying you know thanks for your uh application but you're you haven't been chosen to um we haven't chosen you to to um for the next steps um because yeah even even that is at least good because then you know whether or not um it's proceeding or not um I would say then at the later stages maybe the say the interview stage where it's a bit more of an informal chat I think some bullet points about maybe like you know some pros and cons almost um to to um with regards to that that conversation I would say is is enough I do think that it is useful to have that feedback, and you know, um, usually at that point you'll be interviewing a relatively small number of people, so I don't think it's a huge um, it's a huge impact on yourself to write some feedback for those people, um, but then I think when it comes to say like the take on project stage, um, I would say that because they've put in so much effort um to to work on the project there should be some expectation that um the the interviewers give some feedback and give some you know detailed feedback um on that and that's what we've tried to do I think with with that stage because um you know it's like I said they've put in a lot of time and I think you should then take some time to to give them feedback But then I think that can really help the candidate as well in the future because, um, you know, up until that stage it's all very theoretical, but then at that point you're getting some real feedback from real people and real companies, um, and so that should, if you take that feedback on board, you should, um, if there are, if there are good companies, I guess, and if they're good, sort of rational people who've interviewed you, um, then you should get some good feedback that will really improve your Um, your performance is like going forward, so yeah, I think that's an important stage to give some feedback on.

Tim: Yes, it sounds like the feedback should be commensurate with the stage and the level of effort the candidate's gone to, basically, um, whereas you say for that project where they might have spent four or five or six hours, then it makes sense that they'd get more detailed feedback, maybe also because there's more data for you to base that feedback on. I guess you've got more evidence. That's feedback that can be more, that's a, I guess as well.

James: That's a very good point as well, yeah, because, you know, in um With a CV, it's kind of if you're sending in your CV, it's kind of obvious usually why you haven't made the cut because you know you don't have a specific skill or you shouldn't, um, you don't have enough experience or whatever, but yeah, with, um, with the take on a project, like you say, it's, you know, it can be a myriad of different things. And, um, so yeah, you should be a bit more specific about, um, why they haven't gotten through, um, but then also, like you said, there's more to talk about as well, I guess.

Tim: and when we've run our hiring process, we had the order of the steps that sounds pretty similar to yours, where we'd have, uh, yeah, the take-home test is like maybe a penultimate step before some final interview where we'd dig into it in a bit more detail. They'd run through it, and we'd kind of question, uh, their approach and those kinds of things. We sometimes had a scenario where the candidate had bombed out on the take-home. Like they've done it, they sent it to us, and it's like we know it's a hard no. And we had a conundrum of whether or not we should still give them that final interview, where we're more than willing to sit there and give feedback but under the expectation that they already know they're not getting the job. I'm wondering if you ever had that conundrum, and in that scenario, would you still interview them, or is that almost like putting them through torture to make them turn up, or maybe it's just down to them; like if they had the choice, it should be their choice. I don't know.

James: Yeah, I mean, I think if it's a definite no, um, from, like, say, that take-home, um, stage, then I would say it's beneficial both to the candidate and to yourselves to not do the interview, um, not do, like, a further set of interviews or testing or whatever it is that you do, because it's essentially just wasting your time. time for the candidate and wasting time for you, but I think if I don't know, I guess if there's a scenario where it's more of an interview to talk about the, um, what went right and what went wrong for the take-home test, then that might be, you know, you could offer the opportunity for them, um, to do that. But I think, yeah, if it's sort of that you're just following, I guess, the process to like tick a box almost and say, "Yes, we then also did an interview, um, because we said we should, um, yeah, I don't really think that that's a good use of time.

Tim: No, I agree completely. Uh, what about the nature of the feedback? Uh, like, I can think of, yeah, lots of hiring processes where we would have discussed a candidate's performance internally, and we had an interview with some interview notes. Uh, maybe we graded a test, and we had some feedback there, but we wouldn't necessarily share all of that with a candidate. Some of it was maybe internal sometimes, uh, maybe our feedback would have been too brutal maybe and has to be sort of massaged a little bit. Do you have any thoughts on, like, what type of feedback you should communicate with candidates, um, and how it should land and what you shouldn't tell them perhaps?

James: Yeah, yeah, I think it's an interesting question. I would say that I guess at the end of the day it should be like constructive, and that that is somewhat, um, subjective, I guess, but, um, you know, because there might be something that the candidates have done or said or, um, something like, for example, maybe the English is not their first language and that they've interviewed, um, and that they've struggled with some of the communication. I don't know. I mean, I think that it's maybe constructive to say that the communication wasn't, you know, received or, or, uh, well or, um, something like that, um, but just because, you know, it isn't their first language, I don't know if that's how constructive that really is, um, but I think, yeah, other points where they can actually take something away from the feedback and go, Right, I'm going to implement a change based on this. I think it should be more of a focus rather than just, um, general feedback because, like you say, yeah, sometimes there are points that, um, either come up for me personally that I've written down for candidates or somebody else has said that it's more just that, you know, maybe they get a bad, I guess, from the person or the person's potential, like a little bit arrogant or something like that. I guess you could argue that it's worth feeding that back to them and saying that, you know, you came across slightly arrogant, but then I don't know. I think it should be more focused on, you know, um, you didn't, uh, you didn't think about your audience, or, you know, this is an audience of non-technical people. You should have translated that technical knowledge to a non-technical set of metrics or something like that. I think that's a bit more constructive.

Tim: It's such a tricky problem I find, and I like to think of what the underlying reasons are why feedback often isn't given because that's the status quo. is, it's certainly the early stages of the hiring process. There's very little feedback, and then even later on, some companies, particularly in places like America, have a we don't give feedback blanket policy Sometimes it's for qualms around being sued and legal issues, and fair enough, uh, but other times I feel like there are other problems, like who wants to give negative feedback? Like, that's actually a minority of people who would enjoy giving constructive negative feedback, uh, when you're hiring, it's normally so frantic, and you're focused rightly on the candidates who are successful and trying to get them through the process and maybe trying to hire them. and I don't know about you, but as soon as a candidate drops out, it's like they almost start to disappear from my mind in a sense because I'm like, Well, no, there's two left, the two that have made it, like, they're gone now. I need to focus on the two survivors.

James: Yeah. Yeah.

Tim: And so there must be some kind of reluctance, I guess, just because of those reasons to give the feedback. and it must be easy for it to slip down the list of priorities, I would have thought.

James: yeah well I think it's only when I started being involved in the hiring process there I realized like how crazy it can be um because you know I was always the person who was um applying to positions and I'd get annoyed when You know they they wouldn't give feedback or and I still think that it's good etiquette to you know like I said um even just send a quick email saying sorry but you know you're not a good fit or or whatever but um but yeah it's I think it's it's really just that that you've said around it being um very busy and that you're just desperately more often than not at least desperately trying to fill this position and you've got you know hundreds of candidates and you're trying to you know interview whilst also doing your day to day job as well Um, and so I think that that's kind of probably the last thing that most people want to do is, after an interview, if the candidate's not been successful, is then spend half an hour writing detailed feedback to the candidate as to why they didn't make the cut, essentially. But like, you know, like I said, I think that it depends on the stage. The art, I think, if they've made it to that take on the project stage and they've put in hours of work, is sort of on you to at least give a little bit of feedback.

Tim: a hundred percent uh I wonder whether this is something that AI could really help with so I figure pretty soon at least at that screening stage I feel like it's a no brainer that a large language model could do that CV passing to do an initial evaluation of the CV matching it to the job criteria giving it some kind of score and ranking maybe not automatically accepting or rejecting that's going to break every law in the world at the moment but you know some kind of evaluation of the CV in a more objective way and then it would be very easy for that large language model to know why it's given what it's given, which would presumably be passed on to the company, could easily also be shared with the candidate, like we evaluated your CV as a 30 percent match; here's the main reason why it wasn't 80 percent, um, and that could be automated, and if it's automated, at least then that removes the whole issue of the human having to manually do all this tedious work. I wonder whether the interview might also work. I was speaking to a company recently that had hacked together their own CLAWED implementation using the Zoom transcripts of the interviews, and it was doing the grading of the candidate across the different questions. They're using that internally now; again, that could easily be shared via email with a candidate, maybe a sanitized version, but I feel like we're maybe close to breaking the back of this problem.

James: Yeah, I think, um, like you say, as they progress in terms of the large language models, I think that there's more opportunity to at least, like, leverage them in some capacity. I would say that, you know, I don't know what's going to happen in, say, like, 50 years time, um, but currently I would always like to have some sort of human in the loop, um, to just double-check, you know, that, uh, that, um, accurate and, um, yeah, that's summarizing kind of, um, as you would, um, but yeah, it's a very interesting point because I was initially thinking, you know, usually I'm writing notes. When I'm interviewing someone and, um, trying to sort of, um, take down key points when they've done something good or when they've done something not so good, I was thinking, you know, you could use a, uh, an AI to kind of summarize that and then, um, and, and get the result, but you, yeah, with the transcript, I guess as long as they know they'd have to, the LLM, I should say, would have to know kind of what the company's looking for, I guess. Um, but it could in theory, yeah, look at those transcripts and at least give you kind of an idea as to why they didn't do it, obviously. Yeah, like I say, you'd have to, um, have that human feedback as well, but yeah, it's an interesting, interesting, uh, idea.

Tim: Uh, okay, uh, one final question. James, um, is there anyone in data or in hiring and that kind of sphere that you've personally learned a lot from, from their approach or from how they've, um, perhaps inspired you to do things better?

James: Sure, um, yeah, I mean, there's a lot of people who jump to mind, to be honest, um, so I would say in terms of, like, um, technical learning, like technical, um, stuff, um, there was someone I worked with at PayPal called Charles Polly, um, and he, yeah, he essentially just to give, like, a quick kind of, um, uh, context. So I, um, started with R, the programming language, um, and I then joined PayPal, and everybody used Python essentially, and so I was kind of, uh, the odd one out, um, for using R still, and he, um, kind of spent a lot of time teaching me, um, not just sort of like Python in terms of the basics because I was kind of picking that up from, um, courses and stuff, but, like, the more, I'd say, um, advanced stuff like object-oriented programming with Python. Um, that sort of thing, and um, and we're still very good friends, um, now, even though, you know, I've left PayPal and everything, but I think he was, um, when I look back, I realize, like, how much, um, time he spent with me and helping, um, to, like, uh, to mentor essentially with the more technical side of things. Um, I'd also say, um, Orlando as well, uh, Orlando Mercado. I think his name is, that's how he pronounces his surname, um, at Electrify is, uh, he's essentially like a senior data advisor, um, who works with us, um, and I'd say he's a very good soundboard, um, just to discuss not necessarily just, uh, like the technical aspects of the role but also, um, the other parts and the more, like, strategic elements of, you know, creating a a roadmap, and you know what we should be prioritizing, and also some of the presentations I've done for the company. Again, it's, you know, it's essentially we work with a lot of YouTube creators and content creators, and so it's really key that, you know, you try to tailor what you're saying, and in terms of data and the technical side, you know, To that audience Um, and I think he's been very helpful with sort of, um, uh, highlighting some aspects that I could improve there as well because it's a, yeah, it's a constant kind of learning process. Um, I still sometimes, you know, tend to go a little bit too technical and have to, um, sort of, uh, go back and, uh, think about, you know, how I communicate that, but yeah, he's been very helpful as well.

Tim: And he's provided you some of that constructive feedback, the same constructive feedback that you would provide to candidates as well, so there you go.

James: Exactly, exactly.

Tim: wonderful Well, uh, James It's been great chatting to you today, and, uh, thanks so much for joining us.

James: No problem, thanks, Tim.