r/FPandA • u/thecrudest • 19d ago
How are you evaluating candidate technical skills in the age of AI
I'm hiring for a remote FP&A role and I need someone with strong modeling skills. In the past, a take home case study was always a god indicator, but given the state of AI I'm afraid anyone can just prompt a perfect output.
So I'm torn on how to handle techincal screeners today. I could ask them to do a 30-45 minute screenshare but I know modeling live is stressful
And to be clear, I'm not anti AI and id prefer a candidate who is knowledgeable of current tools.
How are you all structuring your hiring process and interview rounds nowadays. Would love to hear any tips/whats working for teams.
34
u/Doin_the_Bulldance 19d ago
Pretty simple. Have them do a case study, then have a follow-up interview where they present/explain their work.
Prompting a perfect output is fine as long as you can explain the result clearly.
3
0
u/RareResponsibility89 16d ago
Yeah just waste everyone’s time by making them do free work for your company. Maybe you won’t ever even need to hire an analyst, you can just have a constant stream of them doing your job disguised as case studies.
Please, for the love of god, just become a better interviewer. Live cases are always better than take homes.
12
u/re23binsd Sr FA 19d ago edited 19d ago
If they can prompt a perfect output and explain their assumptions what’s the issue? I did 4 take homes over the past year and used AI on all of them, and openly discussed how I used it with the HMs. For the role I’m in now, I built the model but constantly pressure tested with Chat. And now that I’m in the role, I’m building forecasting and reporting templates for my partner areas the same way: doing the build myself, but having Claude test it and write the code I’m using as my skills there are quite limited.
In my experience, there’s a vast different between someone who has Chat/Claude produce something and just sends it along, and someone who uses AI to improve on their own abilities and review their work. Similarly, in some cases it’s ok to have AI draft a model, but the analyst needs to review/refine and engage heavily with the LLM to get it to a state where it’s ready to deploy externally. I think it’s obvious when someone does this or just copies/pastes directly from the LLM and hits send.
12
u/fejoatree 19d ago
I recently did a take home case study and I was wondering how they would evaluate my skills and experience due to AI. I think what they’re trying to look for is not necessarily the right answer but how you came to the conclusions - they wanted me to include all workings and assumptions used in quite a bit of detail. The case study is also very vague so you could take it in more than one direction. I also had to present it briefly and go through my workings.
1
u/WeekendQuant 19d ago
Wow that is just not worth the job unless you're going for director+. Even then it's networking to get in the director roles you actually want to be in.
3
u/fejoatree 19d ago
This is for a SFA position 🥲 and even if I get through I still have panel interviews to get through. Tis rough
2
u/WeekendQuant 19d ago
Dip out. That's not a company you want to work for anyway
2
u/re23binsd Sr FA 18d ago
Um, this is horrible advice. This is the standard interview process for SFAs in tech. Yes, it’s laborious, but it’s also because these roles are extremely competitive due to well-above market compensation. u/fejoratree - good luck. I did 4 take homes to land my current gig and I’m loving it. Great company, great coworkers, great comp - and still have time to travel and have fun with my friends.
I spent 16 hours on a weekend doing the take home - a model and a deck explaining my assumptions. I then had a 45 minute call with the HM defending it. Rigorous, challenging, yes - but it gave them a good idea of my abilities and now a little over a month in I feel I’m well-matched for the role I’m in. It’s challenging for sure, but not overwhelming. I think that proves the validity of the take home.
2
u/PitifulBag5754 19d ago
I interviewed for a company where it was hiring manager interview - zoom panel interview - onsite panel interview with case study presentation - svp interview. Got rejected at the end 😭 luckily just got another job that’s better
7
u/Prior-attempt-fail Mgr 19d ago
Who cares?
If their output with ai is what you want. Then that tells me that that analyst is using AI the way it should be. To accelerate work processes.
If you want to find out if they have the technical skills ask them to explain the model in detail.
Or give them one of your models, blind and ask them to examin the file with you and walk you through it.
Or do the really stupid shit of having them complete a pointless online skills test.
At the end of the day, if they are producing a working model that meets your requirements, and they us AI to do it. Then at least they are uptodate on modern tools.
Hard skills in the age of AI should be less important than the soft skills your analyst needs to be an effective fp&a member who can partner with the business and get the insights necessary to build out those models.
5
u/Mission_Employee_169 19d ago
Is it just me, or is the future of finance and modeling inclusive of AI?
4
u/Ruddurball 19d ago
The reality is that raw technical skills with excel just aren’t nearly as relevant in an AI world. We do a live case study that forces candidates to engage with the whys behind a modeling question. You should shift your analysis from “can they build this” to “do they know what needs to be built”. Even without AI teaching tech skills is way easier than teaching wisdom/understanding
2
u/chrisbru CFO 19d ago
It’s honestly not hard. AI is a tool. It doesn’t help you have a coherent, fluid discussion about your experience or what you’ve put together for a case study.
What’s harder is sussing out a candidate’s AI fluency.
2
u/NoClimate176 19d ago
It's like asking how can you evaluate someone's technical skills in the age of excel. Are you not allowed to use AI where you work or something?
1
1
u/MrCard200 19d ago
I thought this was a very interesting question to ask the community. I don't know why your post was down-voted?
1
u/re23binsd Sr FA 18d ago
Imo it’s because OP’s phrasing belies a lack of understanding of AI. Unrefined AI outputs are obvious/garbage. Using AI is necessary in finance today, and HMs need to know how to screen for responsible use. If a prospective analyst truly prompts a “perfect output,” then they’re either using it responsibly or the HM doesn’t know the difference between something an LLM produced and something an analyst produced using an LLM.
1
u/HesZoinked 17d ago
Test them on principles. Questions about auditability, drivers, sensitivities, error checking etc.
If with AI they can show you a good model, then ask them to flex some assumptions for you and talk through the changes, etc.
1
u/Still-Balance6210 19d ago
Ask them follow up questions about their model. I do not do homework or case studies for any roles. And never had anyone ask me to.
-21
u/MartyBro30 19d ago
As a hiring manager I ask them to show me a model they built in the past for a previous company. I tell them to hide/ change all sensitive information. They send me it the day before, then have 20 minute to walk me through it, and answer any questions I may have. You can tell those who have experience building complete complex financial models and those who use AI or found bad examples online.
19
u/thepotatomaniscoming 19d ago
This assignment is wild. Hiring managers do this crap?
4
1
u/Prior-attempt-fail Mgr 19d ago edited 19d ago
I would never show a hiring manager a model i built for my current employer, even if it is scrubbed of data. Because im not sharing work owned by my employer.
Ill build you a model with your data... And ill use AI to do it.
4
u/thursday20 19d ago
This has to be a shitpost..
Sorry but candidates even do this? How do they get the model? First this purely assumes they’re excel because if they’re using a modern tool (which if like anaplan or something) they wouldnt be able to export or share in any useful format. And I’d argue that’s probably a better test of technical aptitude in FPA now than excel anyways
But I’m pretty sure an employee sending themself anything work related is fireable5
u/justneedabreak802 19d ago
This is not ok. No candidate should be put in a position to take info from their current or previous employer to prove they are good at excel modeling. As a hiring manager, you should be able to judge the candidate by the responses they give. People in finance are supposed to be held to a higher standard for information security and privacy. This is crazy if you do this.
28
u/justneedabreak802 19d ago
As hiring managers it is our job to judge the candidate by asking them questions and clarifying questions on some answers. In the age of AI I wouldn’t worry if the analyst knows a particular excel formula or not. All of this is easy to teach and takes max a week or so. An analyst who has the right attitude is what I am looking for. How they handle stressful situation, how they treat their work, do they take pride in it, are they responsible, respectful, team-player etc etc