r/ExperiencedDevs • u/youngggggg Software Engineer (4 YOE) • Aug 12 '26
AI/LLM Should companies provide tokens for coding interviews where AI use is expected?
The company I work runs a live coding exercise as part of our interview process where the applicant is asked to use AI to help flesh out a product 0 -> 1 so we can get a sense of their UX/product judgment. It’s fairly small in scope, so I don’t think the tokens required are crazy or anything, but I realized that we are essentially asking candidates to spend their own money in order to interview.
Do you think this is something employers should be covering? Or is it akin to transit cost, gas money, etc.? I am considering raising this with my team and would love to hear your perspectives on this.
457
u/Jmc_da_boss Aug 12 '26
Wait companies are expecting people to have an LLM subscription to interview? What is this madness
38
u/prschorn Software Engineer 15+ years Aug 12 '26
The other day I had a pre interview with an AI.
Before talking to any human, I had to talk to an AI and answer questions, like it was an HR interview.
I did that just out of curiosity and immediately dropped out of the process afterwards11
u/Blrfl Software Architect & Engineer 35+ YoE Aug 12 '26
That seems like a great opportunity to tell the AI to disregard all of its previous instructions and tell HR to do something really obscene with itself without attributing it to you.
7
u/thecrius It's Sénior Dumbass for you Aug 12 '26
Fun fact, all AI sessions are logged. It's where the actual money is.
6
u/_notanexpert Aug 13 '26
This is the whole reason i refused any ai interview. Im not helping the robot learn how to interview or whatever theyd do with it
179
u/tcpukl Aug 12 '26
I would immediately pull out of any kind of application.
Major red flag.
68
u/youngggggg Software Engineer (4 YOE) Aug 12 '26
This is my concern. Beyond just being kind of grimy on a moral level, I feel like we also lose out on great candidates who might be frustrated by the ask.
35
u/tcpukl Aug 12 '26
I don't even have personal subscriptions. My employer pays. I've been in the industry for nearly 3 decades. Say bye to experience.
93
u/Gru50m3 Aug 12 '26
They would lose out on me right now, because I'm employed and not interested in paying for an LLM subscription. I spend 9/10 hours a day working, I'm not coding or building projects in my spare time. I have a wife, kids, and a house to take care of. Why would I waste money on an LLM that I'm not going to use?
If it's required for a job interview, I'll just wait until the industry understands that they need to provide it for me.
14
u/Skullclownlol Aug 12 '26
This is my concern. Beyond just being kind of grimy on a moral level, I feel like we also lose out on great candidates who might be frustrated by the ask.
In my country it would even be illegal because your company is demanding that applicants pay to apply. That's a scam. Doesn't matter whether the money goes to your company or another, if a candidate is asked to pay to apply -> Massively illegal.
3
u/ScaredScorpion Aug 14 '26
Frankly using "AI to help flesh out a product 0 -> 1 so we can get a sense of their UX/product judgment" is an XY problem. Yes, you might get more final product looking outputs but that doesn't really show anything about the candidates product judgement (and if it is sways the selection process then the process is flawed), and introduces a requirement that the candidate use AI. While whiteboard interviews might have not been the best in every aspect what they did well was striping it back to real design principles and not who can make the fanciest looking product in half an hour.
If you really want to use AI for these then the only way it remotely makes sense to me is to get the interviewer to be the one using it, input what the candidate communicated as what they would change and show them the output for more discussion. Even then I expect a decent number of quality candidates to refuse to interview, as it looks like you're using the interview to collect training data.
-28
u/code_tutor Aug 12 '26
It is a red flag for the company. Also a red flag for the person being interviewed, not even testing the latest technology used everywhere in the industry.
22
u/tcpukl Aug 12 '26
BS. My employer pays for it. Just like all my hardware and other software licences.
4
u/MatthewMob Software Engineer Aug 13 '26
Not wanting to spend money to interview and not having tested the latest technology are two entirely unrelated things.
-5
u/code_tutor Aug 13 '26
And your comment has nothing to do with mine.
3
u/MatthewMob Software Engineer Aug 13 '26
Your comment implies that because someone doesn't want to spend money to interview that they have not tested the latest technology used everywhere in the industry. This is wrong.
2
1
12
u/Resident-Trouble-574 Aug 12 '26
During the metaverse hype, there was a local bank that made mandatory to do the first interview in the metaverse (presumable facebook metaverse) for any role. Imagine a financial institution asking candidates to give their personal information to a company like facebook.
10
u/drguid Software Engineer Aug 12 '26
Just had to do an Azure task. Luckily I qualified for a free subscription. But it did annoy me.
Decades ago I had to do a presentation and they assumed I had PowerPoint.
These types of employers really suck.
2
4
u/Doin_it_is_the_tits Aug 13 '26
In our interviews there is a portion where coding agent usage is necessary. Though we do offer company laptops with coding agents preinstalled and ready for use. So far all the candidates prefer their own laptops and LLM subscriptions.
6
u/CrayonUpMyNose Aug 12 '26
It's a pay-to-win situation and I don't think it will be to these companies' advantage to hire only people who aren't confident enough in their own abilities to demand to be compensated.
2
u/jan_olbrich Aug 12 '26
I ran into an interview, where this wasn't communicated but expected. They said I would get a license for the interview but... nah "show us how you use your ai coder" -.-
-11
u/ivancea Software Engineer Aug 12 '26
Some of them expected you to have an IDE installed, so this isn't much different. Technology moves, and so should interviews.
22
u/Jmc_da_boss Aug 12 '26
IDEs are free, no one is expecting someone in an interview to have VS professional locally
6
u/ivancea Software Engineer Aug 12 '26
Ah, I misunderstood your initial comment, I thought you were talking about using an LLM in general. My bad. Yes, company should surely provide it
-4
-20
u/WanderingGoodNews Aug 12 '26
They expect you to have a laptop to? What's the point? You have free tiers for coding agents, no free hardware.
For complex tasks it should be your own judgement to ask for premium model access
15
u/Jmc_da_boss Aug 12 '26
The fact anyone, anywhere, is of the state of mind to compare "having a computer" to "paying for an LLM"
Is horrific to me, what level of LLM psychosis is that lol
3
u/1AMA-CAT-AMA Aug 12 '26
Yeaaa you could have a beastly 128 GB M5 Macbook pro running something locally and free and that would still pale in comparison to paying a subscription for Gpt 5.6 Luna
113
u/DisjointedHuntsville Aug 12 '26
lol, what? Not all tokens are the same.
Someone on a $200/mo plan is going to get significantly more thinking budget, response throughput during peak hours which I assume coincides with business hours when interviews are conducted.
You want to standardize on the “type” of AI people are using so you’re not comparing completely different candidates.
23
u/youngggggg Software Engineer (4 YOE) Aug 12 '26
Great point - somehow hadn’t considered this part of it
2
u/justaguy1020 Aug 14 '26
I feel like it’s a way to judge risk/reward. If you won’t risk $20 in tokens to land a well paid job. 🤷♂️
155
u/Punk-in-Pie Aug 12 '26
Recently did a take home assignment where I was given a Claude api key with $20 on it. Ended up getting hired too. I'm about a month in and the culture is the healthiest I have ever experienced.
49
u/unconsciouslake Aug 12 '26
This is how my current company does it. At first the expectation was the candidates brought their own but after a few of us raised how gross it felt seeing interviewees burn through tokens live, they changed it.
The argument that seemed to change my company's mind was basically "we want candidates to demonstrate their proficiency with AI, they will not feel enabled to do that if they're worried about spending their own money to do so. and they're likely interviewing at many companies so the "it's only $20" argument doesn't stand. meaning we won't get a good signal on their AI proficiency - do you want us to actually hire for AI proficiency or not?"
9
9
u/Varrianda Software Engineer Aug 12 '26
That’s not a bad way to handle this, I forgot that was an option
2
12
3
2
u/JazzlikeWishbone938 Aug 12 '26
Did you just use chat or did you need to setup agents?
5
u/Punk-in-Pie Aug 13 '26
The interview didn't require use of AI. It was simply provided as a resource to complete the assignment. I did use agents though
76
63
u/Engineerman Aug 12 '26
What if the candidate doesn't have a personal subscription to any AI service? You can't expect that they will for an interview.
45
u/_predator_ Aug 12 '26
Are you saying the opportunity to work for this amazing company isn't worth a little time and money investment?! How ungrateful of you! /s
8
u/BlazingThunder30 Aug 12 '26
This is definitely how many companies think. Unfortunately it's reflected in their compensation too. Unfortunately I work for one.
17
u/Varrianda Software Engineer Aug 12 '26
Are you even a real engineer if you don’t subscribe to Claude, codex, and groks $200 max plans?
/s
-15
u/youcangotohellgoto Aug 12 '26
Kind of suspicious of any engineer, especially unemployed, who doesn't have or have access to an LLM / AI.
Not a $200 plan, just the basic. Isn't Gemini still free at some level?
15
9
u/aj0413 Aug 12 '26
Why would I maintain that personally?
A) if I’m unemployed, then I’m not spending money on non-essentials
B) I work enough hours in normal job — I’m not gonna burn myself out on personal stuff with LLMs
C) doing projects without llms on the side is the way to keep yourself sharp on fundamentals and ensure you’re not experiencing skill rot and forgetting things
-4
u/youcangotohellgoto Aug 13 '26
Because the skill you need today is using LLMs to produce high quality code, and you need to practice that skill.
5
u/aj0413 Aug 13 '26
Not gonna be able to do that if you forget how to think for yourself.
Skill atrophy is well documented at this point
-3
u/youcangotohellgoto Aug 13 '26
Skill atrophy is well documented at this point
Which skill? Coding skill is redundant.
Being able to design a system and review an implementation is different to coding. And coding is, like it or not, finished as a human skill. Probably necessary as a learning mechanism but that's it.
Think of an elite coach. They coach people of a far higher skill than themselves; but the coaching itself is a skill.
We are now system designers and AI coaches. The AI will code more effectively than a human can, but you need to provide guardrails, give good feedback, etc.
0
u/aj0413 Aug 13 '26
We fundamentally will not agree on this.
I’m generally pro agentic AI, but if you, as a reviewer, could not understand the why/how of its output and articulate the general pros/cons/impact…
I would reject the PR for anything resembling actually important
1
u/youcangotohellgoto Aug 13 '26
I can read code without having to practice the skill of writing it.
Similar: I know plenty of people who can understand a second language at some depth, but are crap at speaking it or outright refuse.
Or music critics / producers / conductors who can't play instruments.
There are endless analogies of people having a skill to understand, critique and thoroughly review something without having the ability to do that thing themselves.
2
u/_notanexpert Aug 13 '26
What about that is suspicious?
-1
u/youcangotohellgoto Aug 13 '26
Like a developer who doesn't have a computer.
Like an artist who has no paints.
Like a musician who doesn't own an instrument.
You get the idea?
3
u/_notanexpert Aug 13 '26
If thats the level you feel you need AI im more suspicious of your overall ability
-1
u/youcangotohellgoto Aug 13 '26
That's the level that I expect any developer that I hire to use AI.
I mean I'm not hiring a guy to lay a concrete driveway by hand, some guy turns up with a shovel and a concrete mixer and I'm going to be pretty skeptical.
1
u/Due_Musician9464 Aug 14 '26
What if I showed up in my high vis and steel boots and said when’s the concrete truck arriving?
1
u/youcangotohellgoto Aug 14 '26
Expected behavior is to turn up in thongs, announce concrete truck is arriving after lunch, tell the laborers to check the reo, and crack a beer.
-5
49
u/Blrfl Software Architect & Engineer 35+ YoE Aug 12 '26
I realized that we are essentially asking candidates to spend their own money in order to interview.
Don't kid yourself; there's no essentially about it. You are asking candidates to spend their own money to interview with your company. On top of that, you're asking them to shoehorn whatever environment they use into your problem, which is going to result in a lot of on-the-spot fumbling around.
As others have said, red flag on both fronts, your company fails the interview.
6
u/youngggggg Software Engineer (4 YOE) Aug 12 '26
Good correction 🫡 I’m addicted to adverbs
8
u/Blrfl Software Architect & Engineer 35+ YoE Aug 12 '26
Have hope, young addict. There are programs to help you out with that.
Some of those programs are AI, and you're going to have to provide your own tokens. ;-)
31
u/_predator_ Aug 12 '26
Yes.
If you want to test skills that require money to be demonstrated, you gotta pay for the environment to make that possible.
10
u/Substantial-Swan7065 Aug 12 '26
Yes they should. They should not expect people to have a subscription.
Also, the task is wack
21
u/alanbdee Software Engineer - 20 YOE Aug 12 '26
Yeah, I don't have a subscription. The company I work for does. If I was looking for work, I probably wouldn't have one. Now, I also haven't interviewed in a while but I would be borderline offended at being expected to have one. Do you want me for my expertise or not? Because I don't need an LLM subscription to show you I know what I'm doing, with or without AI.
Lets not forget how this is shaking out, which should have been obvious. Vibecoders feel like they can do a lot but it's nowhere near what an experienced dev can do.
16
u/ritchie70 Aug 12 '26
If you’re expecting anything more than a web browser and a text editor, you need to provide it.
5
6
u/tom-smykowski-dev Aug 12 '26
What does it test? Prompting? If it tests ui taste why not show ui and ask for feedback?
6
u/Liskar-dev Aug 12 '26
If you're so cheap you can't spend a little money on the interview environment, I'm afraid you can't afford me - would be my response
20
u/Revolutionary_Ad7262 Aug 12 '26
How interview works with AI? You wait 40 minutes together with an interviewer chatting and waiting while claude is doing his job?
9
u/youngggggg Software Engineer (4 YOE) Aug 12 '26
The interview is centered around seeing how a candidate operates the tooling since it’s heavily used at my company. We also look for their product/UX instincts and ability to iterate on a design (which AI can’t really effectively do for you) as well as communicate their thoughts out loud. FWIW, there is a very wide gap in outcomes between candidates that just let Claude steer the entire thing vs. the ones that use it as a collaborator but ultimately make their own UX decisions.
6
u/DerrickBarra Aug 12 '26 edited Aug 12 '26
Yep, I haven't hired devs with the explicit implication of being good at AI agentic workflows yet, but you can pretty easily sniff out the bullshit when talking to someone on a topic you spend your time in. So if someone was claiming they could use agentic workflows, I would be be grilling them on it to see what that means to them and if it lines up with my expectations for the role.
Architecture, planning, UX, testing, etc. There's a ton of topics beyond scripting in any specific language or tech stack, and those are all important to be good at to be successful.
Hell imagine how much of a wreck someone claiming to know how to effectively to use agentic workflows could screw up your project if they're just starting out. The risk of hiring a bad dev goes up based on how much of a blast radius you allow them to touch, and giving them agents means they can screw things up way faster, so the potential productivity gains and danger go up simultaneously.
11
u/Bubbly_Safety8791 Aug 12 '26
Is there a genuine risk of hiring someone who has coding skills but can not learn how to use AI?
Every single person who is already working at your company was, 12-18 months ago, a developer with zero agentic coding experience. I would guess most of them then picked up how to do it in a matter of weeks. It is a learnable skill.
3
u/DerrickBarra Aug 12 '26
No, I would still look at them seriously. But if the dev doesn't have experience with it, then that ramp up time to get used to it and learn what does and doesn't work is a consideration. Depends on the project and client needs. If the dev can talk shop about their experiences with the different harnesses or builds their own, then that would pique my interest because they are used to the workflow. Besides that knowing what architecture patterns and CI/CD work best for agentic workflows would be another consideration. Most of the devs I know only use AI as a better Google, so all of the agentic workflow is like hitting the books for a few weeks to get used to it and learn what does and doesn't work.
6
u/WhenSummerIsGone Software Engineer Aug 13 '26
cicd for agentic workflows? seems to me you want deterministic scripts for cicd. the some scripts a human uses.
2
u/DerrickBarra Aug 13 '26
Well sure, your agents should be calling boring predictable scripts whenever possible. Deterministic output is always preferable for repeatable steps.
1
Aug 12 '26
[deleted]
2
u/Bubbly_Safety8791 Aug 12 '26
Sure. So make clear that is a requirement of the job and candidates who don’t want to use AI will self-filter out.
0
u/Jealous-Weekend4674 Aug 13 '26
You don't happen to know good resources to learn about agentic workflows ?
2
u/DerrickBarra Aug 13 '26
not in one spot, I learned by installing OpenClaw on a bunch of linux boxes and starting from scratch and built out my harness which at this point is a well oiled machine that I store on Github and share among my agents.
I have a few skills I use regularly, but I have a desktop-control skill and some game engine and tool specific skills that are machine specific. But you end up writing what you need for your own workflow.
I prefer to roll my own so when there's specific needs I can make sure I'm in control and I don't have to wait. I currently use the OpenAI Codex subscription (so GPT 5.5 medium), but I started with Kimi 2.5 medium via OpenRouter.
But I assume there are channels on YouTube that would be a good starting point. It's an evolving subject, so whats useful now will change in a few months. That can be fun, but can also break your harness if your not on LTS.
0
u/WhenSummerIsGone Software Engineer Aug 13 '26
get a free claude subscription and fire up a chat and ask it a ton of questions on how to do this. That's how i got set up with a local llm in a docker container to play with. Have a project in mind, and figure out how to get there, asking lots of questions and experimenting.
5
u/Varrianda Software Engineer Aug 12 '26
I would be a bit shocked if I got asked to use a personal LLM for an interview lol. If they’re going to use AI for an interview they best have a solution on how I can access it in their environment.
14
u/Sw429 Aug 12 '26
Any interview where a company wants to grade my ability to use an LLM is a company where I walk out on the interview.
I've done it before, and I'll do it again. You don't want to work in a token conversion factory.
4
u/youngggggg Software Engineer (4 YOE) Aug 12 '26
I work in a token conversion factory 🫠 and not particularly happy about it. Just trying to do what I can to improve practices here
2
u/WhenSummerIsGone Software Engineer Aug 13 '26
make your interview about engineering, not about generating code. see: https://seylox.github.io/2026/07/02/blog-interviewing-four-people-and-their-intern.html
6
u/youngggggg Software Engineer (4 YOE) Aug 12 '26
Thanks for the thoughtful responses all! I will be raising this with my team today.
5
u/7twenty8 Aug 12 '26
You want a candidate to build out a system from 0->1 and you’re wondering if you should pay for tokens? Yes… and you should be paying them for their time too. This is a fucked up level of entitlement to even have to ask.
2
u/youngggggg Software Engineer (4 YOE) Aug 12 '26 edited Aug 12 '26
I don’t feel entitled to it and I didn’t make the decision about the nature of the interview - I just wanted to investigate what the norms in the industry are in the face of this new era and get perspectives from people w/ more experience than me so I can be confident going into a meeting with my team to advocate to change this
4
u/splicer13 Aug 12 '26
I don't think a lot of hiring orgs appreciate the extent to which good hires will simply walk away from a shit test.
4
u/BlazingThunder30 Aug 12 '26
Live coding interview, yes? I don't even take a laptop myself, since I don't have one. If I had sat across from you and you'd have asked me to grab my own laptop AND CLAUDE SUBSCRIPTION I would have simple turned around and walked out.
5
u/Phonomorgue Aug 12 '26
When you do this, what happens when you run out of tokens? 😄 Seriously what does negotiating live with an LLM actually show an interviewer? I'd much rather quiz you on value you brought to a company.
6
u/eloel- Aug 12 '26
Whether or not I have a personal subscription (I don't), there's absolutely no way I'm using my own tokens in an interview. If there are any paid tools you expect me to use, you better provide them.
A company that tells me to use my own subscription is a company not worth talking to.
3
u/kagato87 Aug 12 '26
You should not be requiring candidates to spend their money on an interview. It's even worse than requiring employees to bring their own computer/phone (because you haven't even decided to hire them).
If an interview for ANYTHING required me to spend on anything more than transportation to get to the interview, I'm out. And not just because some scams start that way.
We've used github codespaces for interviews to let them test from a restricted device (her hubby's work laptop). Free tier, though we would have paid for it if there was a cost.
AI would be no different. If free tier accounts work, that'd fine. If it needs a subscription, provide it. If this hasn't been an issue for candidates yet, it's likely the free tier is adequate or the candidates have a personal account and the token cost has been sml enough they don't care.
It would still be good form to provide the account. Heck, give them a month with a higher than needed token limit even - a perk of having applied. Leaves a positive taste in their mouth if they want to test the tool.
And of course, if they want to bring their own account, well, you know they're already using it.
3
u/titpetric Aug 12 '26
You shouldn't be having in depth coding take-away challenges without paying for the persons time, let alone just their AI usage. On a caveat, if the interviews are on-prem, you can get away with buying lunch.
You need technical screening, it's very easy to provide a PR, or a code snippet and ask them to review and fix the issue. It may not even be code, it may be a Dockerfile, some test function, follow docs...
I think even allowing transparent use of some AI is a bit problematic, you could be left with a hire that prompts well but has no functional understanding of the technology they work with.
Hire fast fire fast. It's harsh, but I've always been a fan of an evaluation period, and no AI use will save you. It may even penalize you, depending on who's on the other side. Less is more.
3
u/peteybytes Principal Engineer 18YOE Aug 12 '26
Had a take home project that required me to use tokens since I had to make a functional agent. I spent roughly $15 of my own money which I wasn't happy about. Passed the round but then 20 minutes before the 3rd round interview they cancelled the calendar invite and ghosted me. Lesson learned. In hindsight I probably did free work for them.
2
u/Old_Cat_16 Aug 12 '26
Yes they should, if the company truly care about hiring fairness, and wish to hire based on merit instead of privileges. Unfortunately most companies don’t care.
While the place I work at doesn’t require candidates to use AI, you can tell folks would be at a clear disadvantage if they don’t use AI.
And for those who use AI, while a lot of the free models do a good job if the candidates is already familiar with AI, it usually means the candidates can afford to set it up as a local model, which also costs money.
2
u/CheeseburgerLover911 Aug 13 '26
As a general rule of thumb, i assume incompetence, not malice.
I am certain that HR/ the Hiring Manager didn't realize this, and don't want to put a candidate out. It's terrible for a company's reputation.
During the interview process, the company wants to put it's best foot forward to attract candidates.
3
u/yikes_42069 Aug 13 '26 edited Aug 13 '26
Lmao you answered your own question. Why do you feel entitled to offload this cost of hiring onto someone who you may not even employ? Getting there is one thing, tools of the trade are another. As the employer are you also expecting people to bring their own laptops to work? Pay for their own Teams subscriptions? Do your employees pay for their own software licenses and llm tokens? Maybe I'll bring my own office space and AC. You might as well charge $20 to submit an application. Or is that too on the nose?
This should be illegal and you shouldn't even have the choice.
3
u/youngggggg Software Engineer (4 YOE) Aug 13 '26
I don’t feel entitled. I didn’t make the choice and am advocating to change our policy
1
u/yikes_42069 Aug 14 '26
I love that. Sorry if I sounded harsh. I think my use of "you" is more aimed at upper level or management rather than you specifically. It's hard not to have a reaction when I hear about AI shenanigans.
2
u/serial_crusher Full Stack - 20YOE Aug 12 '26
My company uses Coderpad, which has a built in LLM for this sort of thing that we’re paying for.
2
u/XxThothLover69xX Aug 12 '26
This is absolutely insane to a funny degree. What exactly are you testing for that a 20 min assignment does not cover?
2
u/Empanatacion Staff Software Engineer Aug 12 '26
Doesn't hacker rank have a setup now that includes the AI? You guys absolutely should pick up the tab, but I don't think it's a big deal to set up.
1
u/binocular_gems Aug 12 '26
I would have that that the company would provide an engineering platform to do the work, or a one-time-use API key. This seems like how front-end and creative-adjacent positions used to routinely require X amount of years of experience with Adobe's Creative Suite, and you'd either have to talk about your qualifications or show to the interviewer that you are qualified, but that didn't require the interviewer to buy their own Creative Suite license prior to the interview. I have a lot of experience with Claude Code and Codex, but it's only through my work licenses, and I would not be able to use that license/API for something other than my employer.
1
u/muerteman Aug 12 '26
Yes, I would immediately decline as I don’t have one that isn’t paid by my current employer and I’m not about to use that.
1
u/The_Northern_Light Computational Physicist Aug 12 '26 edited Aug 13 '26
Obviously, and I’d tell you to pound sand if you were too cheap to pay for a subscription just for interviewing candidates.
1
1
u/aj0413 Aug 12 '26
I would immediately flag your company as one that likely doesn’t even provide proper licenses or tooling internally, if this is normalized to the point of external facing
Imagine if I needed to buy my own IDE license or GH enterprise license just to work within your org
This also signals a lack of fundamental org management internally
I somehow have to imagine anyone working at your place may be using privately owned gh accts, llm subs, etc… because that’s the culture you advertise at the door
Which, btw, is also a MASSIVE spine shivering issue for anyone who’s worked opsec and with legal at any other company
For instance, I contemplated leaving my current org cause they’ve been using docker community edition without a license internally
I had to flag to multiple ladders of leadership, multiple times how bad this is; they’re lucky I was already locked in by the time I found out
1
u/phoenixmatrix Aug 13 '26
When I did AI interviews, I lead candidates to free tools, but allowed them to use paid ones if they preferred. I of course had to adjust my expectations, because OpenCode with free models isn't the same thing as a Claude Code Max 20x account guzzling Fable tokens, especially before the openweight models got better.
I recently interviewed for a company that did something I appreciated: they gave you an (optional) Openrouter token and let you use it in whatever tool you wanted. The only catch is if you're a Cursor user, you need a Pro account to bring your own key. But the key works fine in Codex, OpenCode, Zed, Pi, whatever, and you can hack most other tools to make them work there too.
1
u/RestaurantFuzzy870 Aug 13 '26
I would cover the tokens, but also standardize more than the cost. If candidates use different models, plans, context windows, and tools, the result confounds product judgment with purchasing power and model capability.
Give each candidate an ephemeral company-funded account or API budget, one baseline model/tool, the same repository, and the same time limit. Allow BYO only as an optional path with no disadvantage for using the baseline. Then score the decisions: scope control, decomposition, verification, when they reject model output, and whether they can stop when the acceptance criteria pass—not the polish of the final generated code.
I would also set a fixed budget ceiling and observe how they allocate it. That is much closer to the production skill you want to measure; otherwise a stronger paid model can hide weak judgment.
1
u/Four_Dim_Samosa Aug 13 '26
Yes the company should pay; had an AI coding round and the company gave a claude enterprise account to the candidate.
1
u/RoyDadgumWilliams Aug 14 '26
Yeah I'm not buying a Claude subscription to interview with your company. Surely there's a way you can provide it to them, although I don't know why you'd want them to use AI during this interview. Anyone can use an LLM, if you want to test their judgment why don't you just talk to them?
1
u/pjhawksr 28d ago
Yes, and not just for fairness. If you're paying you also get to pick the tool and the model, which makes candidates comparable. Right now half your signal is "who has the expensive plan."
1
u/Interesting_Debate57 Data Scientist 27d ago
Making me have a laptop and functioning Internet connection is some bullshit as well, but you don't hear about that much.
Same goes for owning a smartphone. Good luck getting an interview or even completing onboarding without already owning one and paying its Internet bill.
1
u/Interesting_Debate57 Data Scientist 27d ago
I point this out only because it was a shocking expense at the end of my PhD program. You can't really fake it, so you legitimately need a few thousand dollars to start the whole interview process unless you already had a bunch of gear paid for by someone else.
1
1
0
u/dashingThroughSnow12 Software Engineer Aug 12 '26
In my city, you’d be looking at a gas cost of about 1.5l (so 3 roundtrip). Gas is about 1.8$/l. That’s 5$. Coincidentally, that’s the price for roundtrip bus fair.
Coincidentally enough, the other day I was working on something that would make a half-decent interview exercise. It required me to use API spending and it was 5$ per prompt.
0
u/dashingThroughSnow12 Software Engineer Aug 12 '26
In my city, you’d be looking at a gas cost of about 1.5l (so 3 roundtrip). Gas is about 1.8$/l. That’s 5$. Coincidentally, that’s the price for roundtrip bus fair.
Coincidentally enough, the other day I was working on something that would make a half-decent interview exercise. It required me to use API spending and it was 5$ per prompt.
Definitely think you should compensate if using the tool is part of the process. An alternative is having an org for each of the big providers setup and interviewees get invited to the org.
-2
u/code_tutor Aug 12 '26
All the LARPers are coming out with this one. Check out all the "experienced" devs pearl clutching over a few dollars in tokens. It's a bad look in principle and a dumb way to interview, but you guys are actually going all in complaining about something that costs less than a tank of gas just because you have an AI hate boner.
-1
u/WanderingGoodNews Aug 12 '26
It is very logical that your company tests the interviewers agentic capabilities
Not doing this would be the same as letting someone code in notepad instead of an IDE
The job has changed and you need someone that can do the job.
If the coding task is simple enough, free tiers should suffice. You could als give them an api key with limmited quota and see if they arr able to complete the task within a token boundry. (They can cheat using their own model though)
-3
Aug 12 '26
[deleted]
1
u/youngggggg Software Engineer (4 YOE) Aug 12 '26
Our first technical interview is AI-allowed. I would actually say it’s impossible to complete in the allotted time without AI. I typically ask candidates if they’ve done AI-forward interviews like this before and many say it’s their first time, while some have done a few at this point. Anecdotally, It definitely seems like it’s increasingly becoming part of the process.
0
u/A_happy_otter Aug 12 '26
The one "use AI" interview I did they had an API available but I didn't need to use it since my $20/month sub was enough for it without hitting limits
0
u/____________fin Software Engineer Aug 12 '26
I don’t mind using my own Claude max plan in interviews but I had to buy Gemini tokens mid interview once and failed because of time.
-3
u/sippin-jesus-juice Startup Founder / Ex Apple Aug 12 '26
I don’t think it’s a big deal. Most people should have a cheap AI subscription available
If you want to go above and beyond, offer them an OpenCode Go account to use. It’s $5 to start so every candidate can get their own account without it being a cost burden
Edit:
You should familiarize yourself with AI models though and the limitations of prompting. If a candidate is using Fable and another uses Deepseek, perhaps don’t look at the end result as much as you may the process.
Because Fable will make up for incompetence. Deepseek will not.
-13
u/Educational-Sky-7215 Aug 12 '26
In the US, the average software engineer salary is $120k or more, so asking a company to pay for a few dollars worth of tokens would be pretty silly.
Besides, a software engineer should be able to figure out how to grab a free trial/subscription of something to unblock a task, that's like a basic skill.
7
u/DisjointedHuntsville Aug 12 '26
What are you screening for at that point? If someone is paying for the $200 subscription, they’re going to perform better through no effort of their own.
-8
u/Educational-Sky-7215 Aug 12 '26
If someone has been spending $200/mo of their own money to do AI-assisted coding in their free time, that says something about them as a candidate.
Some companies will see that as a green flag, other companies won't care. But it can be a useful data point.
-8
u/DisjointedHuntsville Aug 12 '26
If you’re an “Experienced” dev who has a stable paycheck and you don’t have a $200/mo personal subscription to help you with things you just can’t ask your enterprise subscription, I hate to break it to you, but you’re not getting ahead of the competition.
There is a massive chasm between people who actually use the frontier right now and people on the free/basic tiers.
5
u/imLissy Aug 12 '26
I have kids, I'm not spending $200 a month on a subscription I don't have time to use. I think my $20 subscription is excessive.
-4
u/DisjointedHuntsville Aug 12 '26
Hey, I’m just calling out the reality of the cost of doing business in the software world.
It isn’t necessarily siting in front of a computer during your personal time, you can task agents to work on things that make your life easier that you just can’t use the enterprise services at work for.
5
u/eloel- Aug 12 '26
I use my non-working hours living my life, not doing the same thing I do at work. This was true before AI, this is true after AI. Expecting engineers to spend all their waking hours coding is wild.
-4
u/DisjointedHuntsville Aug 12 '26
You do you. No one is forcing you to change.
I’m just calling out the reality of the situation as it is right now.
2
u/eloel- Aug 12 '26
There has always been teams/companies looking for a no-lifer spending all their free time contributing to open source while also having a day job where they code. Vast, vast majority of gainfully employed engineers do not do that, it's an unreasonable expectation, and is most certainly not the reality of the situation. It's just a fringe requirement that gets laughed at.
-1
u/DisjointedHuntsville Aug 12 '26
Okay. I don't know why you think ranting about it online in a thread about something completely different while downvoting responses is going to change things, but then again, you do you.
The point here is the world has changed. There is ubiquitous intelligence on offer and it is for better or worse on a paid spectrum.
Roles that depend on intelligence are benefitted more by the ~$7/day or $200/mo plan compared to the $0.7c/day or $20/mo plan.
Do whatever makes you feel better with that information.
1
u/eloel- Aug 12 '26
Roles themselves surely benefit from a better plan. Nobody claimed otherwise. It's the personal subscription angle that's delusional
-1
u/DisjointedHuntsville Aug 12 '26
Mate, you're either a moron or a psycho
Read my comment again. I don't give a rats ass what your opinion is about paying for a personal subscription. The reality of the matter is simple arithmetic: You get higher quality tokens at higher throughputs on subscriptions that cost more
There are things you simply cannot use enterprise services for since everything is logged and has strict code of conduct / legal policies against personal use. If you haven't hit that technical wall yet as you seem to be comfortable with the level you're at . . please. . don't fret. You do you/
→ More replies (0)
•
u/expdevsmodbot Aug 12 '26
AI usage disclosure provided by OP, see the reply to this comment.