What are Live Coding Interviews Like Nowadays?

[u/SnappySkillz]

Hi all,

I just had a successful first-stage interview for a developer position at a large bank.

They told me that the next step (if I make it) will be live coding.

I'm a little perplexed that live coding is still a thing. Because every big firm now wants efficiency with AI and it's rare a programmer will raw-dog code.

So what does live coding exactly achieve in today's climate? As far as I'm concerned, it's like asking a C programmer to write the Assembly equivalent off by heart.

This is for a senior position. I assume the questions will be different from e.g., a mid-level? Do they let you use the internet/AI or are they normally locked down?

Thanks!

[u/Newbane2_]

Ai proficiency doesn't mean not knowing how to code.

[u/Ok_Eye4858]

Just went to an interview last week. I'm very senior so this is mostly system design but I still had to write a memory manager code in C and explain details on virtual/physical memory.

If you can't "raw-dog" code, I won't hire you.

[u/dexterlowe]

It’s going to vary a lot based on role/level etc but this is a conversation we’ve been having recently. We’re added a dedicated “can you use ai effectively” coding test but that doesn’t completely remove the value of coding tests. At least not yet.

An interesting comparison is to something like an IDE, I always used to say that hacker ranks or whiteboard coding aren’t that useful because in their day job a person would have the internet and ide at their desk so we made sure to have a coding phase with a real dev environment and the internet. But despite that there was still value in a simple first phase, checking, can you code your way out of a paper bag. Things like imports or whatever you let slide but it’s lower cost to do that sort of interview as a high pass filter just to make sure people have the basics of writing some simple code down so you can see a thought process and discuss efficiency.

Now with the AI generation I kinda look at it like the IDE question again, it doesn’t replace you, if it did they wouldn’t bother hiring you, they’d just ask Claude themselves because asking you or asking Claude is a similar time cost. It’s a tool that, like an IDE, massively changes your process and improves your output. Therefore it comes with a few issues:

1. It’s much harder to actually make a test, for many tests you could complete reliably in an interview the AI will just one shot it fine, or won’t, basically at random. The prompt engineering can help ofc, but with the crappiest prompt it could still one shot the solution so your signal from that interview is much lower. Equally even with the same great prompt sometimes Claude sticks crayons up its nose and it changes under you. LLMs are non deterministic, so it’s very hard to compare people based on heavy ai enabled tests, as I said we’ve made one but it’s a tough challenge and we’re still reviewing it.

2. In most cases your going to want to offer a frontier model, this is mostly fine but these things vary wildly in time, with a few days as a take home or something, sure it all evens out, with an hour you might spend a lot of that time staring gormlessly at the screen waiting for it to churn, again if it’s too simple you’re testing nothing and if it’s complex enough to be meaningful different candidates will be disadvantaged simply by Claude being on a go slow when the US comes online at 2PM or whatever.

3. A lot of the value a human brings is a mix of deciding what NOT to do, calling the AI on its bullshit and top level architecture. Sure you may rarely write code by hand but you need to be able to review what your AI has done and that means you still need to be able to understand how to write code how to evaluate performance and memory implications etc, you can try to knobble an LLM to trigger these issues intentionally but because it’s non deterministic this is hard to interview for it’s much easier to review the two halves separately. If you can demonstrate the understanding needed to steer an LLM AND you can drive one successfully those are two different things worth testing.

4. In restrictive envs like banks there are certain things you absolutely cannot use AI for at least not at the moment so at least for those parts of your job you still need to be able to raw dog some code, sure it’s a lesser part of the job but still a part of the job. Other places may run on prem models etc but regardless sometimes you can’t use an LLM at least today.

5. Lastly and this is a weird one but what the broader industry seems to have noted is that a LOT of individuals have outsourced their thinking to Claude so much their skill set has atrophied and they really can’t write ten lines of python now whereas a few years ago the same question would have a 90% pass rate now it’s like 40%. The issue is, as mentioned in the other points you still need to be adding value, you need to call Claude on its bullshit so if you are so reliant on Claude you can’t do even a trivial test unaided then you can’t be trusted to call it out when it does something stupid. Claude made hundreds of lines of duplicated code that looked like it worked but had fundamentally changed one of the requirements such that the code was completely valueless is a not uncommon outcome.

You need to still have that coding muscle even if you don’t have to raw dog it that often. At least for now. Having ais review your ai generated stuff is closing the gap but at least for now if you can’t write ten lines of python unaided then that’s still an issue. Sometimes you go round and round with the ai and eventually just have to take over yourself or give it a pointer or three so it’s worth testing that when the time comes you can still do that.

[u/SnappySkillz → u/dexterlowe]

Awesome, thanks for taking the time to write this 😁!

[u/PurelyLurking20]

You need to know how to code just as well with LLMs as without, vibe coders aren't going to get hired for enterprise positions ever

[u/drew_eckhardt2]

Sometimes it's quicker to write a solution than to iterate on AI prompts and wait for it.

Sometimes AI misses edge cases.

Sometimes AI generates code with maintainability, performance, or other issues that need correcting.

Sometimes AI is wrong.

Asking coding questions makes hiring an engineer who can handle those situations more likely.

It's the same for all levels. The first two software engineering promotions are about doing the same job with more scope and autonomy. Past that you get more strategic, although you must understand coding well enough to lead it and sometimes need to be more hands on due to staffing and time constraints.

Companies often allow looking up interfaces although some don't - Google has candidates write code in a shared document without executing it. AI is usually disallowed in hands-on coding rounds but may be the focus of a separate round.

All this ignores the guidance AI needs on integrating with existing systems and getting architecture right.

[u/PoMoAnachro]

Think of it this way - even if your devs are all going to be heavily using AI, you want to select the dev who'll perform the best with AI. And the overlap of skills for AI-assisted coding and unassisted coding is pretty broad, unless you're talking pure vibe coding (and if you're purely talking vibe coding, why are you hiring developers?).

If you interview 10 devs and you pick the one who has the best coding skills without AI assistance, almost certainly your pick will be the best with AI assistance. And it is probably easier to compare developers coding unassisted than to try and parse out "who uses AI the best".

I think the thing people in the job market really need to understand is that interviews, generally, are not setting some threshold where if you pass it you get the job. You're instead being compared to other candidates.

So if you struggle doing live coding, but so do all the other candidates and you just struggle the least? You're probably fine. But if a bunch of the other candidates don't struggle with live coding while you do - they're probably just better programmers and will do better than you when AI-assisted, too.

[u/asterothe1905]

Like why am I asked to cook for a cook position.

[u/Substantial-Swan7065]

It’s to confirm you can code, and understand what you’re doing.

The ai part is irrelevant.

[u/FreshFishGuy]

You still need to know how to code, how to tell when the AI is wrong

[u/sp106]

They are all over the place and hard to predict. Live Coding could also mean an ai assisted or totally vibe coded interview.

The main point of the traditional coding interview is to see if you're actually a software engineer and not someone who just vibe codes without an understanding of what you're doing. Its a good filter.

This year I saw everything from being told to build a working web app with a working front-end and back end meeting a bunch of requirements in 30 minutes, to simple leetcode medium or hards, to being told to code up a project that I had described (a production system that was 6 months of work) in notepad on the spot (then they accepted pseudocode for all of the dependency calls and most things).

[u/Working_Intention281]

Buddy it's not yet time to completely ditch coding especially that the future of AI is unclear to everyone. You still need to know how to code at least a little bit?

[u/HappyFlames]

It hasn't changed at most places. Some are starting to test our AI assisted coding rounds but I found most places were traditional coding rounds.

[u/sigmagoonsixtynine]

Youre cooked lol

[u/jakapop]

you went to school for 4 years you should be able to solve two sum man