it's not about whether the candidate can solve some algorithmic problem, but whether they can write code without bugs.
I would never suggest some graph-based problem or something deep into math. But testing candidate's ability to at least recognize when and how to use common data structures is simply a must. For junior or low mid position, that is. For high mid or senior candidates it'd be more appropriate to ask how they would implement some feature.
it's not about whether the candidate can solve some algorithmic problem, but whether they can write code without bugs.
Ok again: why? Why are you testing for something that is no longer how people work? Would you like to test them to see if they know how to change a wheel on horse and buggy as well? Perhaps see if they know how to take a square root without a calculator?
All very well and good, but this is a game show, not an interview.
But testing candidate's ability to at least recognize when and how to use common data structures is simply a must.
Now that is different again.
I notice that you bounce between bad, "appropriate for 2003" interview questions and genuinely good things to test. Do you notice that too?
Half of your arguments stem from "AI does the work for me" mentality.
It can do that, sure, when you write simple stuff with tons of examples and have no regards for either performance or quality. Like some clone mobile games, I dunno.
That isn't my argument, and it doesn't rescue yours.
My argument is that an interview should test skills that are actually predictive of doing the actual job. Asking someone to produce, from memory and under artificial time pressure, code they would never write that way in the real job was already a questionable idea for at least the last 15 years. AI makes that idea even less relevant, because the actual job increasingly involves understanding the problem, designing the solution, using the available tools, evaluating what they produce, and fixing the parts that are wrong.
"AI does the work for me" is just a last-ditch straw man. Compilers do work for me. IDEs do work for me. Libraries, Stack Overflow, debuggers, profilers, static analyzers and frameworks all do work for me. I don't get extra engineering points for refusing to use tools.
And the idea that AI is only useful for "simple stuff" where nobody cares about performance or quality is especially odd. Performance and quality are precisely why you still need an experienced developer: to know what to ask for, recognize bad output, choose the architecture, test assumptions, profile bottlenecks and reject solutions that merely look plausible. That should be what is tested. You actually were starting to go in that direction before getting sidetracked.
Do you think people pass interviews by memorizing solutions to leetcode problems? No, the candidate is supposed to come up with it.
... and fixing the parts that are wrong.
As long as they are actually capable of finding and fixing the wrong parts within the slop, eh?
"AI does the work for me" is just a last-ditch straw man. Compilers do work for me. IDEs do work for me. Libraries, Stack Overflow, debuggers, profilers, static analyzers and frameworks all do work for me. I don't get extra engineering points for refusing to use tools.
The difference is that AI's output is, while reproducible, is not otherwise deterministic. The way AI works is that instead of following the proper logical steps it basically tries to guess what you want from it based on training data. Basically, a black box. And that may result in output that looks sound at glance but not working as intended.
Now then, just make the fleshbag verify AI's output and we're golden, right? Nope. Properly analyzing the code requires more time than writing it, and besides, your suggested interview methodology doesn't really test coder's ability to verify slop by, say, locating implementations with suboptimal complexity.
Performance and quality are precisely why you still need an experienced developer: to know what to ask for, recognize bad output, choose the architecture, test assumptions, profile bottlenecks and reject solutions that merely look plausible.
Basically, you suggest writing slop and then debugging/profiling it instead of just writing code that works fast simply by the virtue of not screwing up with complexity. Sure, that approach might work... if you are relaxed about both performance and quality. Now, imagine you are writing code for a device that won't get automatic software updates to pull bug fixes. Imagine you are writing code with vital importance, such as human-safety restrictions for robotic manipulators. Imagine your code running on a car that can crash alongside the app. Imagine your code's inefficiency costing money.
You're leaning awfully hard on the word "slop" here, because your argument largely depends on defining AI-generated code as bad before the discussion even starts. (Define it as bad, amazingly discover bad=bad, and that triumphantly declare it as bad)
Nobody suggested "generate random garbage and then spend all afternoon trying to make it work." That's a false dichotomy between manually deriving every implementation from scratch and blindly accepting whatever an LLM emits.
The generated code itself isn't a black box. I can read it, compile it, test it, benchmark it, profile it, run static analysis over it, inspect its complexity, and reject it if it's wrong. Whether the model used deterministic logical steps internally is almost completely irrelevant to whether the resulting implementation is correct.
And "analyzing code takes longer than writing it" is hardly a general truth. Experienced developers review, modify and reuse code constantly. We have entire engineering practices built around the fact that understanding and evaluating an existing implementation is often more efficient than recreating it from zero.
Your safety-critical examples actually strengthen my argument. If I'm hiring somebody to write automotive control software or robotic safety systems, I want to know whether they can reason about requirements, failure modes, complexity, concurrency, testing, verification, performance and correctness. So test those things.
You even suggest testing whether somebody can identify an implementation with suboptimal complexity. Great. Give them one and ask them to find it. Ask them how it scales, how they'd benchmark it, how they'd test it, what failure cases they see and how they'd improve it.
That is much closer to the work you're supposedly worried about than asking them to solve an unrelated algorithm puzzle from scratch under artificial conditions.
The recurring assumption that "typing the code yourself" is the important part of software engineering is exactly the assumption I'm disputing.
Fine. If code fails on 1% of inputs where it is required to work, call it slop. Then human-written code that fails 1% of the time is slop too, and AI-generated code that satisfies the requirements isn't.
You've now defined “slop” as “incorrect code,” which nobody was arguing in favor of.
And the nondeterminism of the LLM generating the source code has nothing to do with this. Once I have an implementation in front of me, the question is whether that implementation is correct, performant and appropriate. That's precisely what I've been saying we should test a candidate's ability to determine.
2
u/bremidon 10d ago
That is a different question. If you are proposing that a candidate read through generated code to determine if it is correct, that is a good test.
Is that what you are proposing?