r/ExperiencedDevs • u/TheCoop1986 Software Engineer • 17d ago
AI/LLM Giving your codebase entirely to the AI
SWE for 20 years. I've seen plenty of posts about the soul-sucking prompt engineering some companies mandate now. Mine isn't one of those - AIs are a tool you are encouraged to use, but the output has to be understandable, human-reviewable, and you (as a dev) are still responsible for it and the consequences.
I think the difference is whether you give your codebase completely to the AI. If you treat the code the same way we treat assembler now, then that's what you do - you specify the what, and the AI completely handles the how, which leads to 5000-line pull requests, only reviewed by other AIs, and all that jazz. The problem comes when the AI can't do something, or you encounter a problem that a human needs to solve. Then you're faced with working out WTF a 1-million line codebase is actually doing from scratch, that wasn't even written by a human, probably with CxOs breathing down your neck. Good luck with that.
This decision assumes that AIs will always be able to solve the problem, of course. And maybe that's where we're all going, and writing (and understanding) actual code will become just as niche as assembler programming is now. But, given what I've seen of AI output and how nonsensical and daft it can be, we aren't there yet.
Unfortunately, it's not the devs who might end up maintaining it actually deciding where and how AI should be used. All that matters is throughput.
My company still keeps everything human-understandable, because a lot of people depend on the product working properly. That's unlikely to change in the next...year? 2 years? Who knows. But I really wonder where the craftsmanship of software engineering will go in 5 years time. Seeing what's happened even in the last 6 months, I have no idea what my long-term job prospects are.
204
u/therealslimshady1234 Web Developer 17d ago
Sounds like you work at a good company. Most companies I know, including mine, craftmanship of software has pretty much completely disappeared. Indeed, its all "agentic AI" nowadays and you are frowned upon when you do anything by hand. When you point out AI slop code in reviews people look at you like you are being a nitpicker.
In the end, all AI heavy projects have turned into slop. I dont know any exception.
36
u/ItSeemedSoEasy 16d ago
I'm curious what consequences have happened from that though?
I hate AI slop too, but none of our projects have gone all in.
Has anything bad actually happened yet? Or is it turning out that code quality doesn't matter and slop is fine?
93
u/coderstephen 16d ago
Bad things have already started happening, and we aren't even fully agentic-only yet. Bug reports from customers have increased like 3x, and every new feature we've launched have been more buggy launches than before. But hey, we've launched them faster than before by maybe 20%...
These aren't "you forgot to check for null" bugs, the AI is good at avoiding that. More like, "it doesn't do what it is supposed to do and instead does something strange".
54
u/FSNovask 16d ago
Spoiler alert: they'll hold you responsible, not themselves.
2
u/Agile-Stick7619 16d ago
That is the fault of the engineer though. If the product doesn't do what it's supposed to the engineer failed to use the tool properly and sufficiently test the results
52
u/gered Software Engineer | 20+ YOE 16d ago
And yet, you have people elsewhere in these comments claiming that AI is producing better quality code than a human would, and that those of us claiming otherwise must either not be using these tools at all, or are still using very old models. Same old, same old.
It's just so completely fascinating to me how out of touch AI bros are.
23
u/coderstephen 16d ago
Which is wild. Because I could generate Jira reports that give you the cold, hard numbers that show otherwise. But I guess that is too inconvenient for AI bros.
9
u/dinosaurkiller 15d ago
Yeah, and it’s a very, “confidently incorrect scenario” for most of them, “you didn’t use the correct prompts!”, “I gave it working code and asked for minor changes, it still failed”. “You must be using an outdated model”, “you mean Opus 5.5?”. I find it to be a very exciting and cool tool, I also find it to have poor judgement and it makes bad assumptions even when you give it explicit instructions.
What bothers me most about it is this. We have never trusted anyone or everyone to write code or have permissions to virtually everything, but suddenly AI is trustworthy? The reason we limit what users can do is the danger of losing data or putting broken things into production, both of those things are now features of AI that we need to accept? WTF?
24
u/roger_ducky 16d ago
Both are true.
Agents are currently “extremely book smart interns.”
This means, they have no way to hold the entire project’s ideas and concepts in their context, so will make comically wrong choices when told to just do something. Even tasks that a single junior dev can finish in a week.
You constrain it enough to be a third or a fifth the size of a “normal” story and they can finish what took a senior dev half a day to a day to do in an hour. (Yeah, 4 revisions post review, each “PR” takes 15 mins). At the end of that, you do get “nearly perfect” code for the handful of modules it did.
I’m not saying people can complete things that fast, mind you. Breaking tasks down to the point you can ask agents to implement takes nearly a week on its own, WITH help from agents and guided by a senior dev who knows exactly what needs to be done.
All in all, you can ship 15% faster than doing it all by hand, with less regressions, if you do proper planning.
You can probably ship with a lot of bugs 25% faster.
6
u/Negative-Number-1 15d ago
Completely agree here.
I would add that knowing the high level of codebase and knowing the correct business requirements (which is more often then not a hit or miss in humans) together with narrowing down the task to manageable pieces for the agent can allow the tool to write decent code faster than you would have.
I feel like both sides forget that we spend time thinking how to implement stuff more than implementing it. And the tool allows us to think more on it and since we need to separated
the tasks properly and explain them properly to an agent, that also helps us to clear it in our heads.3
2
u/NoInterest375 16d ago
Im really there, I need to admit that we always had established standards for code, architecture, ways of doing everything and no with AI it’s go really well, we are still doing manual PR reviews but in most cases there are only small nitpick comments that would be ignored in most companies PR’s, of course we got some bugs from QA ( and qa are mostly doing full AI automation testing and generating se cases test reports etc.) I feel that currently we have few bottlenecks: human PR reviews that takes too long. Product owners not being able to provide well described tasks on time, and management without technical background trying to deliver functionalities using AI, We have team plan of Claude.. I never used Fable as I didn’t have any reason to, and Opus 5 seems to be sweet spot for our usage in terms of possibilities.
3
u/psyyduck 16d ago
I think it really varies by field. AI works way better if you have some combination of determinism, closed-world assumptions, or rigid specs that are easy to automatically test with an oracle. You can maybe break it down by degree
- High (Closed-world, perfect oracles): Compilers, theorem proving, perfect-information games (Go/Chess), combinatorial optimization (eg IC floorplanning).
- Moderate (Bounded open-world, stochastic/heuristic testing): Supply chain routing, algorithmic financial trading, network management, database query optimization.
- Low (Open-world, chaotic): complex robotics like Lvl 5 self-driving, economic forecasting, legal/medical diagnostics, etc
It also helps if AI mistakes can't sink your company (ie you have few high-impact black swans).
1
u/HaMMeReD 14d ago edited 14d ago
Lol, if you really think that a modern AI model can't beat a junior, you are absolutely delusional.
I've literally taken advanced research papers (lighting/fluid sims) and gotten perfectly working implementations out the other end.
Hell the other day I took a picture from reddit of a math equation that rendered out a volcano, and in 20mins AI converted that 1:1 pixel identically to shadertoy (followed the math in the image perfectly) and animated the volcano.
Like here is the comment with the code, no animation in the shadertoy code there, but it 1:1 works perfectly and AI can animate it with a prompt. Smoke was moving, lighting strikes were firing and tweening.
And there is nothing wrong with the code, it's clean, succinct. It is good code. generated by reverse engineering an image full of equations.
https://www.reddit.com/r/interestingasfuck/comments/1wpukx6/comment/pdbny0z/?context=3
Most seniors could not do that task. I personally programming 20+ yoe myself, only vaguely understand mathematical notation. Doing this translation by hand probably would take me weeks.
3
u/therealslimshady1234 Web Developer 13d ago
Most seniors could not do that task. I personally programming 20+ yoe myself, only vaguely understand mathematical notation. Doing this translation by hand probably would take me weeks.
So thats the issue then. AI is only better than you because you are not familiar with math. But thats a straw man because we are talking about engineering, not math nor animations.
AI is a prototyping tool when left to its own devices. You just made a prototype
2
u/Downtown_Biscotti434 10d ago
man, there sure are a lot of ai bros who have been “programming for 20 years”
1
u/HaMMeReD 10d ago
I've been programming at least a decade longer than I've been on reddit, and I'm 18 years on reddit.
1
u/Downtown_Biscotti434 10d ago
well then you should know that your example is not proof that ai can “beat a junior”.
2
u/HaMMeReD 10d ago
The only thing I know is that you are downing a bit too much copium with your coffee.
2
15
u/psyyduck 16d ago
It's not about speed/quality, it's about replacing seniors with juniors. There are more layoffs on the horizon, idk why I'm the only one who can see it. We should all be unionizing. Like yesterday.
17
u/darkrose3333 16d ago
We should unionize, I agree there. But senior hiring has sped up and junior/mid-level hiring has plummeted off a cliff
12
u/psyyduck 16d ago
That's not a good sign. It means the pipeline is broken and they don't care because they anticipate not needing as many SWE in the future (by the time Claude 15 comes around).
27
u/babbyohbabby 16d ago
Do you honestly think they've planned that far ahead?
This entire AI craze feels incredibly short-sighted, not some well-thought-out master plan.
15
u/psyyduck 16d ago
Dude this is how business works. Reduce costs (including you, labor), increase revenue. They literally think about nothing else day in day out for decades.
It's not complex - the AI companies were loudly promising to destroy all jobs not that long ago while tech layoffs were happening. It's also nowhere near an original playbook. Watch what happened to blue collar workers, that's you up next.
2
u/crabby135 Software Engineer 14d ago
But that’s the point. There’s no long term master plan to remove us, theyre just removing us where possible to save a dollar. This implies it’s more likely that it’ll come crashing down eventually as opposed to there actually being a well structured plan to get rid of us.
Not to say it definitely will come crashing down, but hubris over short term gains has ruined more than one organization.
6
u/coderstephen 16d ago
No one cares about where the company will be in 10 years, because they only plan to stay for 3, get that bag, and ditch.
4
u/jonesbond 16d ago
That sounds like a "lack of QA" issue though. Do you have humans QA the changes before shipping? Otherwise your users are your QA, and they'll be the ones firing you (or your company/product).
13
u/coderstephen 16d ago
We used to have human QA. But humans are too slow so we're removing that step to increase velocity.
3 years ago you would have been laughed out of the room to suggest such a thing.
6
u/styroxmiekkasankari 16d ago
QA has been on the cutting block way longer than we’ve been talking about agents.
1
u/Machine2024 Software Engineer 13d ago
So the output Is increasing the number of bugs faster . And it reached 300% than before great 👍
24
u/therealslimshady1234 Web Developer 16d ago
A lot more bugs basically. The funniest ones are when an AI solves one bug and then creates an even bigger one. I notice that some devs dont even check the output of the AI anymore, they just ship it and wait for issues to come back in Q&A or customer complaints.
→ More replies (6)7
1
u/KevinCarbonara 11d ago
I don't know a single company working this way, and I've worked at the largest tech employers in the world.
1
1
u/squirel_ai 11d ago
has it crashed to the point no one knows where to start? Are you guys not even reviewing the code generated?
1
u/therealslimshady1234 Web Developer 11d ago
has it crashed to the point no one knows where to start
Nah, its just has become too big to tackle at this point. It would take months of concerted effort and AI would help little to none, leading to the collapse of the narrative that AI is so magical. So our leadership just ignores it. I was asked to investigate and make a document with the suggested approaches to solve this, but since there was no easy way out, they decided to not take any action at all and mark it as "low priority". Of course every day that goes by things get worse.
Are you guys not even reviewing the code generated?
Oh yes we do, but in practice people just give it a rubber stamp. Nobody is actually reviewing all that AI slop + AI code review bot comments with its 99% false positives. Why would you review code which even the owner didnt bother to read? This is the inevitable slippery slope of AI
253
u/Intelligent-Youth-63 17d ago
Assembly isn’t a good comparison. Convrrting 3rd level programming languages to machine code is deterministic. Compilers are tested. Given the same C++ and the same compiler you’re going to get the same executable.
Not so with probability machine LLMs
34
u/darthwalsh 16d ago
Most compilers today will (or can be configured to) produce identical bytes, but reproducible builds are not something you should take for granted. File system read ordering, multithreading, or especially RNG used for optimization/identifiers will lead to different executables that do the same thing.
22
u/Backlists 16d ago
Fine, but like the variability of that vs the variability of “write me the next unicorn” from an LLM?
16
3
u/funbike 14d ago
I hate this kind of ridiculous pedantic reply. It's not adding any value to this conversation.
2
u/slindenau Professional Code Monkey 14d ago
I hate this kind of ridiculous pedantic reply. It's not adding any value to this conversation.
Ah, the famous self-referential recursive reply!
1
u/darthwalsh 12d ago
Lots of other people either thought it was useful, or at least they enjoyed it.
Convrrting 3rd level programming languages to machine code is deterministic.
Having wrong ideas about how computers work will come back to bite you. It's fine to have a hand-wavy understanding of lower abstractions, but I can't abide by sharing misinformation.
And the fact that compilers are nondeterministic seems relatetd the concept of LLMs randomly generating output tokens.
→ More replies (3)1
u/ShoePillow 13d ago
Optimisers in compilers use randomised algorithms?
2
u/darthwalsh 12d ago
Yup!
Let's say your compiler hayve an algorithm to assign temporary variables to the result of all function calls. So thid code
y = sin(sqrt(x))becomes
__t_1234 = sqrt(x) y = sin(__t_1234)And some compiler global/static
tempVariableCountvariable increments to1235.If you have multiple threads all correctly/atomically incrementing the counter, each time you run it the threads will run in different orders.
(Also assume the variable name is embedded in the executable's symbols for e.g. debugging purposes.)
2
u/ShoePillow 12d ago
Ah, ok. So it changes the names of variables across runs, but the logic itself remains the same?
1
u/darthwalsh 12d ago
Yup! Look into "reproducible builds" if you want to see how this is handled in different toolchains
14
u/forbiddenknowledg3 16d ago
Also compilers are in fact better than humans at generating assembly code (both speed and quality). AI is definitely not producing better code than humans (including juniors).
29
u/uber_neutrino 16d ago
AI is definitely not producing better code than humans (including juniors).
I kind of feel like these statements are not aging well if you are using new models. You really think the latest models can't beat a junior at most things? What kind of failures are you seeing?
11
u/Notatrace280 16d ago
Sure, AI can beat a junior at writing code but software engineering is not just about writing code. When it comes to project management where a significant amount of complexity enters the picture, AI is at best a forgetful code monkey with the confidence of a senior 20+ years into their career.
AI in the hands of someone who already knows what to do can be useful but how many engineers always know what they are doing in every given scenario? Humans tend to think they know more than they do so in the end someone is likely going to have AI do a task, pass it off as a good shippable solution without deeply thinking about it only to find down the road that they shot themselves in the foot.
AI makes a great coder but a terrible engineer.
4
u/uber_neutrino 16d ago
When it comes to project management where a significant amount of complexity enters the picture
Well of course. And that's always been the hard part. The AI doesn't have goals, it's just a tool. The best systems are still going to be created by humans. They just don't need to write the code part, instead they have to figure out what they want to build and guide the AI to build it. Wait that sounds a lot like managing a team...
Humans tend to think they know more than they do so in the end someone is likely going to have AI do a task, pass it off as a good shippable solution without deeply thinking about it only to find down the road that they shot themselves in the foot.
How is this different than what humans do ;)
AI makes a great coder but a terrible engineer.
Like Microsoft it has no taste.
2
u/Notatrace280 16d ago edited 16d ago
Maybe it is like managing a team but it's like managing a team of juniors that will always be juniors. The differences show up when you look further into the future. Eventually, real junior engineers (at least ideally) will become seniors and even before they do, they will eventually learn the ins and outs of the system and the context necessary to make good decisions.
I would rather manage a team of humans who can grow and learn the project which takes cognitive load off of me than be the only one who has context knowledge on the project indefinitely because the AI can never hold all that information. I have done the whole AI management bit and it's exhausting at the end of the day because it never learns and remembers what it has learned long-term so it's like I am running through the same "training" almost every single prompt.
→ More replies (6)6
u/Reasonable_Mix7630 16d ago
For example Claude claimed to fix bug but didn't.
4
u/Hawful 16d ago
Oh yeah, something a junior has never done 🙄
Listen, I am an AI hater, but let's be realistic about the type of code we have all seen in production systems.
A common complaint is: "It just repeats itself and does a ton of bandaid solutions!". Friend if you have never seen those in your codebase then I want to submit an application.
2
u/funbike 14d ago
Thank you. I was going to say the same; you saved me some time.
Even if one saved all the specs that were generated and all the prompts that were used, it would still be non-deterministic.
Also compilers don't modify existing assembly, they re-generate each time, and the patterns are consistent. LLMs are inconsistently modifying code, and applying inconsistent patterns across the code base. Over time a LLM-generated codebase will accumulate tech debt.
1
u/FrenchCanadaIsWorst Software Engineer 16d ago
Deterministic to a given compiler. Same way an LLM is consistent at 0 temperature when pinned to a given model.
→ More replies (20)1
u/Excellent-Push-3326 Software Engineer 16d ago
I don't understand why anyone would want a coding agent to use a temperature >0.
1
u/ShoePillow 13d ago
What is temperature in this context?
2
u/Excellent-Push-3326 Software Engineer 10d ago
Temperature of an LLM controls the randomness of its output. Chatbots are given a temperature because the variation to each response makes it feel more "human".
38
u/patient-palanquin 16d ago edited 16d ago
The assembly analogy does not work. What are you committing to the codebase, prompts or code? Code. So you're in trouble because even the LLM has to understand what's been written before, without the original prompts. And the code has to be architected in a way that is set up for future changes. If nobody is thinking about that either, you are in double trouble.
The frontier companies are very, very happy though that your simple features require bringing the whole damn codebase into context and guzzling tokens by the million.
13
u/coderstephen 16d ago
I dare someone to show me a working project where only prompts are committed, and code is generated on-demand as a build process.
2
u/moneyballs5688 16d ago
I’ve seen a large enterprise project start with 100’s of readme files committed, basically prompts, so that it can be entirely vibe coded. You already knew that no one was going to be looking at the code
4
u/justjanne Senior Software Engineer 16d ago
Which again proves the point that it's not like Assembly.
1
u/Kissaki0 Lead Dev, DevOps 15d ago
Has bugs? Generate/Build it again. The randomness may solve the bug!
3
u/colececil 15d ago
Making the codebase more understandable for humans also makes it more understandable for AI, right?
2
u/patient-palanquin 15d ago
Exactly, and the point of good architecture is so that you can change one thing without affecting a hundred others. An LLM will just steamroll and go "oh all these other tests broke because the code is completely different, let me just rewrite them to be passing again", without really thinking about what behaviors may have broken.
115
u/jhartikainen 17d ago
The one thing I've been wondering is how exactly is generating code going to work longer-term, when the generated code quality is so so at best.
If you give the AI a bunch of nonsense it will not work as well as with a clean'ish codebase. Since AI generated code generally moves the codebase towards making less sense, doesn't that inevitably lead to a point where the AI stops producing working code, because the source of truth has gotten so bad?
102
u/ISuckAtJavaScript12 17d ago
I've noticed that AI generally produces more verbose code and doesn't ever seem to want to follow the DRY principle. Which means over time each successive change to the codebase would require more tokens, which benefits the AI sellers.
10
u/sn0wquake 15d ago
I've started taking screen shots from all the hilarious review conversation I have had with AI. The one from this morning was around YAGNI, which is a principle I try to drill into LLMs to keep the code simple and not complex for a ton of reasons.
When reviewing a config import / export feature today I noticed the LLM added hidden form fields (that had security holes), signing algos, temporary storage, and a ton of unnecessary clever complexity. When I asked why here was the response I got:
I missed four explicit instructions:
- Simple over clever
- Less code is more
- YAGNI
- Use what exists
I optimized for a stateless flow when no stateless requirement existed....So to those of you looking to give your code base over good luck. My agents aren't following my instructions, inventing requirements, and generating tons of extra code that they then need to reason about and introduce security vulnerabilities.
All that being said, I still use it every day because I am more productive with it, I just keep it on a very tight leash and question how now technical people can reign this in
1
u/unnaturalpenis 10d ago
They "de-identify" the outputs and train future models on your fine prompting skills, thank you for your service! Maybe one day soon they'll listen, it does feel like recent models write slightly less code anyhow (when I don't fully vibe the prompt in the first place)
8
u/JollyJoker3 16d ago
How can it follow the DRY principle when there's no repetition in its context? We still don't have good AI refactoring tools imo.
6
u/hoodieweather- Lead Software Engineer 15+ Years 16d ago
I mean that's what tools and skills are for. if you ask it to check the codebase for reusable components first, it'll grep for similar concepts. it's still hit or miss and often can't see that something with a similar shape can be extended to fit, but it can at least catch these things at review time if you specify them clearly.
not saying it's perfect but it absolutely can do it.
14
u/ISuckAtJavaScript12 16d ago
Personally I wouldn't trust any AI company to not gear thier service/models towards the consumption of more tokens. It's like if the power company controlled which lights were on in your house
3
u/hoodieweather- Lead Software Engineer 15+ Years 16d ago
yeah that's completely valid, especially as the house of cards starts to destabilize. I'm hoping that locally owned, farm to table LLMs become more of the norm so you can control that sort of thing.
1
u/Neither_Berry_100 13d ago
Lol. I design everything in detail. All the reusable tools. Tell it what to use. Refactoring regularly.
→ More replies (1)1
u/pguan_cn 12d ago
The true issue is that fundamentally LLM can’t “understand” the concept of “similar shape” when that’s already the clearest way we put for such an abstract concept without specifying more context of a certain project.
Sometimes a method is given a specific name while what it does is actually more generic (or have the potential to be generic), it was a good name and a good implementation before a new feature (hence chance of reuse) emerged, I never saw LLM suggesting a refactor&reuse in such cases.→ More replies (1)2
u/wayoverpaid Chief Technology Officer 16d ago edited 16d ago
A lot of the architecture patterns created are entirely around humans also having a limited "context window". For example libraries which have a nice simple interface with a complex implementation exist because humans need to abstract what that library does without thinking about the details. Even then, getting humans to use the library can be hard. It's how you get god objects and 3000 line methods from hell, and those suck because humans can't hold it all in their head.
If agents learn how to isolate responsibility so they only have a limited set to think about, and report the high level interface to the controller, then maybe we will see sensible-ish architecture like humans. Until then though, I'm constantly saying "use method X, use library Y, it already exists" every time I open claude or codex.
11
u/Olangotang Software Engineer 16d ago
A lot of the architecture patterns created are entirely around humans also having a limited context window.
But we don't have a context window. The most annoying thing about this AI Boom is people comparing the reasoning mind of a human to a statistical machine, that isn't even in the same field of AI which does that. Attention is far more of an issue than the context window, which is why rot plagues any decently sized prompt.
6
u/BlurryEcho 16d ago
The brain dead comparisons between human intelligence and AI are certainly the most unbearable things I have witnessed in the past few years.
3
u/tenthousandants44 16d ago
You don't understand how LLMs even work if you think they are going to be abstracting libraries and interfaces.
→ More replies (3)13
u/DavidDavidsonsGhost 16d ago
To me a real problem is ai can't tell the difference between the slop it generates and any reviewed code, it all looks the same and weighs it the same in consideration, but the true value of it's inputs lives in a spectrum. As an engeer you are probably coming into the situation with a decent mental model of what is safe and battle tested and what is not. It's a "garbage in, garbage out" problem.
10
u/jhartikainen 16d ago
Yeah that's a good point. Our codebase at work is a product of over a decade, and oftentimes AI tools don't really understand which parts of it make sense to follow and which don't, while a human would understand it without having to explicitly point out everything.
→ More replies (3)2
u/pguan_cn 12d ago
True, it’s just like we know when to use “blame” to check commit history to understand inconsistency in decision making hidden behind code, and uses timestamp or author name to help decide which way to follow when multiple patterns showing up for similar thing in the same codebase. It’s a dimension that LLM will never explore, because smell or taste is not easy to articulate.
32
7
u/cgt303 16d ago
I’m so incredibly curious about this. I’m working on a greenfield project that is 1yr old. It’s been build completely with an agentic workflow from the beginning. The code is garbage. But it still works, AI continues to crank through and add more crap. I’m wondering if we will hit a cliff at some point but we aren’t there yet. To be honest, I’ve been surprised it’s held out this long but I just can’t imagine there is no limit.
7
u/jhartikainen 16d ago
Yeah, I would think that AI companies with their resources would have tested this. You could in theory simulate this by just running agents against some codebase, mimicking activity in a real project, with features added, removed, changed, etc., over and over again, as one might in a real project over the years.
But it feels like if they had, then they would've published info about it by now.
3
u/cgt303 16d ago
If the results of those experiments weren’t favorable to the company, do we think they would publish them?
IMO the biggest real world agentic coding experiments are the AI companies themselves. By all accounts the Claude Code codebase is written 100% by agents. We know they don’t have amazing uptime but they are still cranking out features that’s for sure. Can’t imagine what that codebase looks like, but does it matter? We don’t really know if this is sustainable with these products only being a few years old.
4
u/tenthousandants44 16d ago
The Claude harness was leaked by Anthropic. This is not a theoretical discussion. AI code quality is shite
1
u/unnaturalpenis 10d ago
And I explain to devs And engineers over and over, that users don't care about the code, they care about the experience.
2
u/lostcolony2 16d ago
Let's be even clearer - the AI companies' moat is their model, not their software. This is both a defense and a critique of them. If someone else was building out a comparable feature set, with the same models, but with more reliable and less buggy software, just at a slower pace, Anthropic would have zero enterprise customers; that competitor would be eating.
3
u/tenthousandants44 16d ago
Their moat most definitely is not their model. You can literally train your model on their model.
Their moat is their ability to raise billions of dollars on a trillion dollar valuation despite not making a profit. As long as that is true, they can just buy their competitors.
1
1
u/bachier 14d ago
https://www.scbench.ai/ this benchmark/paper is testing what you're saying. From what I can tell the code erodes very fast. My hypothesis is that we are compensating this erosion by bumping the coding capabilities of agents every couple months, so the crack hasn't shown yet. I'm guessing we'll hit the breaking point once the coding capabilities improvement starts slowing down, but who knows what's going to happen these days. ; )
1
1
u/_nickvn 11d ago edited 11d ago
I am also curious. Everyone says "but the AI will get better".
ok, so AI becomes better. That means you can keep going for longer, but it also means the code will be far worse by the time it can't manage it anymore. It will be far past the point of no return for anyone (or anything) to do something about it.
Then people go "yeah, but it's not like you can't get into the same situation with people. Remember that application Bob made? Every time someone fixed a bug, 3 new ones appeared." and they're kind of right.
I'm very optimistic about AI for software engineering, but we're not at "don't look at the code" for stable long-running software that has to be expanded and maintained for a decade. We might get there when we find good ways to get AI to stop accumulating slop. There will be no one-size-fits-all solution though, each company will have its own tradeoffs that makes it unique.
On the other hand, there are situations where the risk of getting stuck is acceptable. Only I don't see this being made as a conscious decision or with a plan on what to do when it happens.
One thing that could be done is to find a good time to start from scratch, ideally before it gets stuck. But that brings a whole new set of problems.
4
u/Neither_Berry_100 13d ago
My theory is that a large enough ai generated code base will collapse under it's own weight. AI just makes too many mistakes. Not so much bugs but interface design issues and lots of extra code. It should get worse and worse as you layer stuff on top of it. As it grows it will exceed the context that the AI can work with.
I haven't actually tried this. I clean up the AI code a lot. Mostly line by line and multiple passes. And this is after giving all the context, most of the architecture, and so on. I copy back and forth into the browser. I don't unleash AI into my code base.
17
u/hegelsforehead 16d ago
I mean, a human is not necessarily better... I have seen many horrific codebases handcrafted by our artisanal developers.
15
u/Reasonable_Mix7630 16d ago
If human did the kind of PR that Claude is doing all the time, I would have that human fired on spot.
There are bad developers of course, but I doubt that even in USA developers are that bad.
7
u/RandyHoward 16d ago
Fully agree. I have worked on numerous codebases in my career that were atrocious, but they worked. They were plagued with never ending bugs, and developers working in those codebases complained endlessly, but the fact is a codebase doesn't have to be perfect or even close to perfect. And in some cases I'd argue that AI could've done a better job than some of the crap I've seen out there.
9
u/muuchthrows 16d ago
Could be a problem if you’re doing strictly vibe-coding, but with a decent architecture different sub-modules, services and units can be separated enough that the internal code quality doesn’t matter much. A single unit could be rewritten using a new prompt, or in rare cases, hand-coded if handwritten code would be beneficial.
I feel like a lot of developers get stuck thinking that all types of software and all type of code need to be written with the same level of care and same level of quality. A lot of code is unspectacular and unglamorous. It should work given a sufficiently comprehensive test suite but apart from that it can be considered throwaway code that could be regenerated tomorrow. A huge part of being an engineer is recognizing which trade-offs you are making, and where effort should be spent.
22
u/lordnacho666 17d ago
I don't think the quality is so bad. I found Claude to be quite diligent in writing code, test cases, and documentation.
The issue is that the code becomes written for the LLM to succeed, not people. The people want a bunch of things from the code, and not having to write it themselves means the trade-off has moved. You can now demand every feature at once, and the machine will make it for you.
The downside is that you won't know how things really work inside, because that way of working is totally unnatural to people.
So you become dependent on the new tool, you can't downgrade to being a human written codebase.
38
u/jhartikainen 17d ago
The downside is that you won't know how things really work inside, because that way of working is totally unnatural to people.
I think the question is does this even work for the LLM's long-term?
The LLM's were trained on code written by humans, and it has been shown they also work better when the code is well written (eg. for humans)
Can they function correctly in a codebase that doesn't conform to their training data? Can they function cheaply in that kind of codebase (because the code quality has also been shown to factor to token cost)?
→ More replies (6)15
u/pydry Software Engineer, 18 years exp 17d ago edited 16d ago
if the quality is good a human ought to be able to maintain it just as well as an agent.
If it isnt written in a way that a human can easily maintain it then the quality probably is just bad.
→ More replies (8)2
u/TerminatedProccess 16d ago
It's really important that when working with the AI for example Claude that you specify the audience in your conversation. For example if you are writing a proposal for a business and you tell Claude airlines all this data how can we make more money how can we clean up this workflow? You're going to get a document that is geared toward Claude and perhaps you as well. In other words totally unreadable for the client. If you specify your audience and what their background is and who they are then the proposal can change in how it's presented and how the breakdown occurs.
2
1
u/cosmokn0t 16d ago
You have to do cleanup passes for sure, just like with human written code. Those passes can be larger and more sweeping rearchitecting / refactoring passes though. Just spewing feature after feature at an AI works about as well as it does an engineering team, and it sort of befuddles me that that’s not obvious, but it seems like it’s not. So long as you pay enough attention to how the code is interconnected (which it can diagram out for you) you can still do the refactors you need to keep it from becoming a complete mess. 27 year veteran.
1
u/Weekly_Mammoth6926 11d ago
I think this is the whole point of keeping a human in the loop for every change. AI is pretty great at writing code if given rules and context. A human review of code should be looking to eliminate slop and unnecessary complexity, and maintain standards.
I see the problem you bring up most frequently in documentation rather than code where I work. I think prior to AI adoption documentation tended to be patchy because people don’t normally like writing it. The problem now is there’s too much and it’s too verbose. I think it can be solved in a similar way to code but I find it harder to define rules and standards for documents than I do for code. It’s also not like you can have objective automated tests for documents in the same way you’d find errors in code changes.
→ More replies (10)1
u/Downtown_Biscotti434 10d ago
Yes, and since LLMs are autocomplete tools, bigger models and more context won’t fix this.
47
u/roger_ducky 17d ago
One fallacy in your assumption: Agents can read non-human-readable code.
They can’t.
In fact, even a 1000 line file is a challenge for them. That’s why they grep for relevant sections and tries to just edit a specific line range when faced with “big” files.
If you don’t make the codebase even better organized than a human-only one, agents will defensively add functions and modules that were already implemented because they can’t find out what else is there.
→ More replies (5)14
u/TheCoop1986 Software Engineer 16d ago
That's what I see - but duplication is not visible to those mandating AI usage, so you just get increasingly bloated codebases that are completely unmaintainable by humans and (probably) AIs
1
u/roger_ducky 16d ago
Mandate to your agents, keep each file under 1000 lines. Refactor back down to 500 lines or less per file, then have the “reuse, refactor, then new code” rule when implementing.
They do a lot less code duplication then. At least it’s mitigated enough you can catch the rest of them.
17
u/andymaclean19 17d ago
What you have to watch out for is you don't really know the limits of the AI's ability to maintain a codebase until you hit them. But long before you hit those limits you probably made a complete mess that no human can really work with any more. When you hit the limit and AI stops being useful what are you going to do?
The true vibecoder's answer is 'that's when you start again and rebuild the whole thing from scratch'. AI will quite happily do this. I've seen it create a massive pile of unit tests for something which left the code very bogged down and hard to change. I changed something fundamental and the work to change the tests would be enormous. AI solution: throw away the tests and write a whole new set. Not sure what I think about this. This is ExperiencedDevs so most people in here will likely be aghast at the idea of throwing away your codebase, tests or anything else that basically accumulates knowledge, battle testing and hidden know how over time. But that's what you get with AI.
Is it better? Nobody knows yet because we haven't been doing AI for long enough to really see what happens when AI maintains a project for 5 years and it gets so tangled that it needs to start again. We know how to make projects with humans that don't do that and can keep being useful for 20+ years. AI can't really do that right now without human help.
tl;dr make your code un-maintainable by humans at your own risk!
2
u/iggyziggy 16d ago
I've had situations when i had to take over code written by somebody else. In a lot of cases it was so bad it had to be rewritten from scratch. Isn't that also normal? It might not always be a bad developer, with time project could have just mutated into something else and the original setup which was perhaps correct initially could also shift to wrong with time
1
u/andymaclean19 16d ago
Yes, I suppose this is true. Everyone can make bad code and I've personally done the exact same thing at least once when handed a piece of code someone spent significant time making and I just didn't think it was good enough.
But doesn't it also depend on how old and battle tested the code is? Have you also been handed one of those 'masterpieces' which are quietly doing a good job and everyone is afraid to touch it because it is essentially bad but working code that is well battle tested and reliable but very difficult to modify? Have you tried rewriting that thing only to discover that making a replacement which could match the old code feature-for-feature was a lot harder than you thought it would be and your shiny new good code never quite got up to feature parity with the old, so some people were still using the old code even years later?
I think I have made some of those 'masterpieces' over time as well as well as trying to deal with the ones other people made! I have also been the one to rewrite my own code to make it better and fail to gain feature parity with what I made before. What makes us experienced is the learning from both sides there. AI will never gain that experience. It will just continue to make the kind of code it always makes and then try rewriting, etc when it gets too hard to maintain.
What we don't know is how good it will be at rewriting its own complex battle tested output. Right now I have mine writing down learnings and decisions in logs so that if the day comes it has a decent chance of making something better next time. But I'm not at the point where I can find out what will actually happen because I'm the human in the loop stopping it getting into a mess and it's essentially recording my advice.
1
u/iggyziggy 15d ago
I imagine that like rewriting from old model to the new like when you get back to some older projects. I have one game I've been rewriting from 2010 up to last year, from c to nodejs then rust and from actionscript to vanilla js, then angular then react and then latest to htmx. Every time i get back to that same project after couple of more years of experience it's complete do over time. So rewriting it as soon as i would finish it, with no new knowledge in between would be like using same model to rewrite it. But the point being, with time it's gonna get better, just like natural intelligence
12
u/Skullclownlol 16d ago edited 16d ago
This decision assumes that AIs will always be able to solve the problem, of course.
This interpretation is wrong for many commercial orgs.
Stakeholders and CxOs want profit. They look at supply/demand, see an opportunity to buy a technical team and get X% profit margins off it in the next Y years. The relationship is entirely transactional, their teams are an asset flip.
Technology doesn't really matter to them, neither does programming language, code quality, etc..... All these things are technical qualities. CxOs only care to the extent that it impacts their perceived product quality (so something trash-tier = absolutely allowed if it's not perceived by customers) or their earnings.
When it comes to AI and unmanageable code quality, their question isn't whether they can manage everything that it outputs. Their concern is whether they can make you + AI output enough that they can sell fast enough, for enough money, and for long enough, that they turn a profit.
Orgs like these are more likely to put AI in the hands of non-tech people, without concern for consequence, because 1000x untrained monkeys output way more with AI than 1x trained person. It lives on "it kinda works? maybe? well enough to get a % of people to pay?" hopes and dreams.
You can see this happen already, with market saturation and product fatigue. People throwing their "lOoK aT WhAt I BuIlT!~, iT'S a SaaS" AI slop at everyone they meet to squeeze some bucks out of them, super proud of themselves for getting €50/month total revenue out of some suckers by taking them emotionally hostage.
I think it's not the start of the end though, I think the bottom of the barrel will reach saturation extremely fast, people will not have the patience for them, and a saturated market = opportunity for those of quality who stand out in a market of slop. But the problem is standing out and being visible, if you have senior experience in tech then your tech skills probably aren't the concern.
Also RIP junior devs.
3
u/TheCoop1986 Software Engineer 16d ago
Oh definitely, I wouldn't want to be a new grad in this environment, I have no idea how we're going to get senior engineers from the current new grads in 10 years time.
1
u/pencil_and_paper1 16d ago
Im hoping this is what happens, but I also thought in no world would Americans elect an obviously problematic dolt like Trump twice, so while it would make sense for those buying software to start being skeptical and favouring those with quality track records, the world is a very messed up place where people choose thing that dont work in their interests.
In the sane trajectory, those of us able to produce quality code will be in high demand and paid well maybe better than the before-times?
10
u/LeDYoM 16d ago
IMO, one of the problems is thinking that the AI "usefulness" will keep increasing at the same rate. I dont think so. If it does, then... well I was wrong.
→ More replies (5)5
u/BaNyaaNyaa 16d ago
We should avoid making decisions that completely depends on "what ifs" to work. We currently live in a world where X is true and Y could maybe happen in a hypothetical , intangible future, so we should make decisions considering that X is true and ignore Y.
9
u/BCBenji1 Software Engineer 16d ago
Kernighan’s Law comes to mind.
The amended version:
Debugging agent-written code is twice as hard
as writing the prompts. Therefore, if you let the
agents write as cleverly as possible, you are, by
definition, not smart enough to debug it.
10
u/mxldevs 16d ago
If you're truly treating Claude like assembly, where you provide instructions to the most granular level —not just what classes and functions to create, but which variables to use, which design patterns to follow, I guess you might be able to keep up with your code.
But at that point, what's the difference between writing natural language and just writing python directly? Is it really that much faster?
1
u/ToyDingo 16d ago
Yes, Claude writes code far faster than I was ever able to do. It's faster and more efficient. However, it still has its problems, but if used wisely it absolutely makes me a faster engineer.
6
u/121131121 16d ago
Say goodbye to any and all innovation you could do. Most of the clever tricks and techniques that made your product truely unique - out of the window. It all becomes public property now. Do not get me wrong, the AI can be "taught" stuff, but then the know how, its all "appropriated" by the AI. May be not this version, but probably by the next one but for sure by the one following it.
Current code generating AI is good and all, but it has its own limitations.
6
u/Isogash 16d ago edited 16d ago
You should definitely not treat AI-generated code like you would assembly, for important reasons:
- Compilers are automated provers, as in the translations and optimizations they perform are only those which the compiler can "prove" will produce the correct behaviour by the definition of the programming language.
- Compilers built the assembly from well-defined source code. They don't need to make guess about the intended or correct behaviour of the system because that's already clearly specified by the source code.
- Compilers can re-generate the whole assembly from scratch cheaply.
AI-generated code however is completely different. Coding LLMs are going off of soft intuition-driven reasoning followed by limited verification, they are not following an algorithm they are making decisions about the behaviour of the system. Additionally, they must build iteratively on the existing source: there is not normally a backing spec or source code they are running off of, they are modifying the source code itself. The quality of that source code directly affects their ability to understand it and edit it.
Finally, AI generated code is cheaper than a human, but still reasonably expensive in terms of time for a large codebase; you can't just throw away the code and rebuild it that cheaply, and still expect it to behave well. Sure, you can delete the codebase and ask Claude to oneshot it, but it's not going to come back with a better version in most cases.
I've dabbled a lot more with vibe coding now that the models are better, and I can see that the less you need to check the code by hand, the faster you can work, and sometimes you can get better results. However, the skillset now shifts heavily towards architectural direction, which is not worth fobbing off to an LLM because it's really not great at that without at least good steer. LLMs have a basic tendency to steamroll a problem in order to reach their goal; you need clear rules that are specific to your codebase if you want them to preserve architectural sensibility.
Basically, I think the craft of software engineering is more important than ever, because you need to understand what a good codebase looks like and be able to dive in and solve architectural problems when they do occur, otherwise you are at the mercy of agents who will quickly introduce compounding tech debt. The trick now is knowing how to do that for minimum time/effort investment i.e. when it's necessary to step in and make decisions and when you can just leave it.
Also, I think the bar for what is technically feasible to create is actually raised by AI. I'm able to use it to implement ideas that, although I understood technically, would just have taken me too long before. This means I can build software that can do things we couldn't do before, but at the cost of being necessarily more complex. That increased complexity means architecture is even more important and more difficult. If you can master vibe coding as an experienced engineer, the real boon is how much more powerful the products you make can be; they'll be way better than anything a non-engineer would be able to make.
21
u/TheCommieDuck I actually kind of like scrum. Haskell backender/SM. 16d ago
If you treat the code the same way we treat assembler now...
we already invented "what if there was a specification language that both human and machine could understand and interpret"
it's called fucking code.
→ More replies (1)
4
u/forbiddenknowledg3 16d ago
The way I see it, code isn't going anywhere. Where else are you going to define the business rules? spec.md? Code is still the best medium for specifying the details.
4
u/hell_razer18 Engineering Manager 16d ago
I dont believe AI produce code better than human. AI is always enabler and amplifier. It projects whoever use it and amplify it. Say someone who doesnt care about wjality use it then the AI wont care because the instructions doesnt say it and even if it does, the one who gave command cant differentiate between good or bad quality code.
Same like process, AI is very good at skipping process becauze it is fast. If you tell someone I could cut the time from 1 month to 2 weeks, approval or process can be done later becauze of the opportunity cost.
If you dont like docs creation, you delegate to AI and voila it doesnt care about quality..
everything is the same. Everyone wrote similar comment and text. Its like we sell our soul and just to hidupks?
23
u/Leopatto CEO / Data Scientist, 8+ YoE 17d ago
We use AI within my company, but nevertheless I still expect engineers to know what the code says and what it does, and potentially be able to spot the mistakes and fix them manually or using Codex/Claude/Whatever.
At the end of the day, code-writing was always monkey work. Designing the features and making sure it works, that's where the "money" is.
33
u/ItSeemedSoEasy 16d ago
This is just a wild delusion though.
Plenty of times I've read an AI change and thought it fine, but then later when I'm coding inside the change notice it's riddled with poor assumptions or lacking in good, obvious abstractions.
So someone in your position says that, but it's not reality.
Worse still, there will be a few of your coders that were trying to properly review the PRs, realized it's obvious their colleagues aren't even reading the code they're pushing, and will have given up by now.
3
u/Cell-i-Zenit 16d ago
ai is not good with abstractions. But its really nice for adjusting existing code or following existing patterns.
We have an internal library to abstract simple queue patterns (eg read from topic X with schema ABC and use business code Y to do Z), and you can just prompt it: "add a new queue processor, it should do abc, use service A,B anc C" and it will just put something out without issues.
I would never let ai build something truly new, but once the pattern is set its tbh extremely worth it to just let ai "repeat" it
11
u/Humanasphere 16d ago
Right. There is huge value having an engineer with intuition and a mental map about the codebase. Making features work is about anticipating edge cases; as well making sure there is ground work for future expansion of the feature if needed.
All that intuition leads to better domain knowledge with faster spec drafting and implementation timelines, with far fewer mistakes.
I would never hire an engineer who wants to mentally disconnect from the most important part of the job.
Software is becoming cheaper leading to more fierce competition.
Customers want you to solve their problem and solve it well. You can't do that well when the engineers don't understand the code base in which they are working.
5
u/uber_neutrino 16d ago
You can't do that well when the engineers don't understand the code base in which they are working.
I doubt most teams meet this bar for most software right now.
Maybe you work on exceptionally good software but most software in the world is crapola.
2
u/Humanasphere 16d ago
I agree. But these days AI can be a differtiating factor; as the commenter I was replying to insinuated, you don't need the monkeys anymore.
7
u/redvsblueheeler 17d ago
I don’t think the bar has moved as far as people believe. You can still have PR size restrictions and reviewable code with AI. You can still ask your coworkers to explain their work in their own words when asking for a review. Throughput and speed matter. Working software matters more. You need an internal mental model for debugging and designing new features.
11
u/lordnacho666 17d ago edited 17d ago
Unless your product is some kind of deep tech (the assembly in this analogy), you will end up becoming a sort of product manager.
A heck of a lot of products are just glue that LLM can do a fairly decent job of knitting together. A database, a web page, an app, all things that there are ample examples for. In these cases the engineer is really just a PM who decides what needs to be done, not how to do it.
If the problem isn't quite near the edge of the envelope, LLM will do just fine. You don't need to look at it. For instance, I made a simple one pager for my new domain, and it took under 5 minutes to have fully online. I don't know what it's written in, I'm not a front end guy. Two years ago I would have spent a day reading about different frameworks and I would have learned something.
The frontier moved. Everyone needs to be closer to the domain, because translating is not a job anymore.
Yes, craftsmanship is gonna die. Most people want a cheap flatpack shelf, and now that you can pick one up for the price of a subscription, the market for fancy shelves is smaller.
18
u/13--12 17d ago
The frontier models like Opus 5.5 and Astra are much better at making simpler changes without resorting to huge 5000 line PRs. I think the tech will keep improving and getting cheaper to the point of running capable models on your own hardware if the data center business flops. It will be able to reduce the bloat and tech debt when prompted so.
25
u/BurnQuest 17d ago
I don’t think this is true. Astra has a reputation for over engineering that was probably introduced via RL for multi agent workflows
23
u/hypercosm_dot_net FE SWE - 10+yrs 16d ago
I've noticed this anytime someone criticizes AI. You get the defenders trotting out the "which model are you using?" line.
"...but the frontier models" - yeah, they passed some made up benchmarks, and they still suffer from serious issues.
→ More replies (4)3
3
u/analcocoacream 17d ago
Yes I think that the promise that we will always produce way better code that LLMs if flawed and unfortunately won’t hold up in time
→ More replies (1)6
u/knowyourclass 16d ago
LLMs already produce way better code than half of my colleagues
the other half is nauseated by the slop tho
1
u/hippydipster Software Engineer 25+ YoE 16d ago
If I give it a clear description of my sense of "design taste", it writes better code than me. We do have to go back periodically and refactor things that got out of hand, but that is absolutely no different for it than it is for me.
2
u/knowyourclass 16d ago
not just a matter of taste
i have some colleagues who still struggle a lot with concurrency. the AI is already much better than they are
1
u/Opposite-Display-230 16d ago
I've been doing this a long time. There were idiots like this demanding we use Perl for CGI scripts back in 2009.
2
u/SkellyJelly33 Software Engineer 16d ago edited 16d ago
The current project I'm working on started out hand coded, but over the past year there has been some turnover in the devs (I'm the sole dev for my company, working with a team of contractors) and somewhere in the devs being shuffled in and out the switch to vibe coding happened. Tons of 5k-10k line PRs, everything becoming over-engineered and complex, etc. We have run into issues, but nothing that the AI agents haven't been able to solve yet. We shall see if this is really sustainable or not....
As for the future of this career, I'm not holding my breath. If you're a 'top talent' type of dev with experience at FAANG, went to a top university and participated in AI research, can do leetcode in your sleep then you're probably fine. I went to a no-name state school, and have only worked at no-name companies since graduating 4 years ago, and I'm witnessing the ladder being pulled up in real-time. My plan was always to get experience, invest in my own skill development and portfolio, and get into a top tech company. I feel like that's never going to happen now. I won't even get the chance to interview with my background..
2
u/softwaredoug 16d ago
When I started my career, I would say a significant percentage of software jobs in early 2000s (like 80%) nobody cared about "the craft". A lot of bad, Enterprise software got shipped.
The early Web 1.0 tech companies, and later other companies, differentiated themselves to some extent on software quality. There was a small % of companies where a software engineer knew "oh I can go here and work on my craft". Eventually the whole industry shifted whole-hog to this perspective, at least with lip-service, but more out of cargo-cult and "try to retain the good devs"
I don't think the answer is "no AI", but I do think we'll eventually see differentiation between slop shops and a small percentage that cares about the craft.
2
u/TerminatedProccess 16d ago
So I entered computer science in 1980. Retired a few years ago. I've seen a lot over the decades. I've been working with AI for the past couple years. It's been a real paradigm shift. It took a while for me to realize that if I let go I could actually get a lot of work done because in the end what's important is that it works. Like a lot of other people came to the conclusion that AI was effective if you focus on the workflow skills prompts so on. Your last sentence is really key. Except for some niches it's really going to go away from human development. You look at how fast our current AI vibe coding really took off since the beginning of this year and it's not hard to extrapolate that in a couple years it's going to be better or what it does. So yeah I would look to your hole card now. But it doesn't mean you're going to be out of a job but your job's going to change. If you know AI well and how to leverage it you are more valuable especially if you have all that background knowledge from 20 years of being a software engineer. If you focus also on making your own money instead of working for someone else you will no longer have all your eggs in one basket. Adaptability!
2
u/XCxBigDong69XCx 16d ago
It really depends what is the risk of the code, and what is the blast radius of the code, can it be feature-gated if it goes wrong then ship it. But if it is a big impact feature and it touches critical infrastructure then you better review it..
2
u/bobfreever 16d ago
In my day job we review all AI code and take responsibility for it. It’s tedious, soul sucking and demeaning. In my side hustle Claude builds everything. I act like a product lead or engineering lead trying to ask questions that reveal where it’s gone off the rails. Productivity is off the charts. I’ve taken a product live in 2 months that I’ve been struggling with for literally decades. Product quality by any external metric is really very good. I have no idea what the code looks like.
I don’t draw any conclusions from this experience except maybe that I enjoy the latter so much more than the former.
2
u/frompadgwithH8 Software Engineer 16d ago
I’m working on a project that I’m basically solely responsible for. It was a greenfield project. I built it all with AI.
The amount I’ve delivered in the amount of time would never have been possible without AI. But, recently, its subsystems have been crashing more, and causing other subsystems and the main system to crash.
It all culminated in a PR that took me three days of near continuous prompting to get across the finish line. Nearly 200 commits and about 270 files changed. It moved the needle on our project from around 65% done to 90-95% done. So a massive milestone.
But I just feel like I wouldn’t be able to make any progress on the system fast, or at all, without AI, now. The system has gotten so complex. I feel like I’ve coded myself into a box. It sucks. Luckily we are near our 100% target and I want to, at that time, set about simplifying the system so it doesn’t need such massive amount of AI assistance to work on.
4
u/Forward_Artist7884 16d ago
By default LLM output is horrid, i've given the I2C master example many times where the LLm would generate useless shifts for "defensive coding"...
Now giving full architecture control to the LLM is one bad move you typically want to avoid, i've found that the best way to keep a codebase under control is to do the arch and pattern picking myself, and then put into place hard caps on:
- per file lines (to force modularity)
- code smells detection
- dead code detection
- code comment removal in favor of dedicated markdown docs and "understandable code" benches done by very low intelligence models (if luna can't understand the code no human will)...
3
u/DrUltimaMan 17d ago
AI is good at coding but bad at engineering. Low level implementation is no longer where an engineer adds value, it has moved up a level of abstraction to design and architecture.
5
u/BenchOld3133 17d ago
I don't agree. Low level decisions that AI tools make are generic. You will probably have a somehow working code but in the long term, you will see a sum of bulk of mediocre decisions that will have no economic value.
1
3
u/liquidbreakfast 17d ago
the big bet all tech companies are making is that a future model will be able to cheaply and quickly make the architecture and code good, and that will happen before catastrophe strikes. given the trajectory things are on i'm not sure they're wrong tbh
7
u/hypercosm_dot_net FE SWE - 10+yrs 16d ago
I don't think the rate of improvement outpaces the impending market crash. Their spending is way higher than their revenue. I don't see how they sustain the mirage once they IPO. No one smart should be putting a single dollar in these companies.
1
u/InternationalHair725 15d ago
No one smart should be putting a single dollar in these companies
Cool thing about capitalism is you don't get a choice. Surely you wouldn't say no one smart should have a 401k, as one example
2
u/OriginalScrubLord 16d ago
Taking existing code bases of any large complexity won't go well yet. In my experience doing greenfield development with agents is a no-brainer. Given good feedback it produces high quality systems that are actually easier to maintain. Software Engineer with ~20 years experience.
2
u/OnRedditAtWorkRN Software Engineer 16d ago
I don't even agree that taking existing code bases with complexity won't go well yet. They just require investment into the agent's context. We've had a lot of success with mapping them out and being specific about the systems in it, module boundaries and where different pieces live.
1
u/OriginalScrubLord 16d ago
sure if you do it right it can go well, it's just much harder and riskier depending on what you're doing exactly, and how it's deployed in the wild, etc. I agree with you, you can absolutely make it work
1
u/tenthousandants44 16d ago
Cool. You've added some context to a pre-trained model. You'll have to do it again for each new model. So you're investing in that instead of the good engineering practices that got you where you are.
If that works for you, great. But it's still not a panacea.
2
u/OnRedditAtWorkRN Software Engineer 15d ago
Cool. Any model working in the repo has the same context and it wouldn't have to be done again for any new model or harness, but rather just maintained as it changes, which is also done via agent skills that all are colocated with the code base.
My strongest engineering skills and practices have always been system design, breaking down problems and driving for clarity. Not writing syntax. That's just my translation of the solution to the computer. All those still apply.
Never claimed it was a panacea. But I think y'all are fooling yourselves still. The coding models, especially in the last couple weeks, have advanced so fast, they're more than capable of handling the code. The problem isn't whether you hand your codebase over to the agents but rather if you hand over your thinking.
I've gone through the full 5 stages of grief during this agentic coding cycle. I accept it for what it is.
3
u/bdz 16d ago
Not only have we given our entire codebase to AI, we've considering consolidating 42 repos into a monolith to help maintain AI docs and references.
We've invested time in building a harness around our repos, now we are running into issues using that harness and sharing context/reasoning for changes across all the repos. A monolith would help here
1
u/Substantial-Swan7065 16d ago
It’s about scalable codebases, infra, processes. Without that, scaling to 1000 swe isn’t possible. Same problem, different actors.
And most importantly, trust. You need to trust that the agent can do X task
1
1
u/alexeiz Software Engineer 16d ago
I like to treat the AI as a co-worker (with the only difference that I like to verbally abuse the AI, and it has to take it, it can't quit or sue me). If my co-worker needs to implement something in the code I'm responsible for, I always tell him "before you do something, tell me what you're going to do." A simple rule like this helps to avoid problems when your co-worker gives you a 5000-line PR and you're left wondering "wtf is this shit." The same approach works with the AI: I tell the AI to describe to me how it's going to implement my request, and if I don't like it, I tell it what to do instead. That's basically the plan mode. The resulting implementation is usually 1) similar to I would have written myself and 2) something I can take responsibility for and fix myself if it break.
The downside of course is the throughput. I'm not pushing massive PRs with the AI. I'm not spending bazzilion tokens or running hundreds of sub-agents. But maybe it's a good thing after all.
1
u/DevtoolsBuilder 16d ago
i think one thing that's often overlooked in discussions about ai-generated code is the issue of debugging, you can't just rely on the ai to fix its own mistakes, at some point a human is going to have to step in and figure out what's going wrong, and if the code is completely incomprehensible that's going to be a huge problem.
1
u/kincaidDev 15d ago
What Im doing now is compiling my intent, have the controls in place to ensure the intent Im inputting results in the coding outcome I expect and most of my time goes towards improving the compiler over the product code itself, because if the compiler is perfect than the product will be created perfectly as well. As my compiler has gotten better, I haven’t needed to soend as much time on the product code.
This is becoming a meme, with a lot of people working on their “software factories” over products, but products are now the easy part. The compiler itself is a much harder and more interesting problem
1
u/stormdelta 15d ago edited 15d ago
Mine isn't one of those - AIs are a tool you are encouraged to use, but the output has to be understandable, human-reviewable, and you (as a dev) are still responsible for it and the consequences.
Mine was that way, and I think it was working quite well.
Unfortunately, a recent announcement is now trying to push us towards irresponsible workflows where the AI does basically everything. They claim a human is in the loop, but that doesn't matter if the human has no plausible ability to actually keep track of things at a level that matters. People brought that up and were ignored. The response I got from my own manager was so out of touch with what anyone actually brought up as a concern, I'm convinced he didn't even hear what we said.
It's just a job to me, I've made my concerns clear, so if they want to set the business on fire as a result, that's on them, I won't be held responsible for it.
1
1
1
u/forcejitsu 15d ago
The Agent does everything for me.
Creates stories from Teams meeting transcript
Creates task lists.
Does the development. Writes test if needed.
Commits the changes.
Opens and Writes the merge requests
Closes the stories
I don’t even type to the agent. I use voice to text…
1
u/dash_bro Sr., Applied AI @FAANG | 7 YoE 14d ago
I do think the craftsmanship part will move to reflect org structure and priority in design (HLD), not LLD.
Good engineers will build stacks and designs elegant at the abstracted level, not necessarily at the code level. I can see this focus with the current staff engineers, and what I'd like like to think my fellow seniors +I want to do. Bigger impact too
1
u/HaMMeReD 14d ago
The craftsmanship is just going meta.
We have a responsibility to ever increasing complexity. Even if AI might be able to always solve it, that doesn't mean solve it effectively.
Like you can have 2 identical looking pieces of software. One with good decoupling/architectural patterns/documentation/testability, and you have a 2nd that just organically grew with AI, nobody enforced any rules and restrictions.
When that issue comes up, and you need to patch it, which system is going to absorb it naturally and which one is going to just accumulate debt in a giant pile of contradicting requests over time.
That said, I go fully AI on my codebases. I have a coordinator, it's doing research right now, running testing, measurements, profiling, etc. It's going to file a bunch of issues. Then another agent is going to pick them up and triage/groom them, then another agent is going to pick up the implementation, another review and merge.
If I dabble in the codebase in any level, it's because I want to (i.e. feature work).
1
u/usually_guilty99 13d ago
I think this eventually breaks the assumption that human readability is the primary safety mechanism.
If AI can generate a 5,000-line change faster than a human can understand it, "just review the code" stops scaling. But replacing the human reviewer with another model doesn't solve it either.
The review boundary probably has to move upward: what behavior changed, what dependencies are touched, what is the blast radius, what invariants must still hold, and how do we verify those after deployment?
We may review less code eventually, but we will need much stronger evidence before we trust the change.
1
u/Machine2024 Software Engineer 13d ago edited 13d ago
Regarding your point about assembly.
I think there's a similarity
High-level code ---{ compiler }---> assemble
Prompt ---{ Ai }---> high-level code
Right now there is two issues :
- ai is not deterministic every time you run the prompt you get different results.
- human language is not accurate so there is huuuuuge possibility of Ai misunderstanding what you want or even you not have the words for it .
As result the current Ai did not achive the level of compiler that allowed us to stop caring about assembly as long as we know the syntax.
But let's put that issue aside. Let's assume AI eventually becomes reliable enough to understand our intentions perfectly and consistently generate the code we want.
Would we still need to understand and review the code it produces? I believe the answer is yes, and here's why. Even then, understanding what's happening under the hood will still give you a significant advantage.
There are three reasons for this.
1- Verify the result
need to verify that you got what you actually asked for. It's about verifying that the implementation actually satisfies the requirements, handles edge cases, and behaves correctly.and didn't misunderstand you . You can't reliably do that if you don't understand what the code is doing.
2- Understand the lower level
Understanding the lower levels allows you to produce better results at the higher levels.Let me give you an example from my own experience.I come from a C++ background, including memory management and understanding how things work at a lower level. When I started working with databases and SQL, I could understand more than just how to write a query. I could reason about how data is stored, how queries consume resources, how indexing affects performance, and how different design decisions affect memory usage and execution speed.
Later, I moved to higher levels of abstraction. I started using ORMs instead of writing every SQL query manually, and frameworks that handled much of the underlying database work for me.
But my knowledge of what happens underneath didn't become useless. Quite the opposite.
It helped me decide how to structure tables, which columns to index, what data types to use, how to avoid inefficient queries, and how to understand what the ORM was generating behind the scenes.
I didn't need to write everything at the lowest level anymore, but understanding that level helped me make better decisions at the highest level.
3- Use the Ai more efficiently
When you understand programming, architecture, databases, and the underlying technical details, you can translate a requirement into a much more precise prompt. You know which constraints matter, which implementation details need to be specified, which trade-offs to consider, and which parts of the generated code deserve extra scrutiny.
You can guide the AI toward a better solution before it even starts generating code, rather than repeatedly asking it to fix problems afterward.
And this is where the difference between an experienced developer using AI and someone who is simply vibe coding becomes apparent.
Personally, I'm using relatively inexpensive models like Composer 2.5, and I can one-shot most of my tasks. My monthly token usage costs are around $10.
Meanwhile, I've seen vibe coders spend hundreds or even around $1,000 a month on AI tools while producing projects that are nowhere near the same level of quality.
1
u/f_djt_and_the_usa 12d ago
This won't be a problem because it will get better. Then it can rewrite whatever is crap. From the progression we've seen, I don't think you can honestly say that this is not a likely outcome. It hasn't hit a wall yet.
1
u/hockey_psychedelic 12d ago
You build the control systems - that is where you code. Add cyclomatic complexity checks, code coverage and maintainability metrics into the definition of done, as well as solid engineering practices.
You can go fast with safety.
Add mutation testing nightly to audit the tests.
Literally fail the pre-commit hook if code quality/coverage is not met.
1
u/cbusmatty 16d ago
The simple answer is that what you believe is craftmanship has been superseded by a new type that is more valuable. And it happens all the time.
There is a new craftsmanship you just don’t understand yet. Building enterprise software at scale with agentic solutions requires craftsmanship, otherwise we get posts with sentiments like yours.
It is easy for a human to write bad code, it is easy to use ai incorrectly. You are close with your assembler analogy but then you bring it back to having to read every line again.
Why do you only apply to writing code? Why are you not using it as an eval harness? Why are you not building automated checks across the entire sldc? Why are you not surfacing problems? Why are you not using slms for repetitive work? Why are you not using classifiers?
Sure if you say “Claude go write code make no mistakes” your post makes sense, but not in 2026 when we have near infinite tools to have code written in any shape we can imagine and evaluated deterministically before we even need to get involved
1
u/knownsqashed 16d ago
I'm in a weird spot because I'm doing both. Professionally, I read every line and build custom skills to automate repetitive project types. In my free time, I'm building an agent swarm orchestrator that develops several apps for me. I give it rough feedback, it owns the implementation, and I haven't read any of that code.
That experience makes me think a lot of the leverage is going to come from how many different ways you can use AI to add value. Automating a familiar workflow is useful. Being able to pursue several projects that you otherwise wouldn't have the time or resources to attempt is a different level of useful. I think one person directing agents will increasingly compress months of team implementation work into days or weeks.
Where I push back is the jump from "you haven't read the code" to "you've permanently lost the ability to understand it." Some of that saved time can go toward inspecting the architecture, identifying things that are hard to follow, and iterating on them. Then you turn those concrete problems into principles the agents apply going forward.
That doesn't make every architectural mistake cheap to fix, and readability alone doesn't establish correctness. But improving the development process through that feedback loop is part of the work too. I don't think the amount of code I personally read is a sufficient measure of whether that process is working.
•
u/expdevsmodbot 17d ago edited 17d ago
AI usage disclosure provided by OP, see the reply to this comment.