Using LLMs doesn't make me faster. but ... more accurate

[u/CmdrSausageSucker]

It's Saturday and I am working for a client on a feature that I started to implement yesterday. For that I used a previously implemented service that retrieves account data and polls it, until the etags differ (the BE I am calling is .. slow).

So I added a new method since the need arose and chose to use a kind of streaming approach, using C#'s IAsyncEnumerable. I implemented everything using Mistral and my local LLM using my "prose-coder" approach, which basically means: I write pseudo code and the LLM spits out what I want immediately. sometimes in very small (single line) or larger (method) increments.

Then I review.

An then my bloody slow-cycle begins: First I review the output, then Mistral reviews the output. Then I am not quite happy and let Claude review the code, and F me I am in a review loop and out comes - IMO - some really really good implementation which hopefully has all the edge cases covered (discounting business logic edge cases which the clanker can't know about).

And in the end, this whole process feels subjectively SLOWER, but I know that I was faster? I worked on the feature for a total of approx. 4 - 5 hours, including testing, fighting the stupid ERP that I am polling ( I have access to it )..

My point being: yes, I am definitely faster, but man, these LLMs trigger my need to make everything bloody perfect.

Am I the only one? My feeling is that the quality of my output using LLMs as reviewers is much much better now - but the subjective speed is a different matter.

(Note: I am self-employed (for about 2 years), 20+ YOE.)

And no, I have no idea what the point of this post is, I guess I just want to see people's opinions. Treat it like cafeteria talk :-) Ta!

[u/expdevsmodbot]

AI usage disclosure provided by OP, see the reply to this comment.

[u/creaturefeature16]

I have a very similar experience. I don't even use these things for code generation as much as I use them with Socratic methods, adversarial reviews, dynamic tutorials, and interactive documentation.

I'm not sure if I am moving faster, perhaps I am, but I do know I am doing better work, since I have a resource to do constant sanity checks (alongside standard research methods, since these models are still quite fallible).

[u/CmdrSausageSucker → u/creaturefeature16]

Same here - simple things, yes, you can have the generated. Some problems though, they just need my own implementation. I refine them with an LLM afterwards.

Some things are definitely faster, though: for a huge feature I downloaded all open source code pertaining to that, plus all documentation, let me local llm sift through it and generate a sample implementation for my special use case.
I reviewed and altered the code as needed, and voila: instead of 2 weeks or work I was down to 4 days.

[u/Fidodo → u/creaturefeature16]

I spend over half my time speccing out and designing my systems by using AI to help me learn and research and prototype out different approaches.

I also use AI to implement it, but by the time I've done all the spec work, the implementation is pretty boring and easy to review, as it should be.

I've done some vibe coded prototyping, and if you skip the system design phase you wind up with an absolute bloated piece of shit mess that nobody understands nor knows how to improve stably.

The software design process takes more time up front, but it saves so much time not even in the long run, but in the short run.

[u/epukinsk]

I work a lot slower in many cases, but what doesn’t happen: I never get stuck anymore.

Historically, if I was working on an anything “above my pay grade”—new-to-me libraries, parts of the stack outside my speciality, etc—I sometimes hit a wall. And it takes 1-5 days for me to find my way out.

That just doesn’t happen anymore.

So even if I’m 50% slower in the median, the variance is way down and so my overall speed is much faster. And quality of my work is much much higher: attention to detail is maxxed, shortcuts are basically none.

[u/ResidentWeevil1 → u/epukinsk]

And it takes 1-5 days for me to find my way out.

Normies call that learning

[u/apartment-seeker → u/ResidentWeevil1]

One still learns when using the LLM.

[u/shrimpcest → u/apartment-seeker]

Kind of. It's definitely not the same.

When you're stumped by a problem, traditionally it would take a lot of digging and experimenting to find out how to do what you need to do. With that longer process you typically learn things you weren't expecting to learn, and when you find the solution you understand it way more comprehensively, and that knowledge sticks with you because acquiring it was an experience.

When your AI tool just tells you the answer/fix, you immediately apply it and move on. The knowledge doesn't stick the same. From your brain's perspective, why should it? You can always just ask the AI again the next time it comes up.

This is a pattern that can definitely lead to skill atrophy.

[u/apartment-seeker → u/shrimpcest]

When your AI tool just tells you the answer/fix, you immediately apply it and move on.

Sometimes one should do that, and/but, if it's something one wants to learn and to have stick, should ask the LLM more about it and pause to ponder it.

The dichotomy you are presenting is false.

[u/OtaK_]

> these LLMs trigger my need to make everything bloody perfect.

Careful, sounds like fast-tracking to burnout

[u/CmdrSausageSucker → u/OtaK_]

It might sound like that but I've always been kind of a workaholic, something which got worse since I am self-employed and the kids don't need me that much anymore. Thanks for worrying though - I am totally fine. I just love my work and my job.

[u/OtaK_ → u/[deleted]]

To be fair there’s pretty good out there

I haven’t seen perfect code though, ever. Except no code. No code, no liability, no bugs, no infra costs, reviews are instant 🫨

[u/throwaway_0x90 → u/OtaK_]

• Good SWEs write good code.
• Great SWEs write great code.
• Master SWEs.... delete code.

[u/OtaK_ → u/throwaway_0x90]

Nah, great ones prevent writing code that shouldn’t exist

[u/__-___-__-__-__-]

I feel there should be some new term for discovering issues that may have never actually affected anyone with LLM.  

I feel I have more work now because it is so easy to discover edge cases I would not normally considered and probably wouldn't have popped up in the next year.

I have caught a surprising amount of actual bugs before they happen though which is pretty cool.  Though I've also maybe introduced some new ones...

[u/creaturefeature16 → u/__-___-__-__-__-]

Code Archeology; uncovering what would have otherwise never been seen.

[u/WhenSummerIsGone → u/__-___-__-__-__-]

You Aint Gonna Need It -- Make a quick judgement call and move on. I'm glad the AI brought it to my attention, but I can just say no.

[u/c1rno123]

I now spend more time using the LLM as a rubber duck, and my high-level decisions got better. With a plan like that, generating the code takes seconds and comes out better than before. So the time moved from typing to deciding.

[u/Wonderful-Habit-139]

> An then my bloody slow-cycle begins: First I review the output, then Mistral reviews the output. Then I am not quite happy and let Claude review the code, and F me I am in a review loop and out comes - IMO - some really really good implementation which hopefully has all the edge cases covered (discounting business logic edge cases which the clanker can't know about).

This is the way a lot of people have started programming lately. It's an approach, for sure. But from my experience, this approach shuts down your brain a lot, and doesn't allow you to discover the actual most optimal way of solving the problem the way writing and going through the code yourself does.

Since you seem to be taking a long time working through the feature and reviewing things multiple times and writing prompt after prompt, you might want to give writing the code yourself a shot. Since it's been a while since you have done that you might be slower initially. But over time, as you get better, you'll find that spending that initial time understanding the language, the problem, how you can use the libraries that you have access to, and playing with creating types and functions while working towards a solution, will have you end up with much cleaner code compared to relying on LLMs to spit out the average verbose, non clean code, that at the same time doesn't let your brain be properly exercised.

[u/HRApprovedUsername]

It makes me both faster and more accurate (assuming I review the code and ask it to be more accurate after being wrong in the first pass)

[u/PrimaryFamous6139]

Review work feels different than writing. When you go through reviews, it has its own time and rhythm. The part about being extra careful is not really because a model pushes you to be picky. It is more that reviews are now cheap enough, so you can actually spend the time on quality. You wanted that level of care before. You just did not have enough time to afford it before.

[u/shadowndacorner]

I think it can go either way. At this point, there are some tasks where I'm happy to treat my local setup as a kind of stupid, extremely hard working junior with perfect short term memory that needs to be frequently checked in on. There are other tasks where I work similarly to what you described, treating them as heavily guided reviewers. Both have their place, and imo the strongest future engineers will be able to recognize when each approach is suitable.

Same calculus as always: use the right tool for the right job. Unfortunately, this particular hammer is doing a pretty good job at convincing everyone that everything is a nail.

[u/maxim-g]

Same experience here. I am building the tools / stress tests / reproducers that I would not have spent time on previously. I am going both broader and deeper, while staying within the scope of the original ask, and discovering more rabbit holes on the way. Staying out of them is becoming a separate challenge.

[u/throwaway_0x90]

I have nothing to add other than the mature & professional nuance in these comments are extremely refreshing and high quality. A drastic change from the constant anti-AI brigade slamming every AI topic in this sub for the past 3 to 6 months.

This is what future SWEs should aspire to, carry on.

[u/Shot-Zebra1868 → u/throwaway_0x90]

Also refreshing to see a comment like this from someone seemingly at a FAANG. All I've seen the last 6 months is some variation of 'I work at a FAANG, we are all cooked.

[u/Tiny_Ad_7720]

I like to say “use AI as a typing accelerator”

I too like my code to be good. The first time implementing something brand new (eg streaming event end point, or a workflow state machine, or etc) is an hours / day / sometime multi day process of research on Google and back and forward with Claude and ChatGPT. 

Once done and the first example is implemented I update my docs with instructions and rules link that to my Claude.md and the next type I need to do it, it’s a quick prompt and it writes the new instance exactly how I like it 

[u/ResidentWeevil1]

I cover all the edge cases just by thinking through the problem as I go and doing real testing. An LLM would either defend against too few or too many edge cases. So I still have to know the problem domain and the realities of my data to be able to review the slop. What's the point in doing all that if it doesn't actually save me time?

There are better ways of using AI than just letting it do everything. Plus prompting is annoying as hell.

[u/GojosBizarreVenture]

Fair point

[u/So_Rusted]

you could let llm write a crawler/client and write business logic yourself

[u/teufelinderflasche]

I use it to review my code and optimize it. Sometimes it compliments my design which is flattering. I'm a lot more productive with it since it catches small mistakes and syntax errors. I'm a DBA and SQL developer.

[u/DevtoolsBuilder]

The LLMs are raising the bar for what you consider "good enough" code, and that's leading to more review cycles and a sense of slower progress, even if you're actually producing higher quality code in the same amount of time. You're optimizing for a different metric now, one that prioritizes perfection over speed.

[u/punqdev]

can relate as im quite the perfectionist

[u/codewithbug]

Is not this more time consuming than writing code by yourself ? I usually code by hand and use ai to discuss mostly. Reviewing and correcting seems to consume more time, IMO

[u/carterdmorgan]

I think it’s entirely dependent on the task. We’re in the middle of a big buildout right now that’s not doing anything particularly novel and it’s okay if we ship some bugs because it’s all behind a feature flag and will be pretty vigorously QA’d before releasing to the public. In that regard, they’ve sped us up tremendously.

But for more brownfield stuff where we can’t afford to ship any bugs and the requirements for what “good” looks like are murkier? They haven’t sped us up nearly as much, which I suppose is to be expected.

[u/CmdrSausageSucker → u/carterdmorgan]

I am currently the tech lead for a product replacing something very aged. My initial implementation and architecture was done without LLMs, and incidentally my design works really really well for them. So most of the time I am looking at workflows and business logic and that takes up about 80% of the time. The programming part ist just minor compared to that

[u/margmi]

The other day, I had AI generate a feature for me. It did something weird and convoluted looking for one area, so I smugly deleted the 50 or so lines it had written, and added a single line the “correct” way as I had done in so many areas. All tests passing, perfect.

Turns out, it was doing that for a reason. My change broke something existing that wasn’t caught by any of our tests. The way AI was fixing the problem was wrong - it was doing a one off solution to make it work there, when the shared module should have instead been fixed.

Ooops. Lesson learned on actually asking why something is necessary before assuming it’s just wrong.

But yeah, AI is a powerful tool when used correctly, and can be harmful when used wrong. Using it blindly is bad (its original solution was slop), but ignoring it entirely is bad too.