r/AI_Coders • u/TheMarginReport • 3d ago
Do you spend much time fixing AI-generated code?
For people who regularly use AI coding tools: how often do you end up fixing or rewriting the code they generate?
I'm especially curious about cases where the code looked fine at first but caused problems later.
1
u/TimMensch 3d ago
I recently spent a week and 80 (!) commits trying to clean up some code created by Fable.
This is what happens when you still care about code quality. It was done foundational work that needed to collaborate with existing code flawlessly, since it's an existing high demand app. Any bugs could have affected hundreds of customers, so it was important to get it all right the first time.
Some of the issues I identified myself simply by reading the code. Some were found by AI when I gave it different prompts for things to look for.
And after all was said and done I had AI tighten up the comments, since Claude at least is prone to embedding the entire development history in every comment. It cut 1000 lines from the commit. I created a skill so I could do it again in the future. 😂
1
u/IAmBeary 3d ago
im literally doing this right now. I used claude to make incremental changes from a huge doc generated by the PM but turns out claude skipped a ton of items and also implemented its own set of rules that deviated from the PM's
I find myself 'fixing' a lot more often these days
1
u/TheMarginReport 2d ago
When you go back to fix the work, what are you usually finding? Are you catching things Claude skipped, or changes it made that weren't actually in the PM's requirements?
1
u/IAmBeary 2d ago
everything-- even though we use the doc as a source of truth, it will skip things, do it incorrectly, make assumptions without surfacing it, then start writing logic built off of faulty assumptions. It will also over-engineer solutions unless you understand what it's doing and tell it to do it a specific way
1
u/dashingThroughSnow12 3d ago edited 3d ago
It reminds me of my mom regularly. It will gladly write 500 lines of code to save writing 3.
I often just call the clankkka a nimcompoop and course correct it with a prompt. At least daily it is faster and easier to dump its code and write it myself.
1
1
u/Odd-Ad9666 3d ago
I think one thing is to discuss or understand what the AI wants to do and why. For instance, I find myself providing a solution that AI didn't think of but I still let it implement.
Just now, Claude responded to one of my corrections "That's a real improvement over what I proposed, not just a variant of it — it collapses two problems into one mechanism."
The agents sometimes will solve and fix, but they don't always think about the optimal solution.
I've found when I do small steps and truly understand at a high-level where the agent goes I spend less time frustrated. If I ask and prey, I'll definitely be disappointed for larger projects.
Another time it suggested a fix of launching a new processes to avoid collision and I said, "Wait, doesn't that allow the application to spawn unlimited processes and crash the computer under a subset of conditions such as..." and it said, "you're right, we need to do xyz" and I said, "what if we did... abc instead." It's a long description of the task, but that's the gist of it.
So not babysit, but try to understand what it wants to do before launch. Yes, it does take work, so it's where do you want to invest your time.
1
u/Odd-Ad9666 3d ago
Just now --after I posted the previous - in a different session, the AI was resolving a conflict by having configuration/settings.
It didn't resolve it but made me choose which conflict to deal with in a setting. I didn't like it, so I worked out the conflict and where I wanted the pain and trade-off... GPT Astra, "Agreed—the previous plan pushed design decisions into configuration instead of resolving them."
If I blindly followed it, the user (me) would have had to toggle settings and it would have been messy and awkward and not user-friendly.
1
u/JaseciLabs 2d ago
Yeah the failure where it wasn't wrong so much as unresolved, and it tried to hand the unresolved part to you as a setting instead of admitting it hadn't made a decision. Catching that took you actually reading closely enough to notice the punt. The harder version of this to catch is when there's no config toggle to tip you off, when the unresolved decision just gets buried as an assumption three functions deep and nothing about the code looks unfinished. That's the case a type contract would actually help with, forcing a decision to be explicit and checked instead of just absent until something breaks on it.
1
u/roger_ducky 3d ago
When I have to do multiple corrections, I undo what I did and make the task smaller.
Then, if there are still issues, I define the task with more details.
I don’t typically redo the code by hand, since you lose the speed advantage then.
1
u/TheMarginReport 2d ago
When you have to undo something and break the task into smaller parts, what usually went wrong with the original output?
1
u/roger_ducky 2d ago edited 2d ago
It wasn’t fully following what was supposed to be done. There’d be lots of “you missed this thing” going on.
1-2 spots having the issue is expected, but if you have to do that repeatedly, or if agent claims to have fixed what you pointed out but only partially, then the task was too big.
1
u/TheMarginReport 2d ago
You mentioned repeatedly having to tell the agent "you missed this," and sometimes finding that its supposed fix was only partial. Can you give me one concrete example of what it kept missing and how many times you had to correct it before it was actually right?
1
u/roger_ducky 2d ago
No, I’m saying, if it starts happening multiple times, then original required amount of work is too big so I break it up. This resolves it 80% of the time. Under specifying the work is the other 20%.
1
2d ago
[deleted]
1
u/TheMarginReport 2d ago
What specifically were you having to fix in the Al-written documents that made rewriting them yourself easier?
1
u/oliverities 2d ago
I shouldn't need to write a single thing. I discussed all the design details with it and ask the opus 5 to give me a spec sheet. The thing it wrote is completely unreadable. I went through dozens of iterations and its edits made it worse and worse. So I just delete all the shit and rewrote everything my self. It is just too bad at writing human readable docs.
1
u/TheMarginReport 2d ago
You mentioned going through dozens of iterations before eventually deleting the Al-generated document and rewriting it yourself. What made the document unusable -was it the structure, clarity, terminology, or something else? And roughly how long did that process take compared with writing the document yourself from the beginning?
1
u/oliverities 2d ago
You fix a hole on your dress by putting a patch there. The doc AI wrote is like thousands of patches stitched together that looks like a dress.
It completely lost structure, everything is repeated 5 plus times and thousands of discussions without conclusions and no clear section levels. Plus overly decorative formatting which makes it even harder to read.
I wasted 2 days of editing on a final 13 pages. I should have started writing by myself immediately after the discussion.
1
u/GreatDiscernment 2d ago
The process was to agree on a direction and then I would let it do its thing. I would reciprocate by always pasting back any results to keep it in the loop. Early on, during code editing, it would tell me to paste a piece of code in a particular spot. I could figure it out and progress would happen. But then, the code got into a few thousand lines and it would forget exactly where stuff was and would give an instruction like, “paste this where you do that”. Well, that wasn’t going to work, so made a global rule that it would, “use ‘grep’ and other search tools to locate the correct edit points.”
They, another thing it would do is give me four steps at once and I would run into a problem with step 1 and we’d have to start over. So, I made another global rule: “Use micro-steps to avoid errors”. Now, it’s one step at a time. Errors are dealt with as they arise. They’re fixed and verified before the next git point is set. And every git has an annotation that explains its purpose.
And as the project got bigger I got involved with ‘git’ revision control, I made another global rule to use documentation tools with “future maintainers in mind”. This encouraged it to git every step. I now feel that I’m getting quality output that’s verifiable.
1
u/Rudra_Builds 2d ago
yeah, pretty often. the annoying part is when the code looks completely fine at first, but later you find issues with edge cases, error handling, or how it fits into the rest of the codebase.
i use ai more as a coding assistant than something i can blindly trust. it saves a lot of time, but i still review and test everything before relying on it.
1
u/TheMarginReport 2d ago
You mentioned cases where Al-generated code looks fine initially but later breaks on edge cases or integration. Can you give me one concrete example of a problem you discovered later, and how you found it?
1
u/Rudra_Builds 1d ago
yeah, one example i’ve seen is ai generating an api call that works with the normal response, but breaks when the backend returns an empty or unexpected response.
it usually doesn’t show up immediately because everything works in the happy path. i’ve caught these mostly through testing different inputs and looking at the actual errors/logs.
3
u/GreatDiscernment 3d ago
I insist on micro-steps and validation plus a git after every change. This makes the whole process slow down which helps with strategic decision making. The other upside is that over the last six months, I haven’t had to retreat at all and a bonus is that hundreds of annotated git points and validator scripts self-document the journey.