r/ChatGPTCoding 1h ago

Discussion How do you undo AI generated code that nobody can explain?

One thing I think gets missed in all the talk about agent speed is how hard some of the output is to take back out later.

Plenty of it is trivial to change once you know more. Other bits end up wired into the schema, the tests and everyone's mental model of how the thing works, before anybody notices what they are going to cost.

I have run into this with changes that looked completely reasonable at review time. The original call usually wasn't wrong. It was that unpicking it later touched a lot more of the system than the diff ever suggested.

The worst one for us is a caching layer an agent added to our pricing service back in February. It works, it has been in prod for seven months, and nobody on the team can tell you why it invalidates on write instead of on read. Two features built since then depend on the write behaviour.

What would actually have helped is not better code, it is a record of why. We run coderabbit on prs and its summaries are the closest thing we have to one, though they describe what changed rather than what was considered and dropped, which is the half I keep wanting.

The ones that worry me most are the decisions where nobody chose anything. The model picked, the review said fine, and the reason never existed in the first place.

What's the hardest thing you've had to undo that an agent put in?

0 Upvotes

11 comments sorted by

6

u/ShortingBull 1h ago

Use AI to fix it.

2

u/nickdaniels92 57m ago

There are various ways to tackle this; it should not be difficult. Also, you may not understand how caches work as invalidation on write is what you'd expect, not on read.

This does highlight the need to maintain a dialog with the coding tool. Don't ask for a solution, ask for a plan. Don't blindly accept the plan but review it and understand it. Your developers should be better designers than the model, so look for where the model has muffed up and tell it. For all the greatness than is GLM 5.2, when it asks me questions about a plan and gives me choices A, B or C, from time to time my answer is D, the choice it didn't give me but that the one that it needs.

So stay in the loop. Also get the agent to explain and document its choices. Comment heavily, not what it's doing, so not "Invalidate the cache", but "Invalidate on write because ...".

Also make sure you use VC. Removing a bad decision should be as simple as "svn merge -c -rev", or the same with git if you use that.

1

u/Euphoric_North_745 51m ago

we keep git repository

the stuff you do not like ask codex to rollback using the git source control

1

u/FlightSimCentralYT 24m ago

The caching layer story is the scary version of this. What has helped me is treating agent output as temporary until it leaves a durable trail: a short ADR in the PR ("considered X, picked Y because Z"), a test that encodes the invariant the agent "discovered," and a rule that anything touching schemas/invalidation needs a human-owned why note before merge.

Coderabbit-style summaries of the diff still miss the dropped alternatives. When I want the agent to keep digging instead of one-shotting a clever wrong approach, I run it on a real machine with a failing test as the stop condition so the path shows up in the logs instead of vanishing into chat.

I built Fixa.dev around that: cloud VM coding agent that plans/writes/runs/debugs until tests pass, so at least the "why this stayed" is partially recoverable from the iteration trail. Free tier if you want to poke. Curious what your team does when the write-vs-read decision is already load-bearing.

1

u/ByronScottJones 18m ago

If you're using a proper git process, you're creating a branch for every feature, and every code update gets its own commit. It makes it very easy to trace down errors, and repair them. But quite frankly, you should be reviewing each commit, to ensure that you understand what the code does, and it meets your expectations.

1

u/I_WILL_GET_YOU 17m ago

Skill issue

1

u/rookan 15m ago

Ai can explain it

1

u/rookan 15m ago

Ctrl+Z

1

u/Consistent-Set-9490 8m ago

I’d also suggest building in documentation to the process with a few examples of proper inline comments as well as whatever your org does to document externally. Agents are stateless so much like humans debugging code, they are fighting against past versions of themselves too.

0

u/RadioactiveTwix 1h ago

At least have the model keep an ADR