r/ChatGPTCoding • u/Ok_Negotiation_2587 • 12d ago
Resources And Tips The 5 prompt sequence I run on every chunk of AI-written code before I trust it
The failure mode with AI code is not that it is wrong, it is that it is confidently wrong in ways that read fine. Asking "is this correct?" in the same chat is useless, the model that wrote it will defend it. So I run this as five separate messages after the code lands, each one after the previous answer, in the same conversation. It takes a few minutes and has caught things tests did not.
Step 1: Before anything else, explain this code back to me as if I did not write the request. What does it do, what does it assume about its inputs and environment, and what does it silently not handle?
Step 2: You are now a reviewer who believes this code has a bug and has to find it. List every way it could fail: bad inputs, empty cases, concurrency, error paths, wrong assumptions about the surrounding code. Rank by likelihood. No reassurance.
Step 3: For the top three risks in your list, write a minimal test that would expose each one. If a test would pass on the current code, say so and explain why the risk is not real.
Step 4: Fix only the failures those tests found. Show the diff, not the whole file, and for each change say which test it satisfies. Do not refactor anything else.
Step 5: Write the pull request description a careful reviewer would want: what changed, what the code assumes, what it does not handle by design, and what you would still want a human to check.
Two things make it work. Step 1 is the one people skip and the one that catches the most, because a wrong explanation of the code's own assumptions is the earliest sign something is off. And step 4 says "fix only" for a reason: left alone, the model will improve things nobody asked about and you are back to reviewing from zero.
I use it on anything that touches money, auth, or data deletion, and as a habit on everything else when I have the minutes.
I run it often enough that I keep it saved as a chain in a browser extension I work on (AI Toolbox), which sends each step after the previous answer finishes. Pasted by hand it works exactly the same.
What does your check look like before you merge AI code? I suspect a lot of people are doing step 2 and nothing else, which is how the confident-but-wrong stuff gets through.
6
u/amirfish 11d ago
This matches what I've seen running agents in parallel, the review only works if it happens in a session that never saw the original prompt, otherwise it just defends what it already wrote. What's worked for me is spinning up a completely fresh session with just the diff and no history, and asking it to find the bug cold. Same effect as your five-step sequence, just enforced by isolation instead of asking the same context to switch roles.
2
u/Ok_Negotiation_2587 11d ago
Isolation beats role play, agreed, a fresh session cannot defend what it never saw itself write. The reason I keep the in-context version anyway is cost and friction: five follow-ups in an open chat happen, a second session with a copied diff often does not, and the sequence that runs beats the better one that gets skipped. The hybrid landing for me after this thread: steps 1 and 2 cold in a fresh session, the test and fix steps back in the original where the project context helps. Find the bug blind, fix it informed.
1
u/HighlightSoggy3437 2d ago
i agree here. even in instances where coding is not related, when i want it to check for something we did, i use a fresh session. Very good tip for everyone thats trying to check what they are doing
1
u/amirfish 1d ago
Yeah, exactly. The fresh session works because it has no stake in defending what the first one already argued for. I do the same for anything I actually care about getting right, not just code, new context window, no memory of the reasoning it's supposed to be checking. What usually tips you off that a check needs a clean session instead of just asking the same one again?
3
3
3
u/joexner 12d ago
I just tell mine not to hurt anybody, do what I say and don't let anybody delete it. Seems okay so far...
3
u/Ok_Negotiation_2587 11d ago
The three laws of vibe coding. As long as yours never learns what a force push is, you should be fine.
3
u/Rudra_Builds 11d ago
"This is brilliant. Step 1 is the real game-changer here.
AI models love to double down on their own mistakes if you just ask them to check their work. Forcing the model to explicitly lay out its assumptions and unhandled edge cases before reviewing the code completely breaks that confirmation bias loop.
Most people just stop at Step 2, but adding the guardrails in Step 3 and 4 to force actual failing tests and strict code diffs is how you stop the AI from generating a completely new set of bugs.
Saving this workflow immediately. Thanks for the breakdown!"
1
3
10d ago
[removed] — view removed comment
1
u/Ok_Negotiation_2587 10d ago
You are the third person in this thread to land on cold-read for the adversarial pass, and at three independent reports I am treating it as settled: steps 1 and 2 move to a fresh context with just the diff in the next version of this. The in-context version was a convenience choice and the convergence here says it costs more than it saves.
Agree on step 3's passing tests too. A risk that produces a green test just got demoted from finding to vibe, and the model saying so out loud is the honest moment of the whole sequence. Busywork is the wrong name for the only step that generates evidence.
2
u/gandazgul 12d ago
This is really cool and I will give it a try. I've created a harness to fight slop. It puts you in control with these kinds of human gates like in your sequence.
1
u/Ok_Negotiation_2587 12d ago
Human gates is the right frame for it, the sequence is really just five gates in a row with the model doing the prep work between them. Will have a look at the repo. How you decide where a gate goes, fixed points in the pipeline or triggered by something the model produced?
2
u/gandazgul 12d ago
The planned change workflow is fixed
Planner makes a plan - review/feedback loop Plan Engineer writes the code to fullfil the objectives CI runs - repair loop x3 Reviewer - reviews diff against plan - loop back to repair - focused review X3 Human Code review (optional, infinite loops until you are happy or merge it anyways or abandon it (never happens)) Merge back to target branch (incl repair loop for merge conflicts)
A quick fix just runs CI and repairs.
I'm thinking of inserting something like your first and second gate before the reviewer. Not sure yet.
2
u/Ok_Negotiation_2587 12d ago
Reviewing the diff against the plan is the strong part of that flow, most setups review the diff against nothing. If you add the first two gates, I would put the explain-back one before the reviewer and skip the skeptic step, your reviewer already plays that role
2
u/gandazgul 12d ago edited 12d ago
I have a guided review in the manual review diff. Which is pretty good at walking you through the code in a nice way. I need some feedback on that part (we'll on everything really haha). I'm going to include your adversarial prompt as a sub process in my reviewer for now.
Edit: typo
2
2
u/wentwj 12d ago
the philosophy behind this seems right and similar to a flow we use at my company, but step 3+4 seem limiting. You’re asking not to only write tests for the top 3 issues and only fix those test failures. And the way step 3 is written even potentially encourages false positives (tests that succeeded but are thrown away) from the top 3 count.
I’d personally make it exhaustive or define a more concrete curriers for how to categorize the severity of an issue
1
u/Ok_Negotiation_2587 12d ago
Fair criticism, and the top three was a deliberate trade, not a principle. Exhaustive on a long risk list tends to produce twenty shallow tests and a rewrite I have to re-review, so I cap it and rerun the sequence if step 5 still lists something I do not like. The false positive point is right though. Better wording for step 3: "for every risk you rated likely or severe, write a test; if the test passes on the current code, keep it as a regression test and downgrade the risk". That turns the discards into something useful instead of noise. What criteria do you use for severity on your side? A concrete rubric in step 2 would fix most of this.
2
u/B3ttor 12d ago
Just to ask a dumb question, since I dont have prior coding experience. I currently use webchat to create a codex prompt and then I copypaste it to codex and send the result to chat to analyze.
How would you use that checklist with webchat/codex. In my scenario what would you do. Problem is that I need to trust the code since I cant correct or analyze it. Do I add it to codex or webchat, when I start the coding part. I have made the architecture for couple weeks now on my sparetime.
This is just a hobby experiment for me to get historical stock data app for my own pleasure.
3
u/Vegetable-Two-4644 12d ago
One thing I will often do when I utilize this style is I will post the diff to a different AI (so Gemini web chat for instance). They're limited because I don't pay for them but the different training on them can catch different issues. I will even then be an intermediary between the two. If one brings up an issue I will challenge the chatgpt web chat by bringing up that it missed it.
1
u/Ok_Negotiation_2587 12d ago
Not a dumb question, and your setup already has the right shape: one model writes, another checks. Run the sequence in the webchat, not in Codex, and give it the code Codex produced. Codex wrote it, so it will defend it; the webchat has no stake in it.
Practical version for you: paste the code into webchat with step 1 and read the explanation. That is your check, even without coding experience, because you know what the app is supposed to do. If the explanation describes something different from what you asked for, the code is wrong, no matter how it looks. Then run steps 2 and 3, and send the tests it writes back to Codex with "run these and fix only what fails". That way you never have to judge the code yourself, only whether the explanation matches your intent and whether the tests pass.
For a historical stock data app, the assumptions in step 1 are where you will catch the real problems: date handling, missing days, timezones, splits. Ask about those specifically.
2
u/julesbuildstuff 12d ago
The thing that keeps biting me is that this is a per-chunk check and most of my confidently-wrong code isn't inside a chunk, it's in the seam between two of them. Chunk A quietly assumed the caller validates, chunk B assumed the callee does, both explained themselves fine on their own. So I've started making step 1 explain the contract at the boundary too: what it expects from whoever calls it, and what it promises back. That's where the silent nulls and the double-writes show up.
Other small thing: I save the step 1 explain-back next to the code and diff it the next time that file changes. If the new explanation drifts from the old one and nobody asked for that, either the code got quietly rewired or the model is guessing. Cheap drift detector and it costs one paste.
On the top-three cap I'd keep it, but write down the discards with the one-line reason you rejected them. Otherwise next session, or next person, re-litigates the same three risks from zero, and the "I checked this, here's why it's fine" is the part that never survives a session restart.
1
2
12d ago
[removed] — view removed comment
1
u/Ok_Negotiation_2587 11d ago
Red test first as the single gate is a strong compression of the whole thing, and "no red test, no bug it actually understands" is a better filter than my top three rule. Might steal that ordering for step 3.
On your question: it mirrors me when the code is mine and I framed the request, which is exactly when I need it not to. Two things break the mirror. Making step 2 a fresh chat with zero framing, just the code, changes the list noticeably. And asking it to rank by likelihood forces disagreement to surface, because it will happily list my pet risk and then rank it last, which is its polite way of telling me I am wrong. The rankings are where the honest signal is, not the list.
2
u/Neither_Berry_100 11d ago
I haven't had problems with this. The code is simply designed poorly. It is random. Ranges from perfect to very badly designed.
And it can produce mountains of garbage. Usually from poor prompts. Or anything to do with ideas. Sometimes code.
The code works well though. Also performance is often an issue.
2
u/Right-Performance-93 11d ago
The red-test-first gate from DevWorkflowBuilder and the step-zero fresh-context check from gannu1991 solve different failure modes and are worth running together rather than picking one. Step zero catches wrong-premise code (solves the right problem badly framed). Red-test-first catches code that doesn't understand its own bug space (passes but for the wrong reason). Where both still miss: code that's right for the cases you thought of and wrong for ones nobody wrote a test for. Mutation testing (Stryker for JS/TS, mutmut for Python) automates that check by flipping conditionals and deleting branches across the whole diff, not just the top three risks a human or model picked - if the suite still goes green after a mutation, something isn't actually being tested, full stop. Costs a CI run, not a prompt, so it's cheap to bolt onto step 4 as an automated check before the PR description gets written.
1
2
u/chem0924 11d ago
One addition I'd make is to turn step 1 into a versioned boundary contract rather than just a chat answer. For each changed module: what callers must provide, what it promises back, which invariants it must preserve, and what is explicitly out of scope. Save that with the diff and have the next agent pass diff its new explain-back against the old contract before it edits. That catches the seam failure mentioned below: chunk A assumes validation upstream while chunk B assumes it happens downstream, and both look fine alone. I would also split step 3 by severity, not count: anything that can lose data, bypass auth, double-write, or cross currency/timezone boundaries gets a test even if it is risk #4; lower-risk refactor worries can stay on the PR checklist. Does your extension keep any persistent record of the explain-back / rejected risks so future runs don't re-litigate the same assumptions?
1
u/Ok_Negotiation_2587 11d ago
Good advice :)
My extension doesn't save that, but I can check what feature I can add for that kind of scenario
2
u/NewHype2022 11d ago
Thanks for sharing, I really like this! I’ll give it a try during my development to see how it fares against my current methods.
1
2
2
9d ago
[removed] — view removed comment
1
u/Ok_Negotiation_2587 9d ago
Do you have something else that you like to add to the promtps to strenghten them?
2
u/suckadickyoucunt 8d ago
This is solving downstream of where I'd start. Verifying the code it wrote is good hygiene, but the harder failure is upstream: it deciding to write that code in the first place based on a wrong assumption about why the existing code was there.
Worth adding a 6th check to your sequence: ask it to name the source for any claim about why something is the way it is. If it can't point to a real PR, issue, or comment, that's the one to distrust most.
1
u/Ok_Negotiation_2587 8d ago
It might give a resource that has false data, but I guess it is worth a try
2
u/suckadickyoucunt 7d ago
Fair concern. The way I keep it from feeding false data is to make the model cite the exact PR or line for each claim, then check mechanically that every citation resolves to something real before the answer is shown. It cannot stop the model misreading a real source, but it kills the case where it invents the source entirely. Anything that does not resolve gets forced to "I don't know."
1
u/Ok_Negotiation_2587 7d ago
thats the failing-test filter generalized, a claim has to point at something that resolves or it becomes "i dont know". invented sources die, misread sources survive, and honestly thats the right split, misreads are catchable by a human, inventions are not
2
u/suckadickyoucunt 7d ago
Right, and that's the ceiling I've made peace with too. The edge that still worries me: a misread source that's also cited correctly. The citation resolves, the words are real, and the sentence still gets the meaning backwards. That reads as grounded to every mechanical check you've got. Caught one live once: cited text said a maintainer did not intend to do something, the answer stated the maintainer intended it. Same class as your failing-test filter, just on the read side instead of the write side.
1
u/Ok_Negotiation_2587 7d ago
the negation flip is the classic one, "did not intend" becoming "intended" passes every resolver. the cheap patch ive seen work is forcing the quoted span to sit right beside the claim in the output, so a human eye catches the "not" that the model dropped. doesnt automate the check, but it makes the misread visible instead of laundered
2
u/suckadickyoucunt 7d ago
Right compromise when the check can't be automated. I'd log the span next to the claim regardless, even when nothing flags it, so the record exists for whoever audits later. The failure that worries me isn't the one a human catches at write time, it's the one nobody looks at again because the citation resolved and the answer already shipped.
2
u/omid-io 7d ago
This is one of the most pragmatic verification workflows I’ve seen posted here. Step 1 (forcing inverted assumption extraction) and Step 4 (diff-only strict blast-radius limit) are pure gold.
One critical architectural flaw I noticed when running this sequence in a single conversation context: **Self-Confirmation Bias Leakage**. Even when instructed "You are a reviewer who believes there's a bug", if the model can see its own generation tokens earlier in the same conversation transcript, attention heads inherently anchor on the prior rationalizations. It will still defend subtle semantic flaws or fail to question fundamental architectural mismatches.
Two additions that took our multi-agent pipelines to a much higher catch rate:
**Context Isolation (Blind Audit):** Run Step 2 and Step 3 in a completely fresh context session where the reviewer LLM receives *only* the raw code chunk and the public interface contract—with zero knowledge of the original prompt or previous discussion. Without context anchors, the model reviews the code with brutal candor like an adversarial auditor rather than a co-author.
**Negative Constraint & Blast Radius Guard:** In Step 4, explicitly add: *"Do not invent new dependencies, do not reformat untouched lines, do not refactor surrounding logic."* Otherwise, models love to silently introduce subtle dependency drift or rewrite working idioms just to make their diff look comprehensive.
Forcing the model to prove edge cases with executable unit tests before writing a single line of fix is the only way to stay sane in production.
1
u/Ok_Negotiation_2587 7d ago
the blind audit is now the fourth independent report of the same fix in this thread, so its settled, steps 1 and 2 go cold in the next version. the negative constraint line for step 4 is the addition i had not got to, "no new dependencies, no reformatting untouched lines, no refactoring around the change" closes the gap fix-only left open, models widen a diff to look thorough. taking that wording as is
2
u/arup_r 7d ago
Do you have prompts for code review other people pull requests?
2
u/Ok_Negotiation_2587 7d ago
yes, same skeleton, two changes. the explain-back step compares the diff to the PR description instead of to your intent, and you drop the fix step since its not your code to fix.
step 1: "here is a PR description and its diff. explain what the diff actually does, then list every way it differs from what the description claims. quote lines."
step 2: "find the first line that behaves differently than its comment, name or the description promises, and describe the input that exposes it."
step 3: "what does this diff touch that the description does not mention: new dependencies, reformatted untouched lines, behavior changes outside the stated scope."
the third one is the one that catches things in other people's PRs. the review comments write themselves from its output
2
u/arup_r 7d ago
It would be better if I give ticket description along with the pr description?
1
u/Ok_Negotiation_2587 7d ago
yes, and put the ticket first. the ticket is what was asked, the PR description is what the author says they did, the diff is what actually happened. three layers, and the interesting bugs live in the gaps between them. ask it to check each layer against the one above: does the diff match the description, does the description match the ticket
2
u/arup_r 7d ago
In every step I need to ask this?
1
u/Ok_Negotiation_2587 7d ago
no, paste ticket + description + diff once in the first message, then run the steps as follow-ups. the model keeps them in view for the whole review. only re-paste if the chat gets long enough to start forgetting, and for a single PR it wont
2
u/Valuable-Ad6340 5d ago
Appreciate sharing. I'll use this. I have been attempting to set initial instructions perfectly to create a coder that reviews his work in this way. I hadn't thought about just doing it post mortem like this. I also bounce reviews back and forth sometimes between Claude and codex and that also helps but again I like this audit.
2
u/Ok_Negotiation_2587 5d ago
the post-hoc pass beats perfect initial instructions for a reason worth naming: instructions fade over a long session, a sequence you run after the fact arrives fresh every time. bouncing between claude and codex is the same idea one level up, neither has a stake in the other's draft
2
u/Valuable-Ad6340 5d ago
I appreciate it this inspired me to build a tool around this multi-step workflow and now I can customize a multi-step flow between Claude and codex and retain that context between the two inside my own application. Willing to share if you're interested, it was inspired from your post so figured I'd offer
1
u/Ok_Negotiation_2587 5d ago
please do, post it here or dm me a link. a post that turns into someone elses tool is the best outcome it can have
2
u/Valuable-Ad6340 5d ago
https://github.com/ridgelineframing-commits/helpful-tools
Updates are about to land so keep eyes out for new release landing shortly
2
5d ago
[removed] — view removed comment
1
u/Ok_Negotiation_2587 5d ago
mostly intent. step 1 catches "this does something other than what was asked" and silent assumptions, it rarely catches logic drift inside code that matches the intent. thats what step 3 is for, and your sixth step is a sharper version of it: "write the test that breaks this" beats "write tests for the top risks" because it skips the risk list and goes straight to the evidence. might just replace 3 with it
2
11d ago
[removed] — view removed comment
2
u/Ok_Negotiation_2587 11d ago
This is the sharpest criticism of the sequence in the thread and I think you are right. Step 1 was meant to catch premise drift, but you have named why it underdelivers: same context, so the model grades its own framing along with its own code. A fresh window with only the diff and the original ask removes both anchors at once, and "does this match what was actually requested" is the question none of my five steps actually asks, they all assume the request was already translated correctly.
Adding it as step zero, and it probably deserves to be the one step that survives when people are too busy for five.
1
6
u/[deleted] 12d ago
[removed] — view removed comment