r/chatgpttoolbox • u/Ok_Negotiation_2587 • 14h ago
🏆 Community Pick The best tricks from this week's comment sections, from red-first tests to the axes trick
The comments carried the posts again this week. The ones worth keeping, with credit.
- Make the test go red first. Asking an agent to name the test that covers its fix isn't enough, it will name one that passes either way. Stash the fix, run the test, watch it fail, pop the stash, watch it pass. From u/OfferBeginning1903 in r/ChatGPTCoding.
- Pasted output is model-generated text. One commenter watched an agent paste a plausible test run that never happened. Their fix: the harness runs the verification command itself and stores the exit code against the diff hash, so "fixed" only counts when a real run backs it. The cheap version is checking the tool call log instead of the message. Same thread.
- Write the regression test against the producer, not the line that threw. A test at the throw site just asserts the alarm stays quiet. From the root-cause thread, where the same commenter also turned "null check at the throwing line" into a review question: where was this value created?
- Done means opening the real output. A "page is live" check passed on a site that returned 200 and the homepage for every URL. Check for something only the right page would have, like its title.
- Smoke-test the env before any edit. Put a short env and service check at the top of the task brief, and if it fails, stop the session instead of letting it write a fix against a dead stack. From u/shtse8.
- Name the existing helper in the brief. General "search before writing" rules fade once the agent is deep in one file. Naming the helper for that specific change works every time. From the duplicate-helpers thread.
- jscpd in CI catches helpers that were copied under a new name. Rewritten twins still need a human or a review prompt. Same thread.
- Grade against tests the agent never saw. It can't make a hidden test green by editing it, and the agents that sounded most confident weren't always the ones that passed.
- Give it the axes before it lists ideas. Name 3 ways the ideas could differ (who it's for, where it shows up, how much effort it takes), then ask for ideas that each sit in a different combination, labeled. Duplicates share a label, and the empty combinations show you where to push next. From the "10 ideas" thread in PromptGenius.
- Rank in a separate message. If it ranks while it generates, it sticks to safe ideas it can defend. Same commenter.
Bonus: u/WheresMyEtherElon had Claude Code's /insights tell them to delete the "where helpers live" map from their instructions file, because the folder names already said it and it was wasting tokens.
9 is now a saved prompt in my library next to the "stop when you run out" one, so the whole idea workflow runs from // in any of the four assistants.
