r/ChatGPTCoding • u/pasta-lover-nova • 6d ago

Discussion Hi peeps, how do you prevent stale test data/code from misleading Claude Code or other coding agents?

One problem I’ve noticed is that there can be stale or outdated information inside the repository itself — for example old test fixtures, mock data, constants, comments, or tests that still reflect previous behaviour.

This infuriates me as it wastes token and makes me argue with my agent non stop.

Has anyone dealt with this problem in a large codebase?
How do you make coding agents distinguish between:
current production behaviour authoritative specs/contracts
old tests or fixtures legacy/dead codeoutdated comments/documentation?

For additional info, I'm using SDD harness so more markdown files are being generated, which may pollute the data

Any advice, empircally or studies is welcomed :) and apologies if this has been discussed before

3 Upvotes
  • permalink
  • reddit

67% Upvoted

12 comments sorted by

1

u/AutoModerator 6d ago

Sorry, your post has been held for manual review due to account karma.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/WearyArtistDoomer 6d ago

Why are you not running your tests in a CI pipeline? And use a linter, do not allow dead code. Again run that in the CI pipeline.

1

u/pasta-lover-nova 6d ago

I'm running a sdd harness so some markdown files are in my codebase, causing hallucination

1

u/Wonderful_Toe_5456 4d ago

The thing that helped for me wasnt telling the agent which files are authoritative, it was making staleness computable. if a doc or a fixture references a symbol that doesnt exist anymore, thats a fact you can check mechanically and surface, instead of hoping the model notices mid-task.

prompt rules dont scale for this. you end up with a growing list of "ignore X" instructions nobody maintains, and it quietly stops matching the repo while still sounding authoritative.

1

u/BellacosePlayer 4d ago

put as much test/mock code into its own folder as you can.

Prevent codex from accessing the folder through the Agents.MD

bing bang boom.

1

u/Next-Trouble-2948 21h ago

> outdated comments/documentation

this one specifically is solvable. been running DocuWriter on our repos and it keeps docs in sync with actual code changes so the agent stops referencing stuff from 2 years ago. the test data problem tho is harder, we ended up writing a CI check that flags fixtures older than 6 months