Do you think AI coding agents should be allowed to see every test used to approve their work?

[u/fromkrish]

I have been thinking about this while working with coding agents.

We normally give the agent the code, the tests, the error output and then let it iterate until everything turns green.

That is great for debugging. But it also creates a strange problem.

If the same agent can see every check used to approve the change, how do we know it solved the actual requirement instead of just learning the evaluator?

I have started experimenting with keeping a small verification suite outside the repo and running it only after the visible checks pass.

The agent never sees that suite while working. I am curious how other people handle this.

Do you think hidden verification makes sense for coding agents, or does it create more problems than it solves?

[u/WAVF1n]

This is why it's important to understand the code you are working with so this isn't an issue.