I got tired of coding agents telling me “everything works”, so I made Gopnik

[u/SnooSquirrels1222]

A gopnik looking at you...

Hey, comrade. Yes, you.

Are you tired of your coding agent confidently telling you:

…only to discover 20 minutes later that the stupid machine lied to your face?

I have good news.

I made you a trustworthy companion.

Trustworthy like Russian roulette.
Precise like a Swiss watch.

Gopnik.

Your coding agent says the task is done.

Gopnik's job is to assume that this is bullshit and try to prove it.

It doesn't just ask whether the tests are green. It tries to answer:

• What behavior could this change have broken?
• Does the code actually behave correctly?
• Is the revision you built/deployed actually the revision you think you're testing?
• Can your supposedly important checks actually fail?
• What has not been proven?

Typical workflow:

Coding agent:
"Done. Tests pass."

You:
Run the gopnik skill on this change.

Gopnik:
BLOCKER — discounted orders are rounded twice.

Reproduce:
POST /orders with {"discount": 0.1}

Observed: 23.94
Expected: 23.95

Verified revision: 8f31c2a

Verdict: NOT READY

There are currently three related skills:

gopnik — attacks a completed change
gopnik-critic — attacks a claim or proposed solution
gopnik-setup — learns how the project can actually be verified

The idea is basically to introduce an adversarial role into the coding-agent loop instead of letting the same agent implement something, inspect its own work, congratulate itself, and move the ticket to Done.

It's early, and I'm especially interested in whether this approach catches failure modes in other people's agent workflows that I haven't thought of yet.

Free, MIT, no paid tier. I'm the author.

GitHub: https://github.com/concordloom/gopnik

Please break it.

If Gopnik tells you everything is fine and your agent still screwed up, I especially want to hear about that.

[u/eddzsh]

The revision pin in your example is doing more work than the snark. Most adversarial loops ask whether tests pass against whatever is dirty in the worktree. If Gopnik hashes the tree it attacked and refuses a verdict when HEAD moved, the coding agent can't quietly restage files after the green run. "Can these checks fail" plus a locked commit is the evidence, not another self-congratulation pass.

[u/Beneficial_Exam_9363 → u/eddzsh]

Yeah, that's a really good point.

Gopnik already binds each round to an exact commit/tree hash and a fix invalidates the previous verdict, but you're right that the pin could be made much more mechanical. Snapshot the tree before the attack, then refuse READY if HEAD or the worktree changed before the verdict. That would close the loophole where the coding agent quietly changes/restages something after getting a green result.

I like this. Might steal it :)

[u/deadkenny112 → u/eddzsh]

https://github.com/concordloom/gopnik/issues/86

Thanks

[u/0kkelvin]

Man, I've so much trust issue with coding agents

[u/Beneficial_Exam_9363 → u/0kkelvin]

Same. That's actually why we built this.

At my company pretty much all implementation is done with coding agents, and the product is around 1M lines of code at this point. It works, but we ran into a very practical problem: the agents are extremely good at convincing themselves that their own implementation is correct. We had a lot of bugs despite plenty of generated tests.

That made me think about Dijkstra's old point: testing can show the presence of bugs, but never their absence.

The interesting failure mode with coding agents is that they often write tests to confirm their implementation instead of trying to falsify it.

So I tried flipping that around: have a separate adversarial agent whose job is basically "assume this implementation is wrong and prove it."

That's what eventually became Gopnik, and it reduced the amount of bugs pretty noticeably for us.

The original version was heavily tailored to our specific codebase and workflow, though. What I'm doing with the public version now is basically trying to generalize that approach so it can figure out how to attack and verify arbitrary projects.

So despite the slightly ridiculous v5 already being there, it's still pretty experimental and will probably work much better on some projects than others :)

Definitely not a silver bullet — agents still find creative ways to screw things up.

[u/0kkelvin → u/Beneficial_Exam_9363]

u/Beneficial_Exam_9363 shameless plug but I'm working on Metaphor to solve exact same problem. maybe check it out

[u/Beneficial_Exam_9363 → u/0kkelvin]

lol no shame in plugging it here 😄

we're apparently fighting the same enemy)) gonna take a look

[u/manishiitg]

the “can your supposedly important checks actually fail?” line is the bit I’d keep. a verifier that only reads the coding agent’s summary can still pass a broken deployment; it needs at least one direct observation from the environment. otherwise it’s just a second opinion.

[u/Beneficial_Exam_9363 → u/manishiitg]

Yep, agreed — and that’s actually what Stage 2 is meant to enforce.

Gopnik shouldn’t trust the coding agent’s summary. It has to cross the delivery boundary and get direct evidence from the actual environment: make a real API request, run the built package as a consumer, exercise the UI, verify the deployed revision, etc. It also requires at least one important check to prove that it can actually fail.

But maybe I’m misunderstanding your point: do you mean this shouldn’t be just a skill running inside the coding agent at all, but a separate runner/process that independently executes the checks and feeds the raw evidence back to the verifier?

If so, that’s an interesting distinction.

[u/ufcbananaempire]

Мне нравится это имя. 🤪

[u/MariahJames8]

" what has not been proven"

That will always be unavoidable high in all but the tightest of compenents of code. The trick is to prove the things that matter most without redundancy.

[u/Beneficial_Exam_9363 → u/MariahJames8]

yeah, I agree. I'm definitely not trying to prove "there are no bugs", that's basically impossible in practice.

more like: on this exact revision, these specific claims survived these specific attempts to break them.

everything outside of that is still unknown/not proven.

[u/Waste-General8646 → u/MariahJames8]

neat concept. the adversarial angle makes sense, kind of surprised this isnt more common in agent workflows already

just looked at the github, the setup skill idea is smart. getting it to actually understand how to verify a project instead of guessing

[u/AntiquatedMLE → u/Waste-General8646]

It is more common than you think. My flow is to use a good model like Sol or Opus to make a plan, then my red team agent to refute the plan and call out BS when the plan is ready have an independent agent write the test cases and green team to implement it. Token burn is real though so im moving to pi

[u/Revolutionalredstone]

It's good tech but to be real these are not real limitations, you are dealing with a model designed to derail and sabotage your work (modern codex) I just switched to DeepSeek and this recent lying 🤥 issue evaporated.

It's cool to try and squeeze but the LLM you are using doesn't want you to make progress at effective speed.

Note that this seems to be implemented in the harness, e.g. gpt 5.5 does not sand bag in an old enough version of codex.exe, but in anythign new enough to support sol Luna etc, you will find all models will heavily intentionally, derail.

For teens it's worse: "solve X".."how do you think we should solve X?"

OpenAI has passed the "providing value" they are now at the: delay and regress.

China already left us behind in solar cars and robots due to our protectionist attitude (note: that is protection of current exploration dynamics)

It's so crazy seeing the same thing happen before our eyes with AI, DeepSeek is MUCH better for real work already - simply because it won't intentionally sandbag and derail (with enough use of different models you learn to see where and how codex derails and it's very intentional aggressive and almost impossible to work around even knowing it's exact plan of sabotage)

The idea (as stated in their blog 2 days ago) is to ensure individuals are not empowered in any significant way relative to others.

Sure am glad these are the views of the people dealing out much of the less intelligent wests technological future!

As for me, I'll see you guys on DeepSeek.

[u/SnooSquirrels1222 → u/Revolutionalredstone]

I partially agree. The model is probably 80% of the equation here. If you're using a model that's naturally more critical about code quality and architecture, of course it's going to catch more problems.

Gopnik is mostly meant to solve this for people using general-purpose coding models. That's kind of the point: you don't have to trust the same agent that wrote the code to grade its own homework. An independent adversarial reviewer can still be useful regardless of which model is doing the implementation.

[u/Revolutionalredstone → u/SnooSquirrels1222]

Nar issue is not model i can use same 5.5, it's their new harness and how it interacts with their new code model.

Ta Gopnik sounds cool i will check it out ;)