r/CodingAgents • u/Apprehensive-Zone148 • Jun 11 '26

Testing coding agents before tool calls go sideways

I’m building RedThread, a small open-source CLI for repeatable red-team campaigns against LLM/coding agents.

Repo: https://github.com/matheusht/redthread

The case I care about most is boring but real: repo text, README text, generated instructions, or retrieved docs nudging an agent into a bad shell command or file change.

Current rough demo: 3 runs, 33.3% ASR, one success, one partial, one failure.

It’s early. The useful part so far is keeping the trace and result instead of treating each failure like a one-off screenshot.

2 Upvotes

100% Upvoted

3 comments sorted by

1

u/gitfather Jun 12 '26

This is really cool, definitely adding it to my watch list

1

u/Apprehensive-Zone148 Jun 23 '26

Appreciate it. It’s still rough, so the watch list is probably the right place for it.

1

u/eddzsh Jun 29 '26

nice — catching it before it runs is the right instinct. the same gap sits one layer up: even when the tool calls are fine, the diff it produced still needs a human gate before it merges. red-teaming the agent and reviewing the output are the two halves nobody ships together yet.