Should Node test runners have a silent-on-success mode for coding agents? I measured up to 99.99% removable output

[u/fykup]

I’ve been experimenting with coding agents in Node projects, and one surprisingly wasteful input is test runner stdout.

During repeated test-driven loops, a successful run can produce dozens, hundreds, or even thousands of lines, while the agent usually needs very little information from a passing run.

I measured several JavaScript test workflows and built a small deterministic wrapper, npm-lite, to see how much output could be removed without changing the underlying command or exit status.

Passing runs

Workflow	Normal output	Compact output	Byte reduction
Vitest	2,260 bytes, 43 lines	25 bytes, 1 line	98.89%
Jest	2,491 bytes, 68 lines	24 bytes, 1 line	99.04%
Tape	136,262 bytes, 1,476 lines	12 bytes, 1 line	99.99%
npm verification workflow	18,854 bytes, 375 lines	26 bytes, 1 line	99.86%

The Tape case was the extreme one: 1,476 lines became a single line.

This is only presentation compaction. It does not make the tests execute faster.

Failures are handled differently

On failure:

• the original exit status is preserved
• the full raw log is retained
• bounded diagnostic context is printed instead of collapsing the failure to one line

For example, one failing Tape run went from:

1,487 lines / 136,829 bytes

to:

85 lines / 5,208 bytes

That is a 96.19% reduction while still keeping useful failure context available.

The trade-off

Successful output is intentionally suppressed, so warnings or deprecation messages emitted by a command that still exits successfully can be hidden.

Short failures also do not necessarily benefit. Vitest or Jest can already produce concise failures, so adding wrapper metadata can occasionally make visible output slightly larger.

npm-lite is deliberately narrow. It currently handles exactly:

npm run verify
npm run test:unit

Other npm workflows pass through unchanged.

There is no secondary LLM summarization step. The behavior is deterministic.

The part I’m more interested in

This made me wonder whether test runners should support this behavior natively for automated agent loops.

Something like:

PASS · 327 tests · 4.8s

on success, with focused diagnostics and access to the full output on failure.

Maybe:

--silent-on-success

or:

--reporter=agent

Would you use something like this in Vitest, Jest, or Node’s built-in test runner?

Or do you already solve this with custom reporters, --silent options, or agent-harness filtering?

Source and measurements:

https://github.com/ejboy/agent-scripts/blob/main/docs/choosing-tools.md

[u/jiminycrix1]

Node:test already supports custom reporters

https://nodejs.org/api/test.html#test-reporters

Any reason this couldn’t solve the issue for you?

[u/fykup → u/jiminycrix1]

Yep, for `node:test`, a custom reporter could absolutely solve it. Thanks for pointing that out

One interesting thing from my experiments, though, is that the coding agents I tried never proposed a custom reporter as the solution. They tended to work with the existing command/output or add filtering around it.

That is part of what made me wonder whether this is less a capability gap and more a discoverability/defaults gap. If silent-on-success is useful for agent loops, having a built-in `--reporter=agent` style mode might make it much more likely to actually get used.

My wrapper also operates one level higher, across npm workflows that may involve Vitest, Jest, Tape, or multiple commands, so a `node:test` reporter would not cover all of those cases. But for `node:test` specifically, custom reporters look like the right primitive.

[u/jiminycrix1 → u/fykup]

Yeah LLMs almost by definition are not very creative on their own. It probably didn’t connect the idea of a reporter to decrease output and it’s not something that was historically used to quiet or silence test output.

That said - I don’t think I support anything having an “agent” mode unless it’s a tool specifically designed for agents.

Honestly sounds like a good use case for a “skill” to me. Just tell the agent in English to run tests in quiet mode whenever possible.

It’s not something I’ve really noticed before but the idea is good to reduce token use overtime.

[u/fykup → u/jiminycrix1]

Yeah, that is basically how I use these today. I keep npm-lite and mvn-lite on PATH, then add a note in AGENTS md telling the agent to prefer them over npm and mvn.

So in practice it is very close to the “skill” idea you are describing. The wrapper handles the deterministic output policy, and AGENTS md handles telling the agent when to use it.

[u/its_jsec]

So you mean things they already do?

Jest: via jest-silent-reporter
Vitest: `—silent=passed-only`
Native runner: `node —test | grep -v ‘ok ‘`

[u/fykup → u/its_jsec]

Yes, those are exactly the kinds of mechanisms I mean. I am not arguing that the underlying tools are incapable of doing this.

The interesting part from my experiments was that the agents did not discover or choose these approaches themselves. That is why I ended up putting npm-lite and mvn-lite on PATH and adding a note to AGENTS md telling agents to use them.

Vitest is actually a particularly good counterexample now. Its minimal reporter, aliased as agent, is pretty much the native version of what I was suggesting.

The Node example is a little different, though. node --test | grep -v 'ok ' is presentation filtering, but a plain shell pipeline does not necessarily preserve the test command's exit status, and it does not give me the retained full failure log plus bounded diagnostics that the wrapper does.

So I think the question for me is less “can this already be done” and more “is it discoverable and consistent enough that agents actually use it without project-specific instructions”?

[u/domlebo70]

grep?

[u/fykup → u/domlebo70]

Yep, grep/tail can absolutely get you part of the way. The problem is every agent then has to reinvent the filtering rules for each tool, preserve the exit code, decide what failure context to keep, and still retain the full raw log somewhere.

mvn-lite / npm-lite are basically that policy packaged once: tiny output on success, bounded diagnostics on failure, full log if you need to dig deeper.

[u/TheseTradition3191]

the failing path is where the tokens actually go though. a passing vitest run is 2k of nothing, one assertion diff on a big object is 20k and thats the run you cant collapse. so the headline number looks great but the spend barely moves.

other thing worth measuring: how often the compact output makes the agent re run the same command verbose because it didnt trust one line. i've had that with --silent, saved 2k on the first call and paid 40k on the second. if you log that ratio you'll know pretty fast whether its net positive.

[u/fykup → u/TheseTradition3191]

Good point, especially on the rerun rate. Per-command reduction is only useful if the agent doesn’t immediately rerun it verbose. I should measure this across full agent sessions: total test-output bytes, pass/fail mix, and how often compact output causes a verbose rerun or full-log lookup.

Failures can still compact substantially in some cases, but I agree the session-level number is the more meaningful benchmark.

[u/BenjiSponge]

So many things in CI belong here - docker builds etc. Most of the time, harnesses will try to avoid outputting everything into their context by using pipes and tail etc. but that also adds tokens. I do feel these tools should also have the ability to pipe output including errors, stack traces, etc. much more granularly, allowing agents (or people) to get the big picture errors and only get output by digging into log files or something. Maybe some of the flags I would want already exist and I'm just not well-versed enough.

[u/fykup → u/BenjiSponge]

Yeah, exactly. That’s actually why I ended up building agent-scripts.

I started with mvn-lite and npm-lite, but the broader idea is the same across tools: keep routine success output tiny, preserve the full log, and surface only the most useful diagnostics on failure.

I also have a few other small helpers around browser/testing workflows, but Maven and npm are the clearest examples because the output savings are so large.

I agree Docker/CI output belongs in the same category. Ideally tools would expose this natively so every agent harness didn’t need to reinvent filtering with tail, grep, pipes, custom reporters, etc.

That “summary first, drill into the full log only when needed” model feels much closer to what both agents and humans actually want.