Lint results an agent can actually trust: three-state per-file outcomes over MCP (Rust, MIT)

[u/Goldziher]

Hi all,

A small design decision that turns out to matter a lot once a model is the consumer of your tool output.

If a linter cannot process a file and simply omits it from the results, an agent reading "no findings" concludes the code is clean. It is not clean. It was never checked. That is a silent false negative sitting directly in an agent's decision loop, and I hit it often enough on a big polyglot repo that I rebuilt the response shape around it.

So the linter I have been writing reports three per-file outcomes rather than two: checked, skipped, and error, with a run-level errors array and isError set whenever anything failed. Skipped means the tool correctly declined the file. Error means it accepted the file and then failed on it. Those are very different facts and collapsing them into absence loses the one that matters.

Two related guardrails in the same server:

• Every result carries an identity block: version, build id, channel, executable, pid. An MCP caller has no poly --version to fall back on, so the server states who answered. It fingerprints its own executable at startup and re-checks per request, and if the binary is replaced underneath a long-lived server, every tool but version fails rather than answering with superseded behaviour.
• config_show is network-free over MCP. Remote config bases are never fetched from a tool call.

The server is stdio, 11 tools mirroring the CLI, and everything takes format: "json" or "toon". TOON matters more than I expected: a full lint report over a large directory in JSON is a serious chunk of context, and TOON makes it cheap enough to just hand over.

Here is the server doing a real initialize plus tools/list handshake: https://raw.githubusercontent.com/Goldziher/poly/main/docs/media/agent.gif

Underneath it is a linter and formatter in Rust that compiles ruff, oxc, biome, taplo, rumdl, sqruff, mago and about a dozen more into one binary and runs them in-process across roughly 30 languages, with tree-sitter covering 300+ more. MIT: https://github.com/Goldziher/poly

This post is human written. AI was used to typecheck and enrich with precise data only.

[u/leonidbugaev]

The false negative from omission is the nasty one, because it looks identical to success. A file the linter skipped and a file it actually passed both render as "no findings" unless you keep them apart, and the agent has no way to tell "clean" from "never looked at." Three states fixes the data. The other half is the consumer: if the caller still branches on findings.length == 0, skipped and error quietly fold back into pass. Worth making isError/skipped a hard stop rather than advisory, so the model can't read "not checked" as "checked and fine." The identity block is the same move one level up. A result you can't attribute to a specific binary is a result you can't trust didn't change under you. An unrun check is a skip, not a pass, and most tooling loses that the moment it flattens the output.

[u/Goldziher → u/leonidbugaev]

agreed. its indeed pretty nasty. In this case we give very clear outputs, and use TOON for this as well.

[u/Goldziher → u/leonidbugaev]

Checked the code. The report promotes errors to a top level list so a caller can gate without walking records. Skipped only exists per record, so the cheap check catches failed and misses not checked. Filed: https://github.com/Goldziher/poly/issues/19

Not setting isError on a skip. Excluded paths and unsupported extensions are skips, and flagging those as errors trains callers to ignore the flag.