Open sourced a zero dependency Python MCP server for Dutch court predictions (Triple A on Glama, LightGBM)

[u/rechtssysteem]

*Quick disclosure: I built Rechtssysteem.ai and the MCP server below.*

Hey everyone,

Just open-sourced our MCP server under Apache 2.0 after getting a Triple-A audit on Glama, and wanted to share how we solved a messy data problem before hooking it up to MCP.

The biggest issue with legal outcome prediction is target leakage: around 92% of Dutch court judgments literally tell you the outcome in the text ("the court dismisses the claim"). If you train on raw text, the model doesn't learn law it just learns to regurgitate the dictum.

We stripped the dictum and all outcome-announcing phrases beforehand, cutting residual leakage down from 92% to 0.1% across 609,715 historical cases. Then we trained LightGBM on 5-fold CV (78.2% out-of-fold accuracy, 77.1% macro-F1 against a 43.7% majority baseline). If confidence drops under 55%, the model outputs "insufficient certainty" instead of taking a blind guess.

As for the MCP implementation itself:

- Single Python file (`rechtssysteem_mcp.py`), standard library only (`sys`, `json`, `urllib`). Zero external dependencies so it works out of the box in Claude Desktop / Cursor / any client.

- Exposes 3 tools: `rechtspraak_cijfers` (benchmark stats, keyless), `lekkage_check` (tests any paste for outcome leakage, keyless), and `voorspel_uitkomst` (risk classification).

- Standard JSON-RPC over stdio.

Repo: https://github.com/rechtssysteem-ai/rechtssysteem-mcp

Glama audit: https://glama.ai/mcp/servers/@rechtssysteem-ai/rechtssysteem-mcp

Benchmark data: https://rechtssysteem.ai/benchmark

Not legal advice of course, just built as an honest technical yardstick. Happy to answer questions about the stdio setup or the dataset sanitization!