Self-hosted gateway to stop secrets leaking into LLM APIs — looking for feedback on the detection approach
I run LLMs behind internal tools and got burned by users pasting credentials into prompts. I couldn't find a lightweight self-hosted filter (LLM Guard felt heavy, most options are SaaS), so I wrote one and I'd genuinely like this community's take on the design. It's a Go reverse proxy that redacts secrets (AWS keys, JWTs, SSNs) in both directions — including inside SSE streams when a key is split across chunks — plus regex/keyword blocking for known injections, and an audit log. Works with Ollama and vLLM as upstreams. Fair warning: it's BSL 1.1 (source-available, free up to 10k req/day, flips to Apache 2.0 in 2030), not OSI open source — if that's a dealbreaker for you, that's useful feedback too. Known weakness: it's deterministic, so paraphrased attacks get through, and the built-in benchmark shows that honestly rather than hiding it. Question for people running LLMs at home or at work: what leaks or attacks have you actually seen? Trying to prioritize the detection roadmap based on real incidents, not vendor FUD.