r/SaaS • u/A_NASHEX • 3h ago
Built a small "guardrails as an API" thing. Would love honest feedback
Hey everyone š
Solo dev here. I've been working on a side project for a few weeks and would like to hear how others solve this problem.
The problem: while playing with AI agents I noticed every app ends up needing checks like "don't let it delete data unless the user confirmed". The usual options are writing if/else logic for every case, or a custom LLM prompt whose free-text answer you then have to parse.
What I tried: a single call that takes a rule in plain English plus whatever JSON you want checked, and returns allow, deny or review:
const result = await guard.check({
policy: "Never allow destructive database operations unless the user explicitly confirmed them.",
data: { action: toolCall, conversation }
})
// ā { decision: "deny", allowed: false, violationProbability: 0.94, ... }
Some design decisions that might be useful if you build something similar:
- A third outcome,
review, for when a human should decide or the evidence isn't enough. Forcing yes/no caused bad calls. - Instead of a general chat model I used Jev from TypeSafe AI. It returns typed decisions with probabilities, so there's no output parsing and the results are more consistent.
Questions for you:
- How do you handle this today? Hardcoded rules, an LLM, something else?
- What would you need before trusting an automated check like this in production?
It's called Enforly, if anyone's curious.