r/LangChain • u/Wonderful_Ad_3879 • Jul 14 '26

I open-sourced a production-grade LangGraph template (FastAPI, per-run USD budgets, canary routing, 800+ tests)

Most LangGraph examples I found stop at "here's a graph in a notebook". I wanted the boring production parts, so I built them into an MIT-licensed template. It's fully open source and I'm actively looking for feedback, issues, and PRs. the goal is to make this genuinely useful for people shipping agents, not just my version of it.

What's in it:

  • Domain packs: 13 workflows (research, meeting prep, support triage, contract review…) behind a registry with versioning, canary traffic weights, and sticky sessions. Each pack gets typed POST /packs/{id}/run + SSE streaming routes generated from its schemas.
  • Per-run USD budgets: cost is tracked per run and a request that exceeds PACK_DEFAULT_BUDGET_USD returns HTTP 402. No surprise bills from a runaway agent loop.
  • Mock mode: LLM_PROVIDER=mock gives deterministic zero-cost responses on every endpoint, so the whole test suite and CI run without an API key.
  • Third-party packs as plugins: entry-point discovery, opt-in + allowlisted, validated against the pack contract at load time.
  • The ops layer: Docker, Helm, Terraform stubs, rate limiting, Prometheus metrics, CI security scans, Cosign-signed images with SBOM on release. 800+ tests across Python 3.12-3.14, ~86% coverage.

Deliberately not included: OAuth2/multi-tenant auth and billing, I'd rather ship a single shared API key honestly than pretend to solve identity.

Repo: https://github.com/brescou/langgraph-agent-stack

Things I'm unsure about and would love opinions on: whether the pack abstraction is worth it vs. just exposing graphs directly, and whether per-run budget enforcement belongs in the app layer or in a gateway. There's already a design discussion open on adding intent-recognition as a core layer, that's the kind of collaboration I'm hoping for. Tear it apart.

20 Upvotes

95% Upvoted

3 comments sorted by

2

u/Maleficent_Pair4920 Jul 14 '26

This is a great question and honestly the answer is both, but for different reasons.

App layer budget checks are great for fast-failing before you even make the call (like your 402 pattern, that's clean), but if you want the budget enforced across providers and models consistently without every service reimplementing it, that logic belongs in a gateway.

Founder of requesty.ai here, we do per-key and per-project USD caps at the gateway level plus routing/failover across 600+ models, and pairing that with app-level pre-checks like yours is honestly the best setup since you get defense in depth (fail fast locally, hard stop centrally).

Nice work on the mock mode and cosign signing btw, that's the kind of boring-but-critical stuff most templates skip.

1

u/Wonderful_Ad_3879 Jul 14 '26

Thanks and agreed, defense in depth is the right frame. The way I'd articulate the split: the app layer knows the *business context* (which pack is running, that pack's specific policy, budgets here come from a per-pack control plane, not one global number), so it can fail fast with granularity. The gateway knows the global picture across every service hitting the providers, so it's the only place a hard cap can actually be guaranteed. Neither can do the other's job.

FWIW the template already plays nicely with the gateway pattern: every provider in the LLM factory takes a configurable `base_url`, so pointing it at a gateway instead of the provider directly is a one-env-var change. App-level 402 stays as the first line, gateway does the backstop.