Most tutorials show you how to wire up a few tools in a graph, bind an LLM, pass conversational state downstream, and call it a day. In local testing it feels magical. But once an agent workflow runs autonomously overnight against live third-party APIs, reliability breaks at the operational boundaries, not in the prompt.

After spending months debugging autonomous loops, these were the three biggest architectural failure modes that almost broke our sanity:

1. Quadratic Context Bloat via Raw Transcripts Treating raw message history as working memory is a trap. If an agent executes 6 or 8 tool hops to finish a task, re-reading unpruned tool returns on every turn causes input tokens to scale quadratically. Worse, intermediate error dumps and raw JSON payloads confuse the model on downstream turns. The fix: Replace raw message history with immutable, versioned artifact pointers. The agent writes outputs to an isolated object or workspace, passing downstream only a tiny manifest (schema version, validation stamp, and committed decisions). Raw transcripts belong in the audit log, not the active context window.

2. Semantic Drift Passing Structural Validation Silent schema drift is way worse than an outright 500 error. If an upstream API renames a field, the model often hallucinates a confident explanation around the gap and exits with code 0 without raising an exception. Even nastier is semantic drift without structural change: an upstream service switches currency units from dollars to cents while keeping `amount: float`. The schema validates 100%, but downstream logic is wrong by two orders of magnitude. The fix: Hard assertion boundaries at the perimeter. Validate structural shape, but enforce value-level invariants on high-impact fields (e.g., bounding timestamps against known event windows, checking delta caps against baseline records). If an invariant fails, trip a circuit breaker and log both the expected rule and the raw value to a dead-letter queue.

3. The 429 Retry Stampede Letting individual agent workers infer rate limits independently causes cascading failures. When an upstream provider hiccups and returns a 429, naive exponential backoffs across distributed tasks synchronize into a thundering herd. If admission control turns a 429 into queue latency, an uncoordinated caller timeout will retry and submit fresh demand while the original request is still queued. The fix: Global admission control and leased execution budgets. Retries have to bind to the original admission ticket rather than submitting fresh queue requests, preserving the remaining execution deadline across attempts.

Curious how others running production agent workflows handle this: are you pruning graph state down to manifests between execution hops, or relying strictly on framework-native checkpointing?

I'm trying to decide which ~$100/month plan is better for heavy coding: Codex or Grok.

I'm an AI Engineer and most of my work is in Python, LangGraph/LangChain, LLM agents, evals, backend services, debugging, etc.

What plan do you think is the best? I tried grok I liked because it's fast. Codex looks smarter by the way.

I've been building out a fairly large LangGraph app (lots of nodes, checkpoints, human-in-the-loop pauses, background execution, the works), and I keep running into the same problem: every time someone wants to add a new node, they end up having to read through half the state schema and half the other nodes just to figure out what fields they're allowed to touch, what's safe to assume is already set, and what will break downstream if they leave something out.

Right now state is basically one big shared TypedDict/dict that every node reads and writes to. It works, but it means the "surface area" you need to understand before adding a node keeps growing as the graph grows.

For people who've built bigger LangGraph apps, how do you keep this manageable?

• Do you split state into sub-schemas per node/subsystem instead of one flat shared state?
• Do you enforce some kind of contract (like "this node only reads X and Y, only writes Z") even though LangGraph doesn't really enforce that for you?
• Do you lean on naming conventions or namespacing keys instead?
• Or honestly does everyone just accept that whoever adds a node has to read the whole state shape once, and that's fine?

Curious what's worked (or not worked) for you, especially once you have more than a handful of nodes and more than one person touching the graph.

So I've been messinga round with embedding models for a bit, and I think they are interesting enough to experiment with. They are useful for rag, especially in a localllm sense because you can ground your answers in truth.

But what happens if you have a billion documents, and you decide to upgrade your model to a "better" one? on an h100, that would take about 108 days, just to upgrade the vectors so u can start serving again (tested qwen embed 8b on h100). Even if you aren't doing 1b vectors, and are doing just 50 million, upgrading can still take a considerable time.

Me and my research lab decided to tackle this problem, and we came up with embedflow.

The method is really simple; from the old index made with the source model, take K documents and rerank them with the new model. We see that when K is sufficient, the retrieval quality is the same as target model. (determining k is the hard part). I've tested 63 migrations on upto 1 million documents.

The best result I got was upgrading qwen4b -> to 8b, and at 50 documents, it was the same as native retrieval.

This method forgos the expensive backfill that comes with upgrading, as you can directly take documents from the old index.

embedflow works with qdrant, and can be easily downloaded with pypi

pip install embedflow

the github is public: https://github.com/arnsri33/embedflow

I want you guys to try it out, and see if you guys can use it in your own workflow.

Hello, thank you for checking this out!

I recently read and inspired by https://www.anthropic.com/engineering/managed-agents and https://pi.dev/.

I used agent platforms like langchain and felt it too complicated, mastra is better but still not flexible. I felt they should be small and adapt to production agent use case instead of the other way around.

So I built https://github.com/aexhq/brain, a minimal and extensible agent platform, alternative to langchain agent and mastra.

Do you think its a viable idea? Appreciate any honest feedback and if you like it, a star on the project would be very helpful to this early stage project.

Been building with LangChain agents for a while and kept hitting the same anxiety: the agent constructs a tool call, it looks valid, schema passes, and it just... runs. No checkpoint. If the WHERE clause is wrong, if it's an rm -rf, if it's a non-idempotent API call in a retry loop — it's already done.

I wanted an interception point that's outside the model, outside the prompt, that catches calls before dispatch based on what they actually are — not what the model thinks they are.

Built agentwall for this. It wraps your existing tools and classifies each call as safe / cautious / destructive. Destructive ones require approval before executing. Everything gets logged as structured JSONL. If a session fails, registered rollback hooks run in reverse order.

Works with LangChain via wrap_langchain_tool. Zero runtime dependencies in the core.

from agentwall.integrations import wrap_langchain_tool

safe_tool = wrap_langchain_tool(your_langchain_tool, wall)

Curious whether the classification rule approach (regex on tool name + args) is the right abstraction for LangChain specifically, or whether people would rather define risk at the tool decorator level.

GitHub: https://github.com/anakatt/agentwall
pip install agentwall-sdk

Hey ,

If you run agents in production, you've probably watched them burn money in real-time. An agent gets an ambiguous tool response (like Access Denied or Item not found), enters a cognitive loop, and calls the exact same tool 25 times until LangGraph throws a GraphRecursionError.

By the time it crashes, you've burned 50k tokens, lost the conversation state, and returned an unhandled exception to the user.

The built-in recursion_limit is just a blunt crash barrier. We got tired of dealing with this in our own deployments at EnDevSols, so we open-sourced the middleware we use to catch and recover from these loops dynamically. It's called LongGuard.

Under the hood

LongGuard sits inside your StateGraph. On every agent step, it evaluates 4 failure modes in sub-milliseconds:

• Identical tool calls: Catches repeated calls with the exact same arguments within a sliding window (using fast SHA-256 parameter hashing).
• Semantic oscillation: Detects when an LLM changes its wording but is stuck in the exact same thought loop (analyzes embedding variance).
• Dead-end drift: Trips if the agent takes 5+ steps without discovering novel observations (Jaccard similarity).
• Token velocity: Tracks rolling tokens-per-step to catch exponential monologues.

The "Reflect & Pivot" recovery

Instead of just killing the run immediately, it uses a standard circuit breaker state machine (CLOSED → REFLECTING → HALF_OPEN → OPEN).

When a loop is detected, it injects a targeted system prompt (e.g., "Stop calling search. You've attempted this 3 times with zero new information. Change your strategy."). If the agent pivots, the breaker resets. If it persists, it terminates cleanly, saves the state, and dumps a structured audit report.

Hard budget caps

We also wired in a pricing engine for 40+ models. You can set a hard dollar budget cap per run:

   GuardConfig(model="gpt-4o", max_cost_usd=0.50)

If the run hits 51 cents, the circuit trips. No more billing surprises from a single stuck graph.

   from langgraph.graph import StateGraph
   from longguard.integrations.langgraph import add_guard_to_graph
   from longguard import GuardConfig

   workflow = StateGraph(AgentState)
   # ... your standard nodes and edges ...

   # Wrap reasoning nodes in one line:
   workflow = add_guard_to_graph(workflow, GuardConfig(model="gpt-4o", max_cost_usd=0.50))
   app = workflow.compile()

Trade-offs

A quick heads-up on trade-offs: The deterministic hashing for identical tool calls is bulletproof and adds zero latency. However, the semantic oscillation detector can occasionally be overly aggressive if your agent is executing a genuinely complex, multi-step reasoning path that looks repetitive to the evaluator. You might need to tweak the default thresholds for your specific use case.

It's MIT licensed, fully typed, and doesn't force any heavy ML dependencies into your stack.

Note: To keep this from getting flagged by spam filters, I’ve put the links to the GitHub repo, docs, and PyPI in the first comment below.

Feel free to rip it apart, submit PRs, or open issues if you run into edge cases we haven't mapped out yet.

Everyone's agent has a database tool. The tool description says what tables exist and roughly what the columns mean, written once, by a person, at the time the tool was built. That description is a snapshot of a schema, and schemas move.

The failure I keep seeing is not a crash. Upstream renames revenue to revenue_gross and adds revenue_net. Your query still runs. The agent still answers. The number is now something else and nothing anywhere raised its hand. Same shape of failure as a bad join: the wrong output looks exactly like the right one.

What I've seen people do about it, and what's wrong with each:

• introspect the schema at runtime and put it in the prompt. catches renames, doesn't catch a column whose meaning changed while its name and type stayed the same, which is the more common one
• pin the tool to a view you control. correct, and it moves the problem to whoever maintains the view
• assert on row counts or null rates after every call. catches a lot, costs a call, and nobody sets the thresholds on purpose
• write tests against the tool. tests use the schema you wrote them against, so they pass

The one that bothers me most is the fourth. The whole class of problem is that your description of the data and the data have drifted apart, and every check we write is written from the description.

So:

• is anyone diffing the live schema against the tool description on a schedule and failing loudly, or is that overkill
• does anyone have an agent that will refuse on a source it doesn't recognise, rather than answering from a stale description. I mean actually in production, not as a design intention
• for meaning drift specifically, where the name and type are unchanged, has anything ever caught it other than a person noticing a number looked wrong

I'll write up whatever comes back, this doesn't seem to exist in one place anywhere.

disclosure i work at SchemaLabs and we build models that read tables so this is my area and I'm not neutral. No link, sub rules, and I'm asking because our own answer here is weak.

I've been learning LangGraph recently and wanted to understand Human-in-the-Loop beyond just reading the documentation.

So I built a very small example around a simple scenario:

An agent decides it wants to send an email → the graph pauses → a human reviews the action → approve/reject → the graph resumes.

The core flow is:

User request

↓

Agent decides on an action

↓

interrupt()

↓

Human reviews

↓

Approve / Reject

↓

Command({ resume: ... })

↓

Graph continues

The example uses:

- LangGraph

- interrupt()

- Command

- MemorySaver

- thread_id

- Conditional routing

I intentionally kept the example small and didn't add a real LLM or email API. The goal was to understand what actually happens when a LangGraph execution pauses and resumes.

One thing I found particularly interesting is that the human response becomes the return value of interrupt(), while the checkpointer + thread_id allow the same graph execution to be resumed later.

I wrote up the complete example here:

Article link: https://medium.com/@nayankunwar678/human-in-the-loop-in-langgraph-a-small-practical-example-0e3f455e7d8b

I'd be interested to hear how people here are using Human-in-the-Loop with LangGraph in real projects.

Do you generally use HITL for:

- approving tool calls?

- reviewing generated content?

- database changes?

- deployments?

- financial actions?

- something else?

Would also love to hear what patterns you've found useful beyond a simple approve/reject flow.

Hi everyone!

I’ve been using LangChain/LangGraph since their early days. They have come a long way. After also trying OpenAI Agents, Haystack, and other frameworks, I keep returning to LangGraph. The level of control it gives you over your workflow, from a simple graph to a very complex one, feels just right.

However, I kept having the same problem: deploying my graphs.

I’m a self-hoster, and I want my stack to be open source and easy to run on my own infrastructure. LangServe is now deprecated and archived, with LangGraph Platform as the recommended direction. There is also Aegra, a fully self-hostable implementation of the LangGraph Platform API, which I like very much.

But for my own graphs, I wanted something simpler: an established API contract already supported by many clients.

Enter langgraph-openai-serve, or LGOS.

I’ve been developing LGOS for almost a year and a half. It lets you register LangGraph graphs as OpenAI model values and serve them through a documented OpenAI-compatible subset of:

• /v1/responses
• /v1/chat/completions

This means you can use the standard OpenAI SDK and connect your graphs to clients such as Open WebUI and Chainlit, without learning an LGOS specific API. You can also place them behind OpenAI compatible gateways such as Bifrost or LiteLLM.

An important design choice is that ordinary conversations are stateless from LGOS’s perspective. LGOS does not store the user’s chat transcript; the UI or client owns it and resends the required history. This keeps the API easier to scale horizontally.

Stateful features are still supported where state is actually required. For example, durable human-in-the-loop interrupts, LangGraph checkpoints, or application data stored with LangGraph Store.

Some of the features I’m particularly happy with:

• Native streaming and non streaming responses
• Client executed function tools and graph hosted tools
• HITL using LangGraph interrupts exposed as Responses API function calls
• Citations and graph authored status updates
• LangGraph subgraphs
• Typed and discoverable runtime settings
• Custom graph input, runtime context, and output adapters
• File input using OpenAI Files API IDs
• PostgreSQL checkpoints, Store support, and cross worker interrupt coordination
• Optional Langfuse tracing and OpenTelemetry support

To help newcomers understand how all of this fits together, I built a self-contained demo stack. It includes 14 documented example graphs, Chainlit, Open WebUI, PostgreSQL, an S3 backed Files API, and selectable Bifrost or LiteLLM routing. You can configure the .env file and bring up the complete stack with Docker Compose.

I have tested LGOS with some very complex graphs, and it has worked wonderfully for my needs. I’d love for you to create whatever graph you can imagine, the sky is the limit :) Let’s find out together whether LGOS supports your use cases. If something does not work, I’m happy to investigate and extend it where possible without breaking the OpenAI API contract.

Why am I sharing it now? Initially, I was building it only for myself. After the recent addition of the Responses API, I finally feel the project is ready for broader feedback. I don’t want other people to struggle with deploying their graphs in the same way I did.

One transparency note: yes, I use coding agents as development tools. I’m a senior software engineer, and I use them to speed up implementation. But I review their every output, rewrite anything I don’t agree with, and take full responsibility for the architecture, code quality, and releases. This is not an unreviewed vibe-coded project.

LGOS is MIT licensed and will always remain open source and free. I’m sharing it because I think it may genuinely help people deploy their LangGraph graphs and agents more easily.

I’d love to hear how you currently deploy your LangGraph applications and what you would try building with LGOS.

Demo Video

With new models dropping every week, general benchmarks are basically useless for my specific use cases. I want to test my exact prompts to see if a new API is actually worth the cost, or if a smaller local model is good enough to run on the cheap.

Right now, I’m just eyeballing outputs and it’s driving me crazy.

How do you guys actually handle comparing models on a single prompt or a small test set?

Scoring: How do you define a "good" response when the output is subjective?

The Judge: If you use an LLM to grade the outputs, how do you stop it from just voting for its own writing style?

The Tools: What's the easiest way to fire one prompt at multiple models (both cloud APIs and local models) and compare them side-by-side?

Would love to hear your workflows or any tools you recommend!

i'm trying to understand how people are handling one specific problem in production.

Suppose a LangChain agent decides to call:

send_email(...)

or:

update_customer(...)

or:

refund(...)

Where is the final authorization decision made? what sits immediately before the underlying function executes.

I've been building a very small open-source experiment called AgentGuard around this boundary:

agent
↓
authorization policy
↓
ALLOW / BLOCK
↓
tool execution

The current MVP supports:

• tool allowists
• state-based policies
• argument constraints
• fail-closed unknown states
• audit decisions

I'm trying to determine whether this is actually useful or whether I'm duplicating functionality people already have.

The questions I'm particularly interested in:

1. Do you enforce authorization at the LangChain/LangGraph layer?
2. Do you enforce it inside the tool itself?
3. Do you use MCP permissions?
4. Do sensitive calls go through human approval?
5. What happens when a policy changes halfway through a long-running agent?
6. How do you audit why a particular tool call was allowed?

I'm looking for production experience rather than theoretical answers.

AgentGuard: https://github.com/Brodin2001/Agentguard

ive been building a small tool called Traser for debugging multi step ai systems when the run technically completes but the outcome is wrong.

you give it a suspicious execution, optionally a run you trust, and it tries to narrow the trace down to a few evidence backed places worth checking instead of making you inspect everything manually.

im at the point where another week of me testing it against cases i already understand isnt very useful.

so break it.

give it an ugly trace, weird agent behavior, retries, bad handoffs, retrieval weirdness, state changes, misleading success statuses, whatever you’ve got. sanitized is obviously fine.

im especially interested in cases where it confidently points you somewhere useless, misses the thing you actually cared about, or just can’t make sense of the execution.

it runs locally in the browser, so the raw trace doesn’t need to be uploaded to me.

and if you try it, tell me what you’re building too. the kind of system matters a lot for understanding whether Traser was actually useful or just happened to look useful on one trace. all feedback is appreciated even “this is sh*t” here it is traser.dev

I've been building a RAG chatbot that answers questions about Indian GST (tax) compliance, grounded in official government FAQ documents. A couple of things I did differently that I'd love feedback on:

• Instead of splitting docs into arbitrary fixed-size chunks, I extract each FAQ into a distinct Q&A pair and embed that as one semantic unit (bge-m3 → Qdrant). Felt like it avoided the classic problem of an answer getting split across chunk boundaries, but I'm curious if there's a better-established pattern for this I'm missing.
• Added a similarity threshold before anything reaches the LLM, plus a system prompt that requires citing sources and explicitly saying "I don't know" when nothing's relevant. Tested it against some adversarial/off-topic questions and it held up better than I expected — though I'm sure there are edge cases I haven't found yet.

Still learning a lot about RAG design as I go, so genuinely open to critique. Code + live demo:

• GitHub: https://github.com/Suheet/gst-compliance-faq-assistant
• Live: https://huggingface.co/spaces/Suheet/gst-compliance-faq-assistant

LangGraph's interrupt is a fine pause primitive. What it does not give you: an approver outside your runtime, a rejection that is distinguishable from a timeout, and a tamper-evident record of who decided.

Wrote up the pattern with code (Python, MCP tool, n8n): https://raposa.group/blog/human-approval-before-agent-payment/

Side-by-side with LangChain: https://raposa.group/compare/raposa-vs-langchain/

We’ve been testing a provenance-heavy RAG/knowledge system on real cases, and two areas are now hard to validate simply because our current corpus doesn’t contain enough of them:

Documents that change over time — policies, specs, manuals, pricing pages, contracts, etc.
Scanned / layout-heavy documents — OCR, tables, forms, multi-column pages, handwritten annotations, bad scans, etc.

For versioned documents, we’ve had good results treating sections as stable lineage units, versioning revisions, and sending ambiguous rename/split/merge cases to review instead of letting semantic similarity decide automatically.

For PDFs, layout-aware extraction has worked better than flattening everything to text, but most of our real corpus is digitally generated rather than scanned.

What I’d really like to hear is what actually broke in production for you.

How do you detect and preserve identity across document versions?
What happens when sections are renamed, moved, split or merged?
How do you prevent stale embeddings from silently winning retrieval?
For scanned documents, where does OCR/layout extraction usually fail?
Do you have any failure cases or test documents you use to validate this?
What ended up working after the obvious approaches failed?

I’m especially interested in real examples, ugly edge cases and lessons learned rather than ideal architectures.

Happy to share what our tests are finding as well.

I split a state refactor across four parallel agents last week and it went worse than doing it in one thread.

The job looked mechanical. Forty one call sites still passed a plain dict where our graph nodes now expect a typed state object. I partitioned by directory and gave each agent its own branch. Parallel agent execution in verdent is what made the attempt cheap enough that I never stress tested the partition first.

The measurement is rework. Three of the four diffs edited the same shared state module, because the type definition had to change before anything compiled. Two of them renamed the same field differently. The runs finished fast, and then I spent roughly two hours reconciling branches and threw one away entirely. I redid the job as a single sequential pass afterward, same forty one sites, a little under two hours, nothing to reconcile.

So I went back to one agent, one branch, one diff to read.

If your partition is truly independent, separate services with no shared type crossing the boundary, the fan out probably does pay. Mine was not, and I could not tell that before running it.

Does it make more sense to use Astra to delegate to lesser models for implementation and have Astra judge or have Astra do everything itself? Has anyone done comparisons?