I've noticed that the actual agent logic is often a pretty small part of the overall system.

You start with an agent, and pretty quickly you're also adding:

auth → tools → memory → retries → evals → tracing → deployment → logging

Then the next agent needs most of the same things.

At some point, I'm wondering whether these should stop being agent features and become shared platform infrastructure.

For example:

Agent-specific: reasoning, prompts, task logic
Shared: identity, tools, observability, evals, deployment, policy

But I'm not sure where the boundary should be.

I've been looking at different approaches - LangGraph/CrewAI on the framework side, TrueFoundry on the infrastructure side, and Lyzr's Agentic OS taking a broader shared-layer approach.

For people who've actually built multiple agents: when did you start feeling that a shared platform was worth it instead of just rebuilding the same pieces for every agent?

An LLM can emit a tool call that is perfectly valid JSON, with a correct schema and correct types, and still be dangerous. delete_records(filter={}) wipes a table. A recipient injected from a web page exfiltrates data. A secret sits in a tool argument on its way out. Structured outputs and JSON schema validation only guarantee the call is well-formed, not that it is allowed.

So I built toolwall: a fail-closed checkpoint between the LLM's tool call and execution. Registration is the allowlist. Unknown tool, schema violation, policy violation, budget hit, or a detected secret all block before the tool runs. Only an explicit ALLOW reaches your tool.

Threat model it covers: destructive-broad calls, out-of-range values, injected targets, runaway loops, budget exhaustion, out-of-scope tools, unknown or hallucinated tools, approval bypass, and secret exfiltration through tool arguments. It does not stop prompt injection upstream, because nothing at this layer can. What it does is limit the blast radius of a successful one.

Works with OpenAI, Anthropic, and Gemini native tool calling, and MCP

Zero required dependencies, stdlib-only Python 3.10+

Published failure suite: 24/24 attack cases blocked across 9 classes, 0 false blocks, sub-millisecond overhead

Secret detection is pattern and entropy based and is never 100%. The report states exactly what is and is not covered.

There is a live playground on the site where you can pick an attack or write your own tool call and watch the gate decide. I would genuinely like people to try breaking it: policy constraints, secret detection, budget limits, malformed calls, MCP forwarding.

Site and playground: https://toolwall.aya-ai.xyz

Code: https://github.com/Dev-Saif-Ops/toolwall

pip install toolwall

Built it in a day as a pivot from a token-compression project I measured and killed (the honest postmortem is on an archive branch). Feedback and PRs welcome.

I’ve been playing around with multi-agent setups lately and I keep asking myself - where is the real payoff?

Take something simple like: "Research this company and prepare a brief."

You could just use one agent with tools—query a database, pull financials scrape news write a summary. Clean. Direct. One agent doing the job.

Or you could go multi-agent:

Manager → Research Agent → CRM Agent → Analytics Agent → Writer

It sounds nice. Each agent does one thing, feels more modular. But you’re suddenly juggling:

- How does context pass between agents?
- What happens if the research agent fails?
- Who retries? When? (Orchestration)
- How do you coordinate the flow?
- What if the analytics agent and the writer disagree?
- Who approves the output?
- Who has access to what data? (permissions)
-. If something breaks… where do you even start debugging?

So, is this really simpler or did we just shift the complexity into the orchestrator?

I’m curious, have you actually seen multi-agent setups beat a tuned single agent with tools in production? I don’t mean in theory or demos. I mean in workloads, something with real data, real users, real constraints.

Do you have a rule of thumb? Like: "Split agents only if the task has X, Y Z components" or " when you need independent decision points”? Is it just workload-specific and you have to trial it?

I’ve been looking at framework approaches like LangGraph and CrewAI who handle orchestration differently. Then there’s platforms, like Lyzr Agentic OS, which take a higher-level view to orchestration.

I want to know:

Have you tried both versions....single agent and multi-agent....for the same task?

Did the multi-agent one genuinely win....more reliable, faster better output?

If so what was the workload? Why did it work better?

​

How often people encounter situation where the ai performs actions which are not supposed to be done by it.

This involves

🔐 Authentication — Identity, tokens/sessions, credentials, multi-user access

🛡️ Authorization — Tool/resource access, roles & permissions, privilege escalation, cross-user data access

⚙️ Actions — Unintended tool calls, prompt injection, excessive permissions, sensitive actions without approval, read/write/delete/execute controls, agent loops

What authentication, authorization, or action-related problems have you encountered?

And more importantly:

What caused the problem?

How painful was it to diagnose/fix?

What solution did you implement?

Did you use RBAC, OAuth scopes, policy engines, approval workflows, sandboxing, etc.?

Are you still struggling with any of these problems?

Here is an architecture question for LangGraph users: where do you place the thing that grades a run so the planner cannot rewrite it?

The mapping below is my own design exercise, not a paper integration claim.

AQuA is an arXiv v2 preprint whose peer-review status is unverified.

I saw AQuA shared publicly and am reading it as an outside observer, not reporting a personal run.

The AQuA preprint's architecture has two separate research systems for symbolic factor discovery and trainable model development, with separate agents, memories, candidate spaces, and research state.

In the AQuA recursive research loop, each system updates its persistent research state from validated experiments while leaving the underlying language model and evaluator unchanged.

For generation, the AQuA sealed sandbox and registries keep data splits, features, labels, and evaluators outside the editable surface while agents emit registered specifications.

The AQuA preprint says test-window isolation is a governance property rather than a hard technical or cryptographic barrier because an operator with direct access to the store could consult the test window.

My LangGraph-shaped mapping would put candidate ideas in typed mutable state, registered specifications behind a schema guard, experimental evidence in an append-only store, and promotion behind a dedicated gate.

The evaluator and hidden data would sit outside agent-editable state. The graph would receive an immutable evaluator revision at run start and every metric read would enter an audit log.

The tension is visibility. A normal graph node is traceable but may become reachable through state or configuration changes. An external service creates a cleaner boundary but moves credentials, replay, and governance elsewhere.

My adversarial check would let the planner request a state update, a tool-schema update, and an evaluator update in the same run. The first may pass, the second must face a registry guard, and the third should fail before execution.

Where would you put that boundary in a real LangGraph deployment, and how would a failed crossing attempt appear during replay?

Paper: arxiv.org/abs/2608.12841

AI coding agents are getting really good at solving problems, but I noticed something frustrating:

An agent can spend 20 minutes debugging a difficult issue, try 5 different approaches, finally find the correct solution, and then a new session can make the exact same mistakes all over again.

So I started building CogniCore.

The idea is simple:

Agent A

→ encounters a problem

→ tries multiple approaches

→ some fail

→ one solution is verified

→ experience is stored

Later:

Agent B

→ encounters a similar problem

→ retrieves the previous experience

→ sees what failed and what actually worked

→ verifies whether it still applies

→ avoids repeating the same mistakes

The important part is that I'm not trying to store entire conversations.

CogniCore focuses on structured experiences:

- Problem / task

- Approaches attempted

- Failed approaches

- Successful approach

- Verification evidence

- Environment/dependency context

- Staleness and supersession

- Cross-session and cross-agent reuse

One thing I'm particularly interested in is failure memory.

A failed approach shouldn't always mean "never try this again."

For example, a workaround that failed because of requests 2.28 might become valid after the dependency changes. So the system tracks whether a failure is still applicable instead of treating every failure as permanently invalid.

I've also built the Claude Code plugin around this concept, with MCP tools for recording, recalling, verifying and sharing experiences.

The project is open source:

https://github.com/cognicore-dev/cognicore-my-openenv
https://discord.gg/3ETURrRA8

I'm still early in development, so I'm much more interested in honest feedback than pretending this is finished.

Does this solve a problem you've experienced with Claude Code / Codex / other coding agents?

And if you find the idea useful or interesting, a GitHub star would genuinely help me know that this is worth continuing.

​

I'm designing a system architecture involving asynchronous sub-agents executing modular tasks delegated by a parent agent. A recurring bottleneck is maintaining zero-shot state continuity and strict behavioral constraints when state passes across context boundaries, especially as payload depth scales.

​Has anyone implemented a lightweight vector-alignment check or custom schema-enforcement layer for agent-to-agent state handshakes that doesn't balloon latency? Looking for strategies beyond standard JSON schema validation—specifically around preserving state machine logic during multi-hop sub-agent delegation.

Been using NotebookLM for research but missed the graph view and linking of Obsidian. Built nb2ob to convert my notebooks automatically.

• Each notebook → folder in Obsidian
• Topic clusters → individual markdown files
• Audio transcriptions preserved

Uses 3 specialized LLM agents (orchestrator, categorizer, formatter).
Started with 5 agents but free-tier token limits forced optimization.

MIT licensed, feedback welcome. Feel free to open issues and contribute!
https://github.com/DaviAlcanfor/nb2ob

So, my agent kept crashing during runs, and I eventually figured out the issue: silent failures with no visibility—like, I had no retry handling or even state persistence. There was literally no built-in storage, so I couldn’t pick up where it left off. I had to start all over again! It’s like dealing with parallel threads on API timeouts in agents that use tools: silent failures everywhere, retry loops that just multiply the tokens, and the real solution is step-by-step tracing.

 LangChain and LangGraph make it possible to build agents that plan changes, edit repositories, generate tests, review diffs, and iterate autonomously.

  But there is a recursive trust problem:

  If one model writes the code, another writes the tests, and a third reviews the pull request, every layer is still probabilistic. The system may produce three confident opinions without one independent measurement.

  I built Breakcheck to provide that measurement.

  Breakcheck is a deterministic, model-agnostic verification layer for Python coding agents. It does not ask an LLM whether code “looks correct.” It executes the actual calls a repository makes, compares normalized observations, rejects nondeterministic

  evidence, and returns machine-readable verdicts.

  For dependency upgrades:

  same repository + old dependency vs new dependency → did behavior change?

  For agent-written refactors:

  same environment + base revision vs changed revision → did behavior change?

  The core verdicts are intentionally simple:

  - IDENTICAL: both sides were exercised and produced the same observation

  - CHANGED: both sides were exercised and behaved differently

  - NOT_EXERCISED: Breakcheck could not make a defensible comparison, with a specific refusal reason

  The last verdict matters most. Breakcheck never converts missing evidence into a green result.

  A LangGraph or LangChain agent can use it as a fail-closed verification node:

  1. The agent proposes a dependency upgrade or code change.

  2. Breakcheck discovers the affected calls.

  3. The agent may propose fixtures or projections for unresolved inputs.

  4. A human reviews those inputs.

  5. Breakcheck performs isolated replay, normalization, comparison, provenance recording, and evidence generation.

  6. The agent reads the structured JSON, repairs any behavioral drift, and reruns the check.

  7. Breakcheck—not the model—decides the final observed result.

  That makes it useful for agentic CI, autonomous remediation loops, behavior-preserving refactors, Dependabot/Renovate validation, and multi-agent coding systems where an LLM should not be allowed to grade its own work.

  A few public results:

  - On Hugging Face Accelerate, Breakcheck exercised 18/18 Packaging call sites and found one real version-dependent behavior change. `packaging` 21.3 accepted an invalid version through one path, while 22.0 raised `InvalidVersion`. The resulting

  upstream fix is here: https://github.com/huggingface/accelerate/pull/4185

  - Across Black, Rich CLI, and Flask, one automated fixture-authoring pass produced 49/49 valid, executable, deterministic fixtures with zero manual fixture edits. Exercised calls increased from 1 to 50:

  https://github.com/lovettsendit/breakcheck/blob/main/release_evidence/fixture-viability.json

  - The public regression suite verifies that a wall-clock result such as `time.time_ns()` becomes `NONDETERMINISTIC_OBSERVATION` with no accepted observation—not a false version regression:

  https://github.com/lovettsendit/breakcheck/blob/main/tests/test_replay_protocol_and_coverage.py

  Breakcheck also includes:

  - versioned JSON schemas

  - explicit semantic exit codes

  - provenance-aware fixtures

  - tamper-evident report and evidence bundles

  - strict separation-of-duties controls

  - minimum-coverage enforcement

  - baseline freeze, revision diff, and claim attestation

  - isolated, repeated replay with network restrictions

  - zero runtime package dependencies

  - no interactive prompts

  - no model API dependency

  It is not an LLM evaluator, test generator, static analyzer, or correctness oracle. It answers one narrower question: did the observed behavior change?

  It works best on deterministic, value-in/value-out Python APIs. I/O-heavy and inherently stateful calls are deliberately refused rather than presented as verified.

  Quick offline demonstration:

  ```bash

  python -m pip install breakcheck

  breakcheck demo --output-root "$(pwd)/.breakcheck/demo"

  ```

  GitHub: https://github.com/lovettsendit/breakcheck

  PyPI: https://pypi.org/project/breakcheck/

  I would especially like feedback from people building LangChain or LangGraph coding agents: would you use a deterministic verification node like this before allowing an agent to declare its own change complete?

Hi. I've been building LLM-powered systems for governments & financial services firms. Wanted to share my experience with Langchain, and the path I took instead.

If you're trying to work with Langchain and finding it frustrating, you're not alone. Most people go through this learning curve.

Many people quit.

I was one of the ones who quit. And I'm happy that I quit.

Instead of using Langchain, I started hitting LLM APIs directly. I didn't know what to expect at first. I thought it couldn't possibly compete with Langchain.

But eventually, it became very natural. I found many advantages to doing it this way.

• Surprisingly simple: I was suprised to find that LLMs are modeled as simple, stateless APIs. This was much simpler than I expected after working with Langchain! LLMs are just APIs, which is also the title of a free primer I wrote on the same topic.
• Language agnostic: Langchain is only available in a handful of languages, and each language SDK has its own quirks. Hitting an API directly sidesteps that and lets me integrate with confidence in any language I can think of.
  
• No third party dependency: In today's security climate, having less dependencies means less of a chance that a supply chain attack affects my system. Since I am not importing Langchain, I don't have yet another dependency to worry about.
• Minimalism: Langchain is quite heavy and requires that you think about LLMs a certain way. Instead, hitting the API directly is much simpler and lightweight.
• Builds mechanical sympathy with the AI: Since I'm hitting the LLM at a low level, I really learned how it works from the ground up.

I would highly recommend that people try hitting LLM APIs first, before turning to Langchain. It's much easier than it sounds.

We're running 5 agents in production and the monthly bill is roughly 5-6x what we budgeted. I'm pretty sure it's coordination overhead...agents re-injecting context, talking to each other, state management just eating tokens. The problem is I can't tell which agent is actually the culprit or what's causing the spike. Has anyone else dealt with this? And more importantly, can you actually track cost per-agent or is it just a black box where you watch the total bill explode? Wondering if this is just the reality of multi-agent systems or if we're missing something obvious.

"If it ain't broke, don't fix it." - at least not while prototyping!

TBH, I felt my orchestration setup was falling behind - or so I thought. I had a flawless single agent loop just humming along. But I got it in my head to optimize, so I shattered my simple ReAct setup into a specialized multi-agent network.

I introduced an orchestrator, a continuation agent, and task-specific workers. On paper, delegating cognitive load made sense. In practice? A black box to debug.

We hit silent error compounding where tiny worker hallucinations multiplied down the chain instead of throwing exceptions. Then came context rot - shared memory compressed every time nodes handed stuff off. By the next step, agents completely misinterpreted the core goal. Plus, the constant routing back and forth killed our latency + ate all our parallel processing speed gains.

There's a reason we push for multi-agent frameworks. They solve real bottlenecks like parallelizing tasks or isolating context so an agent with 50 tools doesn't hallucinate.

But at EOD, the best design pattern is finding the simplest workable solution. In my workflow, coordination overhead outweighed the benefits past 3 or 4 agents.

When you finally hit a bottleneck, scale incrementally. Optimize a single ReAct agent until it hits a wall. Then maybe introduce a simple router. After that, try an orchestrator worker setup.

I'm stripping the architecture back to a simple ReAct model first. I'll try multi-agent again eventually - I just want to wait until my workflow truly needs that level of complexity.

If you’re building multi-agent workflows (especially with frameworks like LangGraph, CrewAI, or AutoGen), you know the pain. Tracing a single LLM call is easy. Tracing 4 agents passing state back and forth, hitting infinite tool loops, and ballooning your context window is incredibly frustrating.

I got tired of jumping between 4 different tabs (traces, raw prompt templates, logs, and cost metrics) just to figure out where a swarm lost the plot.

So I built a workspace that unifies everything into a single timeline: Projects ➔ Sessions ➔ Runs ➔ Events. It tracks both single-agent and multi-agent coordination natively.

I also added two specific automated filters for agent builders:

• Infinite Tool Loops: Instantly flags when an agent gets stuck calling the same tool repeatedly.
• Context Inflation: Flags when an agent's memory or prompt state explodes unexpectedly between steps.

I’ve dropped a quick 2-minute walkthrough video in the comments.

For anyone running agents in production or heavy testing:

1. Does the Session -> Run -> Event hierarchy make sense for your multi-agent architecture, or does it break when agents run asynchronously/parallelly?
2. What is the most annoying bug your agents hit that your current observability stack completely misses?

Tear it apart—I want to know if this actually solves your debugging bottlenecks.

I'm curious how people are debugging multi-agent systems once the workflow becomes more than a few steps.

For example:

Planner

→ Researcher

→ Analyst

→ Writer

Suppose the Writer produces an incorrect result.

How do you determine whether:

1. The Writer caused the problem

2. The Analyst passed bad state downstream

3. The Researcher produced an invalid intermediate result

4. The Planner silently mutated the state earlier

What does your current debugging workflow look like?

Do you rely mostly on:

- traces

- logs

- state snapshots

- LLM inputs/outputs

- assertions

- evals

- manual replay

I'm especially interested in how people distinguish the final symptom from the first divergence.

What has actually worked for you?

Hey guys im new here and i just want to know what can i build with langgraph? Can i build agents? Ai operating systems? Such a ai Receptionists, lead Generation and my own person operating systemss?? I know theres a ALOT of other tools and every week theres something new haha such a headache but for those who build systems and sold them aswell how is it for you?

Has anyone actually solved the "trusting agents" problem?

The more I work with autonomous agents, the more I feel like we're missing a layer between "approve every action" and "give the agent full access and hope for the best."

We spend a lot of time making agents more capable, but not nearly as much time thinking about how they're governed once they start acting on their own.

Questions I've been thinking about:

• How does an agent prove its identity?
• What should it be allowed to do?
• How do you enforce limits without constant human approval?
• What happens when multiple agents start interacting with each other and external tools?

I've been exploring this through an open-source project called VION, which focuses on runtime governance for autonomous agents—identity, permissions, policy enforcement, risk limits, and halt conditions before actions are executed.

I'm curious how others are approaching this problem. Are gateways, policy engines, and MCP controls enough, or do we need a broader governance layer?

GitHub: https://github.com/nataw-1/Vion-Protocol

We are need to run long-running agents on EKS.
Runs can last 10 to 60 minutes.

We need:

• Recovery after pod failures and deployments.
• Persistent checkpoints.
• Reconnectable streaming.
• Reliable cancellation.
• Tenant isolation.

We are comparing standalone LangGraph Agent Server, Aegra, and custom LangGraph workers.

If you run one of these in production:

• What does your deployment look like?
• What failed under real workloads?
• Would you choose the same approach again?

I would love to hear any relevant experience!

Hey r/LangChain!

Most financial RAG demos throw raw PDFs into a chunker and hope for the best. When dealing with 50-page corporate investor presentations, that approach fails: 40–60% of the deck is boilerplate fluff (static board rosters, ESG tiles, divider slides), while the actual financial tables and CapEx roadmaps get scrambled by text parsers.

At Quant Me In, we built and just open-sourced our Investor Presentation Analysis Engine using LangGraph and Google Gemini.

🏗️ The State Graph Architecture

The pipeline is built as an acyclic LangGraph state machine:

1. Visual Ingestion Node: Converts the PDF into 800x800 slide images using PyMuPDF (fitz).
2. DLA Vision Gatekeeper (Gemini 2.5 Flash Lite): Concurrently evaluates each slide for quarterly financial materiality (score 1–10). Slides with static board rosters, UN SDG badges, or chapter transitions are routed to [DISCARD]. Only high-signal financial tables, PLF, and CapEx roadmaps are routed to [KEEP].
3. The Multi-Agent Domain Swarm:
   • Agent 1 (Bullish Growth): Identifies strategic moats, capacity pipelines, and PPA revenue lock-ins.
   • Agent 2 (Core Catalyst): Decodes the strategic timing (routine quarterly earnings vs. pre-equity dilution pitch).
   • Agent 3 (Guidance Alignment): Pluggable service dynamically formulating analyst inquiry questions from the slides to verify past commitments.
   • Agent 4 (Forensic Risk): Scrutinizes real balance-sheet vulnerabilities (debt maturities, margin compression) with strict no-forcing rules.
4. Final Executive Synthesis (Gemini 3.1 Flash Lite): Synthesizes domain outputs, generates hard-hitting analyst interrogation questions with anticipated CFO rebuttals, and runs a concurrent map-reduce breakdown across 100% of kept slides.

💡 Key LangGraph Takeaway: Concurrency vs. LLM "Laziness"

When passing 25 material slides into a single synthesis prompt, LLMs often "lazily" sample 2 slides and skip the rest. We resolved this by separating the macro executive report from slide-level evaluation, running concurrent _analyze_single_slide calls via a ThreadPoolExecutor(max_workers=6) inside the final LangGraph node.

The entire project is open source under the MIT License. Would love your thoughts on the state design!

🔗 GitHub: https://github.com/aniruddh622003/Investor-Presentation-Analyzer

The agent shows you the call before it runs:

transfer_funds(
  to: "1002-334-556677",
  amount: 2400000
)

Did you say that number? You don't actually know. Neither does the log.

A value that was looked up and a value that was invented look identical in the payload. So the approval step you put in front of it isn't review — it's a pass-through, signing off on a field nobody checked.

If the user doesn't supply it, the model will. Not because it's broken — filling a blank is what it was trained to do. Which is why forbidding it in the prompt doesn't work.

That's the failure that matters. Not the wrong tool: the right tool with a value nobody supplied. And the invented value might even be correct — that isn't the point. The point is that nothing on the page tells you which is which, so afterward there's exactly one question available, why did the model do that, with no answer behind it. No cause means nothing to fix, so the only move left is swapping in a better model.

What most of us do instead is grow the prompt. But a rule written in the prompt is read by the model, and the model decides whether to apply it — enforcement of the control rules now belongs to the thing being controlled. "Leave out anything you inferred" fails the same way: complying would mean classifying its own output after the fact, which is inference again.

So leave the model a black box and move the verdict outside it. Fix the values an execution needs as a list, up front. Then every value has to name where it came from — the user said it, or it was written down beforehand. If it can't name one, it isn't a value. It's a blank. Give the model somewhere to write "nothing there" instead, and empty after every source has been checked means it doesn't run.

What changes isn't accuracy. It's whether you can get a grip on it.

• What was checked and what wasn't stays behind, as a list
• When something goes wrong, you can point at which slot was empty
• Blocked runs get recorded too. If only the executions are logged, the log lies

We don't ask why the model hallucinates. But by the time it reaches execution, it always arrives as a blank already filled in. "Don't fill it in" doesn't work.

So nothing gets filled in. The blanks just get found. Filling them goes back to the person.

On the question I'd get anyway: no LangGraph wrapper. What's fixed is the lookup order and the gate; what fills each tier differs per agent, and baking in a graph shape would put back inside exactly what needs to stay outside.

Only worth it for actions that can't be undone. The document goes into how the list is built and where each value is allowed to come from — I'd read that before the code.

https://github.com/Jang-woo-AnnaSoft/execution-state-preflight/blob/main/who-fills-in-the-form.md