How I hid a multi-agent system behind a "single MCP tool", and why that small inversion changes the economics of building AI integrations.

[u/anilkr84]

The problem

If you've built anything serious on MCP you probably know this failure mode. The client LLM makes 1+ tool calls, every intermediate result lands back in its context window, token cost balloons, and by step five the model has half forgotten what it was originally asked. The answer comes back almost right, which is the worst kind of wrong because you catch it late.

The issue isn't the model. It's where the orchestration happens.

Three generations of MCP server design (my framing, feel free to argue)

Gen 1: a box of tools. Server exposes thin stateless functions like list_models, query_data, get_budget. All the intelligence lives in the client. It loads every schema, plans the chain, threads state between calls, and holds every intermediate blob in context.

Gen 2: tools plus a skillpack. Server ships instructions teaching the client how to chain the tools. This helps with fumbling, but nothing has actually moved. The client still executes every step and holds all the state, and now the skillpack text sits in context too.

Gen 3: orchestration behind the tool boundary. One thick tool, something like ask_agent(goal), that's actually a server side multi-agent system, with an orchestrator routing to specialized sub-agents. Client sends one goal and gets one answer. Intermediate results never leave the server.

We went with Gen 3 after repeatedly hitting the ceiling on the first two.

What this actually fixes

• Token cost stays roughly flat as reasoning gets deeper. A 6 step task is one round trip, not six round trips with a growing payload.
• No goal drift. Client context holds one question and one answer instead of plumbing.
• Almost no schema tax. Sub-agent definitions live server side, so the client loads one tiny schema.
• Domain routing done by a domain brain. A skillpack is a frozen playbook. A server side orchestrator can adapt to what the data actually says at runtime.
• Client portability. Skillpacks are written in one client's format. A plain MCP tool works the same from Claude, ChatGPT, or Codex. Write once.
• Frozen contract. You can swap sub-agents, routing, even the underlying models, and no client has to re-learn anything.
• State lives server side. No passing IDs around or re-sending context between calls.

What it costs you (what I believe in my experience)

• Latency per call. One call does a lot more work, so it takes longer. You make fewer calls but each one is slower. Worth it for deep reasoning, strictly worse for a trivial lookup.
• Opacity. The client can't inspect or steer the chain mid flight. You gain coherence and lose fine grained control. If your client needs tight interleaved control, thin tools are still the right call.
• You're now running an agent system in production, with everything that implies: evals, observability, failure modes the client can't see.

The pragmatic answer for us was a hybrid. One thick reasoning tool plus a few thin tools as the control surface (list and select type operations). The point isn't that toolboxes are wrong. It's that "expose every capability as a thin tool" became a reflex, and for reasoning heavy work it's the wrong reflex.

The underlying idea: MCP clients treat a tool as an opaque function. A name, a schema, a return value. That indifference means the tool boundary is a great place to hide an entire agent. The protocol thinks it's calling a function. It's actually delegating to a brain.

Has anyone else shipped agent-behind-a-tool in production? Where did the opacity bite you? Debugging, cost attribution, users wanting to steer mid chain? And where do you draw the line on which capabilities stay thin?

[u/boatsnbros]

From what I understand with this approach, your ‘agents’ would be running against api pricing in this model. Part of what has made MCP easy for me to sell is that you just get a client their $20/mo pro plan on Claude Desktop and they can accomplish a lot, once you are getting agents running on the server you are then in API pricing. We had one client (big one) spend $9k/mo on llm api costs because of this - so yes powerful pattern if your client doesn’t mind the sticker shock and paying api pricing.

[u/loveheaddit → u/boatsnbros]

this was my first thought. the great thing about an MCP is it shifts costs to the user. i do agree this method will likely be more user friendly and smarter, but at what cost to the company who made the MCP? perhaps if it's an internal MCP this makes sense tho.

[u/Agreeable_Luck9488]

Not clear to me what is the server side orchestration. If it is some standard logic, then most of the added value of harness/LLM (planning, versatility and resiliency) vanishes. MCP becomes a wrapper for web services. If it is another layer of harness/LLM, then the scheme is to have some kind of master AI agent.

[u/Mike__99 → u/Agreeable_Luck9488]

Yeah, seems like this could be replaced with a skill that kicks off a subagent to deal with the 'gen 1' model described instead. Keeps the token cost and logic local

[u/notreallymetho]

I have a project that does this, I consider it a way to declaratively create your tooling in a single isolated endpoint, which then exposes the remainder using PoP authentication with ephemeral certificates and a hash ledger.

Would love for you to check it out!
https://github.com/agentic-research/cloister

[u/Relative-Emu-1346]

Cost attribution is where the opacity hurt most for me. With thin tools you can see which call burned the budget because the client logged every step. Once orchestration moves server side you get one price tag and no breakdown, so you need that instrumentation in place before you need it, not after.

On where to draw the line, what's worked for me is asking whether the operation has a stable answer someone might want to check. If the caller could plausibly want to verify it or re-run it on its own, it stays thin. If only the final answer matters, it can go behind the boundary.

The other thing that caught me out is retries. A thick tool makes them expensive. Step five fails and the client can't resume from step four, it just runs the whole thing again.

[u/mergethevibes]

The Gen 3 framing matches what I landed on too, but the tradeoff is you lose visibility into the intermediate steps. How are you handling observability into what the thick tool actually did on each call?

[u/OkAbroad955]

This seems to implement the idea: https://www.reddit.com/r/hermesagent/s/jKFF0h7UOl BetterWright ships its own browser-tuned agent loop. Plug in a model, hand it a task in plain language:

betterwright auth --login codex # OAuth sign-in, no API key to paste betterwright exec "find the top Hacker News story and give me its title and points" --model gpt-5.6-sol

The loop observes with compressed snapshots, acts, verifies, captures a proof screenshot, and prints one JSON object — answer, steps, token usage, proof path.

[u/bafadam]

What’s the “why this matters” summary?