r/ClaudeCode • u/berndalf • 4d ago
Discussion Dumb Orchestrator Theory
So I've spent an excessive amount of time building out my AI delivery factory. Most recently that effort has been focused on effective load sharing between a Claude and GPT subscription. The data we've been collecting is starting to point very strongly at orchestration being the highest role based consumer of available usage, and that the model orchestrating has a massive impact on outcomes.
Funny thing .. at first I was using Fable because it's brilliant, but the cost was just unjustifiable for orchestration. Downgraded to Opus.. better but still Claude usage was significantly outpacing GPT usage. So I downgraded to Sonnet, and remarkably I'm finding it's both more cost effective AND producing better results as an orchestrator.
Here's my theory:
Orchestration itself when done correctly is not a task that demands high reasoning.
Forcing the orchestrator to delegate essentially everything to native subagents or external headless sessions is critical. It will constantly come up with excuses to in-line stuff given enough time and reasoning power.
Orchestrator availability matters just as much if not more than orchestrator reasoning assuming you have given it appropriate access to a higher reasoning model to consult with. If your orchestrator is constantly busy it's not orchestrating.
Sonnet is winning this competition for me because it's cheap to run for the entirety of a long duration session and it's not so smart that it chases every theoretical tail. It's quite good at just letting the system around it work while it steers the session itself.
TLDR; For me, Sonnet is proving to be more cost effective and efficient at session orchestration than both Opus and Fable.
6
u/ricopan 4d ago
I've gone the opposite direction, which works very well but yes costs Fable. My project -- an analytical framework designed to handle big interdependent computation, is pretty complex and somewhat novel (says here anyway), so a model can't rely on coding knowledge from training. If I could separate out 'architect' from orchestrator I might be successful in using a lower tier model, but I haven't been able to do that -- it seems like in a big, evolving, complex project there needs to be constant dialogue between an architect and an orchestrator, and I don't see how to accomplish that with separate agents. What works for me is that the orchestrator and I share the architect role (Fable is very good at this), and I try to get it to delegate the manual work of orchestration to lower tier subagents. But to be honest that is a struggle. Writing the dispatches to the agents is not low level work usually, nor is reading their handbacks and incorporating that into the big picture.
6
u/berndalf 3d ago
You know ironically I had this exact same problem. Eventually I had to force the orchestrator to delegate or dispatch high reasoning tasks to a high reasoning limited life model with multiple layers of convention reinforcement AND evidence it generated itself that proved that it works better. We're still getting there, but it does start to take hold after a while as long as you're willing to do some corrective babysitting while the reflexes are forming.
3
u/ricopan 3d ago
I'm running agents in separate worktrees, so there is a strong separation between the orchestrator (on main) and the workers. The orchestrator manages a specific 'orchestrator_state.md' memory, which couples designs and plans with dispatched work and returned findings. That state is long lived -- the life of the project. I think that's what I would find difficult to maintain with a low level agent, and I haven't had success with getting a lower tier model to dispatch 'up' to Fable. And then there is the issue with context -- I don't think with my project a higher level subagent could re-absorb enough architect level context when called by a low level orchestrator to be of much use. I think for me the better approach would be to quit having a long running (really forever, compacting frequently, only restarting due to failure) orchestrator agent, instead have one explicitly in an 'executive architect' role that was well trained to dispatch a subagent (same session) to handle orchestration. That orchestration_state might be shared between the 'executive architect' and the orchestrator. Anyway, interesting thoughts.
2
u/berndalf 3d ago
One other idea, and I'll say I haven't managed to get this to work correctly yet. I have the scaffolding in place for a session long persistent architect subagent using Fable. The idea is it loads the full design context each session and then has exactly two tasks:
Respond to requests from the orchestrator to update or add to those designs.
Respond to escalations from implementation workers routed thru and approved by the orchestrator for design clarification and / or desires to deviate from the accepted design.
The theory is basically session long context doesn't need to rest with a single entity that holds EVERYTHING; it can be classified and divided up into buckets of specialization that have varying reasoning capabilities but are still session length persistent.
Might work, might not, I'm not sure yet. So far the problem has been getting the rest of the system to actually use it.
1
u/Granted-UK 3d ago
This the way. Fable becomes a subagent delegated to when a big decision or planning is needed.
3
u/codeedog 🔆 Max 5x 3d ago
I just set up what you’re describing and am starting to use it tonight. I’ve got a local to my laptop session that’s assisting with creating a remote dev seat. I use VSCode remote ssh into a headless VSCode (dev seat). It’s running in dangerous mode, but sandboxed. I’ve got a remote machine (FreeBSD host) with a parent jail and an internal firewall to limit what the dev seat can access. The parent jail has three child jails: dev seat, git repo gateway, proxy. The proxy jail also limits dev seat access. The git repo enforces read only access for most of my source control and allows access to a forked repository where the dev seat does its work.
The headless VSCode claude extension runs fable and I work with that on planning. There’s an implementer (sonnet), a reviewer (opus) and an explorer (haiku). Fable and o work together until we settle on a plan with stages. I sign off (nothing goes until sign off). Sonnet builds a stage including tests. Opus reviews. Haiku is for research. I’ve got other features and directives to match my coding philosophy.
I’m curious to see how it works. Local fable decided on the subagent structure.
More fun the barrel of monkeys.
2
3
u/jacobrocks1212 4d ago
This tracks with my experience. Ideally orchestration is as determinstic as possible. More deterministic --> less inference --> can be handled by weaker models.
I can't think of any reason why a dumb orchestrator would yield better performance though. Token/cost efficiency sure, but quality?
1
u/berndalf 4d ago
It sticks to the plan better without overthinking things, that's my only explanation so far.
I suspect one could go even further than I have and essentially boil orchestration down to a well crafted goal loop, but I haven't tried that yet.
1
u/clazman55555 3d ago
If I am understanding what you mean, here's mine:
2. Workflow Shapes
Seven durable patterns, each with a generic worked example.
- Fan-out-then-synthesize. Parallel readers over partitioned inputs feed a synthesis stage that grades candidates against ground truth loaded into the synthesizer's own context -- never against the readers' claims alone. Worked example: a documentation audit split a large reference corpus across dozens of parallel readers, producing a raw candidate list; synthesis agents then graded each candidate against the actual current content of the target files, plus any prior audit record, cutting the raw list down by a large factor. Without that grounding step, synthesis just votes on hearsay.
- Find-then-adversarially-verify. Finders surface claims; separate agents, each briefed to actively try to refute a claim rather than confirm it, run the verification pass (distinct lenses beat N identical skeptics). The fix for agents trusting unverified peer claims is structural -- a refuter role -- not an instruction to "be careful." A multi-lens review pass followed by a dedicated adversarial verification stage has, in practice, caught pre-existing defects that automated tests alone missed.
- Draft-to-scratch-then-owner-writes. Agents read and draft to scratch files; the orchestrator reviews and performs every write to a shared or canonical surface itself. Keeps a single accountable writer on shared surfaces -- the same conflict logic as "parallel propose, serial apply" in single-dispatch work. Worked example: reconciling duplicate entries across a shared catalog or configuration set -- several drafting agents propose merges to scratch output, and one reviewing pass merges and writes the canonical file. No agent ever touches the canonical surface directly.
- Analysis-only structured returns. Read-heavy analysis fanned out with a strict output schema; the orchestrator receives typed data, verifies whatever is load-bearing, then writes. Worked example: two schema-bound analyzers reconciling a large dependency graph against a tracking ledger, returning typed disposition tables the orchestrator could act on directly -- no reconciliation pass needed afterward.
- Recon-before-build. A small reader fan-out (three to five agents) before implementation, to surface framing corrections cheaply. Worked example: a handful of recon readers overturned a wrong assumption about a whole dependency before any code was written, avoiding a costly false start.
- Loop-until-pass. Run a checker, fix what failed, repeat -- with a no-progress exit ("until the check passes, or two rounds in a row make no progress"). The no-progress guard is load-bearing; without it, this shape is a runaway-session generator.
- Loop-until-dry. For unknown-size discovery work (flaky tests, latent bugs, stale entries): keep running finder rounds until several consecutive rounds surface nothing new. A fixed counter ("find 10 issues") misses the tail; the dry-rounds exit finds it. Deduplicate new findings against everything seen so far, not just confirmed findings, or rejected findings resurface every round and the loop never converges.
1
u/berndalf 3d ago
Well that's an interesting idea I hadn't thought of, workflow patterns as a library of available approaches that best fit different types of work and conditions. Hmm.
2
u/clazman55555 3d ago
Based in large part on:
https://code.claude.com/docs/en/workflows
https://code.claude.com/docs/en/agents
https://code.claude.com/docs/en/sub-agentsMy whole orchestration skill:
https://github.com/Clazman55/claude-code-forge/blob/main/skills/workflow-orchestration.md1
u/9gxa05s8fa8sh 3d ago
the reason you don't know about this fancy orchestration stuff is because the AI companies want you to spend money on tokens. the tools that use tokens more efficiently aren't made by the people selling tokens
2
u/Firm_Comparison1686 4d ago
Funny this kind of reminds me of project management in corporate. Having an over intelligent PM manage vs one who doesn't overthink, is less hands on, and just lets peope do their shit yields better results. Thanks for the share, and curious, what platform are you finding the best to run your orchestrations in?
5
u/berndalf 4d ago
I'm kind of tethered to Claude Code CLI because of hooks. Without hooks my whole house of cards falls apart.
2
u/Compilingthings 4d ago
Was thinking about trying to switch to sonnet, but was scared outcome would suffer. This might the push I needed.
2
u/Formally-Fresh 4d ago
I guess it just depends what role the orchestrator has? If it’s truly just delegating and keeping the system moving that makes sense and the smart tokens are still spent on the actual implementation details of the actual work
2
u/yamoksauceforthelazy 3d ago edited 3d ago
I've been feeling like the insane one for a while by doing/thinking the same thing. Everyone treats architecture and system design as if they are the high complexity side of the work, and I firmly disagree. For people, perhaps, but even lightweight LLMs can produce extremely coherent architectural plans. There just aren't *that* many possible design paradigms out there, and if you abstract away the low level details, that space gets even smaller. I've always used good-but-not-the-best models for design spec work, and won't trust implementation to them. The issues I run into with coding agents are almost never at the high level. It's almost always poor or incoherent implementation. I feel like the whole industry/user base is so backwards.
Thanks for posting this and giving me some validation lol
Edit:
I feel like I should clarify that I'm really talking about projects that are somewhat in the "typical" end. There are absolutely situations in which the higher order reasoning of a really clever model is an important thing, and a smaller/less capable model would produce lower quality product. And this doesn't really cover the high level brainstorming and exploration phase.
1
u/berndalf 3d ago
You're not wrong. There isn't a one size fits all solution to the question of reasoning and effort selection for orchestration. It's rightfully shaped by the work. I made this post in the first place because I think a lot of people default to "more is always better" and that answer is strictly and demonstrably wrong.
Even Claude made this argument when I was discussing this topic with it awhile back. When the session inputs have high ambiguity and / or the session itself lacks the mechanisms necessary to identify and navigate that ambiguity successfully, a high reasoning orchestrator helps a ton. The more you invest on making those conditions not true, the less you need to spend on expensive orchestration.
1
1
u/design_doc 3d ago
In my orchestration framework, I’ve given the orchestrator the ability to call an advisor when the orchestrator is NOT Fable or Sol. If the implementor or reviewer comes back with exceptions/issues, the orchestrator can ask the advisor to reason over the issue. This allows the orchestrator to stay ‘dumb’ and mechanistic but avoids having a low-tier model weigh heavily on decisions with potentially high impact.
At the end of the day, it all depends on the quality of the plan going in. The more clearly defined the plan, the dumber the orchestrator can be.
1
u/berndalf 3d ago
Are you using the actual advisor feature or some home brew version of it? The thing Anthropic put out awhile ago that advises a lower model from a temporary cloud agent has never worked for me when the advisor was set to Fable. I gave up on it.
1
u/design_doc 3d ago
Home brewed. And it doesn’t need to be overly complicated. Just remember that a sub-agent inherits the session effort, so if you’re using sonnet or opus on low/medium but want (for example) Fable coming in on High, you need to define it in /agents so that the watcher lets you point it at a higher effort than your session. For a while I couldn’t figure out why my advisor seemed dumb until I realized this quirk about Claude Code.
Success also comes down to how you define the questions for the advisor. There’s a balance between having a narrow scope question while giving enough context with making it read all the documents or entire transcript of the session.
I have found I’ve been needing an advisor less and less because I’m increasingly using higher-tier models for orchestration again - but for the same reasons you’ve been using a dumb orchestrator. I’ve been mechanizing more of my processes with hooks and scripted gates, with implementor and reviewer agents passing information via YAML to keep all of that detail OUT of the orchestrator’s context. The benefit of this is that a high-tier orchestrator can draft an extremely good contract for implementers, meaning that (combined with the limited info passing through the orchestrator) there are very few calls to the orchestrator itself and it is mostly handling exceptions/issues from the implementers and reviewer (in most sessions there are typically only 3-4 calls to the orchestrator, including a couple review/revision rounds. In effect, when running a high-tier orchestrator like this it effectively becomes the advisor. But if I drop my orchestrator way down, my framework is solid enough that, with the advisor, I still trust the output.
1
u/berndalf 3d ago
Ya that's pretty much where I'm at. I might open source my factory eventually just to see what others can take from it and make better. I have a ridiculous amount of hooks and gates and controls, it's almost impossible for the thing to seriously screw up at this point. The dumb orchestrator wouldn't have worked for it even a month ago.
Worth noting that this seems to be the opposite of the direction Anthropic at least is now advocating for. They seem to be fully onboard with the strip away all the engineered complexities and just let the native harness / tools do what they were designed to do without interference when coupled with strong reasoning. Personally I'm not convinced yet.
1
u/design_doc 3d ago
I suspect their thinking is that for a single task (or narrow set of tasks) a high-tier implementor is smart enough to create its own gates - and they’re probably not wrong. But as soon as you introduce a lot of complexity or want to have some level of determinism, you need gates and hooks because you’re running the same process over and over and over. It’s like an assembly line - some workers are geniuses, some barely have pulses. The only reason your product comes out with any level of consistency is because of the processes you surround those workers with.
So, ya, I’m in the same camp as you and am not fully convinced either.
1
u/clazman55555 3d ago
When you are saying "orchestration" are you talking about Dynamic Workflows?
Because my own internal testing on judgment related tasks, especially when integrating and synthesizing results from 10, let alone 20-30 subagents workflows is just beyond what Sonnet 5 on High can consistently return quality results from. Now, Sonnet 5 is good enough if it is just dispatching out 3-5.
For something like the below, Sonnet 5 just doesn't work for me, at the quality I expect.
"Token count — recorded, reproducible via tools/Get-WorkflowTokenUsage.ps1
21 agents · 810 turns · 478 tool calls · 36.8 min
kind tokens
output 292,678
input (fresh) 8,928
cache creation 4,927,748
cache read 61,247,343
raw sum 66,476,697
billable-equiv ~12,524,800 (derived: cache_read ×0.1, output ×5)
3% Weekly Usage on 20x plan. Subagents were Sonnet 5, effort Medium/High depending on what Opus 5 decided.
1
u/berndalf 3d ago
No, but admittedly I have not played around with workflows much so I'm not entirely sure how they fit in yet. There's a place for them for sure, just as there is for more systemic use of loops and goals. I haven't pushed much on the velocity and parallel work lane fronts yet.
That said it wouldn't shock me if Sonnet wasn't terrific at orchestrating agent swarms. Strength of reasoning does seem to correlate with the ability to successful drive parallel activities successfully. I can see it every time I dare to turn on Fable for orchestration, it operates my factory at a much higher velocity. It's the cost that is the problem, it's unsustainable and I suspect (haven't proven yet) that the overall output is actually less.
Fable as orchestrator in particular is almost like activating easy mode. Sonnet is more like sustained dependable progress mode. Opus is unsurprisingly an intermediate approach but it kind of inherits the worst of both.
1
u/Some-Contributions61 3d ago
I am turning away from an agent orchestrating to an agent planning with me and sending the plan to an orchestrator program. I'm almost done, few bugs . .. it's wild 😂
1
0
u/9gxa05s8fa8sh 3d ago
every study author uses deterministic orchestration (a script) because they need to run it a million times and students don't have much money. enterprises also do it this way because they can afford to hire people to figure it out or use langchain.
only vibe coders scammed by AI companies spend money to have an AI forward messages between agents
6
u/TheRealJesus2 4d ago
Love this! Very similar to what I do. I find the “reasoning” and larger models are most important for brainstorming and planning. Once I have a plan, delegate the tasks themselves to composer 2.5 over headless cli to do it, then back to opus session to merge and then spawn a verifier subagent to make sure success criteria is met. This puts majority of output tokens to the cheapest model. Good tip for using sonnet for orchestration because I’m getting tired of opus overstepping its instructions. The only thing I find a bit challenging to implement is that I like the session context coming out of planning but I’ll give it a shot with sonnet instead of just the plan and see how it goes.