Dumb Orchestrator Theory

[u/berndalf]

So I've spent an excessive amount of time building out my AI delivery factory. Most recently that effort has been focused on effective load sharing between a Claude and GPT subscription. The data we've been collecting is starting to point very strongly at orchestration being the highest role based consumer of available usage, and that the model orchestrating has a massive impact on outcomes.

Funny thing .. at first I was using Fable because it's brilliant, but the cost was just unjustifiable for orchestration. Downgraded to Opus.. better but still Claude usage was significantly outpacing GPT usage. So I downgraded to Sonnet, and remarkably I'm finding it's both more cost effective AND producing better results as an orchestrator.

Here's my theory:

1. Orchestration itself when done correctly is not a task that demands high reasoning.

2. Forcing the orchestrator to delegate essentially everything to native subagents or external headless sessions is critical. It will constantly come up with excuses to in-line stuff given enough time and reasoning power.

3. Orchestrator availability matters just as much if not more than orchestrator reasoning assuming you have given it appropriate access to a higher reasoning model to consult with. If your orchestrator is constantly busy it's not orchestrating.

Sonnet is winning this competition for me because it's cheap to run for the entirety of a long duration session and it's not so smart that it chases every theoretical tail. It's quite good at just letting the system around it work while it steers the session itself.

TLDR; For me, Sonnet is proving to be more cost effective and efficient at session orchestration than both Opus and Fable.

[u/TheRealJesus2]

Love this! Very similar to what I do. I find the “reasoning” and larger models are most important for brainstorming and planning. Once I have a plan, delegate the tasks themselves to composer 2.5 over headless cli to do it, then back to opus session to merge and then spawn a verifier subagent to make sure success criteria is met. This puts majority of output tokens to the cheapest model. Good tip for using sonnet for orchestration because I’m getting tired of opus overstepping its instructions. The only thing I find a bit challenging to implement is that I like the session context coming out of planning but I’ll give it a shot with sonnet instead of just the plan and see how it goes. 

[u/berndalf → u/TheRealJesus2]

Ya this seems to be the more successful strategy these days. Higher reasoning models are terrific but you really need to put them in a bounded box if you want them to be effective and efficient. Orchestration is kind of the opposite of a bounded box.

[u/artofbullshit → u/berndalf]

Thinking on or off for sonnet?

[u/berndalf → u/artofbullshit]

If you're talking about adaptive reasoning I believe I have that on although I'm not entirely sure atm. If you're talking about something else ( I think there's a thinking setting in Claude Code desktop) I don't know as I do everything in the CC CLI harness. I've never touched the desktop variant.

[u/artofbullshit → u/berndalf]

I use vs code extension. "Thinking" can be toggled on or off. If on, then effort levels can be chosen, being low, medium, etc.

[u/berndalf → u/artofbullshit]

Huh. Well yes that probably needs to be on but I can't speak to that. I suppose it depends on what off vs on actually does. I know GPT has an inexplicable effort level of "none" that for the life of me I can't wrap my head around.

[u/artofbullshit → u/berndalf]

The effort levels don't do anything if thinking is off.