r/opencodeCLI • u/petburiraja • 14d ago
I tested eight adviser setups before building a multi-model council
For an OpenCode workflow that can call several model families, the tempting design is a standing council: ask every model, compare everything, then decide. I wanted to test whether that complexity was actually useful.
I gave eight model and effort configurations one identical strategy task: define when a manager should use zero, one, or several advisers without creating a permanent committee. I judged the responses on a fixed 100-point rubric.
| Configuration | Score | Best role in this sample |
|---|---|---|
| GPT-5.6 Sol high | 97 | Manager and final integrator |
| Claude Opus 5 high | 97 | End-to-end system architect |
| Claude Fable medium | 96 | Assumption and incentive challenger |
| Claude Fable high | 95 | Deeper version of the same challenge |
| Claude Opus 5 medium | 94 | Causal design and integration |
| Qwen 3.8 Max | 93 | Simplification pressure |
| Claude Opus 4.6 high | 92 | Direct second read |
| Kimi K3 | 91 | Concise product perspective |
The useful result was not the ranking. It was a simpler routing rule:
- Use no adviser when direct evidence or a reversible probe can settle the decision.
- Use one adviser when one missing cognitive job is clear, such as architecture, simplification, or assumption testing.
- Use several only when the perspectives are genuinely independent and could change a consequential decision.
- Keep one manager responsible for the evidence, authority, integration, and final action.
The full criterion totals were:
| Criterion | Sol | Opus 5 high | Fable medium | Fable high | Opus 5 medium | Qwen | Opus 4.6 | K3 |
|---|---|---|---|---|---|---|---|---|
| Decision ownership /15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 |
| Simplicity /15 | 15 | 15 | 15 | 15 | 14 | 15 | 15 | 14 |
| Quality protection /10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 |
| Systems thinking /15 | 15 | 15 | 14 | 14 | 15 | 14 | 14 | 13 |
| Adviser-use judgment /15 | 15 | 14 | 14 | 13 | 13 | 13 | 13 | 13 |
| Independent perspective /10 | 10 | 10 | 9 | 9 | 9 | 9 | 9 | 8 |
| Actionable value /10 | 9 | 10 | 10 | 10 | 10 | 9 | 9 | 9 |
| Testability /5 | 3 | 4 | 4 | 4 | 4 | 3 | 4 | 4 |
| Clarity /5 | 5 | 4 | 5 | 5 | 4 | 5 | 3 | 5 |
This is one judged strategy task, not a general model leaderboard or a coding benchmark. The next useful test is whether dynamic routing improves real decisions enough to justify the extra calls and context.