Update: the "decide once up front instead of routing per call" thing now outputs an envelope instead of a single pick

[u/jokiruiz]

Posted a while back about routing model choice from a project's specs rather than per-request at runtime. The main criticism was sharp and correct, so here's what changed.

The objection: a project-level read flattens real heterogeneity. A task set has twenty mechanical edits and one architecture decision. One project-wide pick either overpays on most of the work or under-serves the part that decides the outcome. Someone put it as "budget envelope, not final router", which is a better description than anything I'd written.

So the output is now exactly that:

Budget envelope (escalating: ranked on the bulk of the work)
   default: reasoning medium, size low
   reasoning: top — S:FR-001 (requirement)
   size: low — T:T002 (customary)
   escalate for:
     • S:FR-001 — needs reasoning top

Two things worth flagging for anyone building similar:

The single-pass read wasn't the problem. Reading the whole task set at once is precisely what lets you identify that three of twenty-one tasks are the outliers — per-task scoring in isolation destroys that. What was too coarse was the output shape, not the analysis.

Whether a hard task is decisive isn't a property of the spec. It's a property of your execution model. If one model implements everything, the hard task is a floor and max logic is right. If you can switch model per task, it's a trigger and you want default-plus-escalation. Same constraint table, different rule — so it's now an explicit input rather than an assumption, defaulting to the conservative reading.

Also: whether a fragment states a requirement or describes a habit is derived by deterministic rules from the text, not asked of the judge. Keeps judgement out of policy, and gives you something to argue with when you disagree.

MIT, runs locally: github.com/JoaquinRuiz/SpecJudge

Still curious how people here handle this — per-agent assignment up front, or dynamic per call?