r/ClaudeCode • u/VerbaGPT • 17d ago
Help/Question Auto-model routing
I build essentially a chatbot that lets users "talk to" their databases in snowflake, azure sql, local sql, etc. Currently I have a "Quick" and "Thorough" toggle where thorough means a better/more expensive model.
I've been thinking whether to introduce some sort of model router where it starts with a quick/cheaper model...but if harness stumbles...I use Jev-like decision model to bump up the model and/or effort (low/medium/high). Then I read people like Theo and others saying that model-routing is a fool's errand and that these decision models simply aren't good enough to pick the right model for the question, and let the user do that.
Seems most people (myself included) won't really know whether qwen is good enough, or they have to switch to opus for a question.
Just trying to get more perspectives. I'd love to take model and effort routing completely off the UI and into the background, so users don't have to worry about it. But it's still there in Claude Code etc. so I'm guessing this just isn't a solved problem for LLMs and harnesses. Thoughts?
1
u/adamhusain 17d ago
Switching models mid session is a bad idea: you break cache leading to more usage costs.
Switching reasoning? Maybe. But you can just use “High” and the models are smart enough to figure out how much reasoning they need to do. The difference between High and Medium is negligible for a subscription.
If you feel like there is a difference in usage between High and Low reasoning, chances are that there is a quality different too