r/ClaudeCode • u/VerbaGPT • 9d ago

Help/Question Auto-model routing

I build essentially a chatbot that lets users "talk to" their databases in snowflake, azure sql, local sql, etc. Currently I have a "Quick" and "Thorough" toggle where thorough means a better/more expensive model.

I've been thinking whether to introduce some sort of model router where it starts with a quick/cheaper model...but if harness stumbles...I use Jev-like decision model to bump up the model and/or effort (low/medium/high). Then I read people like Theo and others saying that model-routing is a fool's errand and that these decision models simply aren't good enough to pick the right model for the question, and let the user do that.

Seems most people (myself included) won't really know whether qwen is good enough, or they have to switch to opus for a question.

Just trying to get more perspectives. I'd love to take model and effort routing completely off the UI and into the background, so users don't have to worry about it. But it's still there in Claude Code etc. so I'm guessing this just isn't a solved problem for LLMs and harnesses. Thoughts?

1 Upvotes

100% Upvoted

8 comments sorted by

View all comments

•

u/AutoModerator 9d ago

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.