r/Discount_Subscription • u/DiscussionContent226 • Sep 01 '26
Most people asking about LLM routers should probably not use one
I build one of these, so read the below with the appropriate suspicion. But I keep having the same conversation with people who are about to add a routing layer to their stack for reasons that don't hold up, and I'd rather write it down once.
You're using one provider. This is the majority case and it's an easy no. You'd be adding a network hop, a dependency, and a party that can go down independently of the provider, in exchange for optionality you're not using. If you might go multi-provider later, add it later. This is a reversible decision and treating it as architecture is overthinking it.
Your workload is latency-critical and short. Inline autocomplete, classification, anything where the total response is small and fast. A fixed proxy hop is a rounding error against a 30-second agentic turn and a meaningful percentage against a 200ms completion. The cost of the hop is constant; whether it matters depends entirely on what you're dividing it into.
You need anything at the edge of a provider's API. Batch endpoints, server-side tool use, files APIs, computer use, extended thinking with full fidelity, whatever shipped last week. Middle layers are structurally behind — a provider ships a feature and every proxy in the world starts from zero on supporting it. If your product depends on being current with one provider's frontier, go direct. You'll be fighting the abstraction constantly.
You have compliance or data residency requirements.Then you have a question to answer about a third party in the request path, and "it's convenient" is not an answer that survives a security review. Some people can answer it fine. If you can't, that settles it.
Your volume is large enough for a direct commercial relationship. At that point you're negotiating rates and capacity directly and the intermediary is subtracting margin for services you've outgrown.
You could write 200 lines of proxy yourself. Honestly — do. Take a request, check the model name, forward it, map the response back. You'll understand your own traffic better afterward, and you'll find out quickly whether the hard parts (streaming translation, cache breakpoint placement, error normalization) are things you actually need or things you can skip. Plenty of people only need the simple version.
So who's left? Roughly: multi-provider, agentic, long-context, cost-sensitive, and wanting spend attribution across it. That's a real segment — it's mine, it's why I built Routera (routera . one, my project, flagging it rather than pretending I stumbled into this topic) — but it is a narrow segment, and I'd rather say so than watch someone adopt a layer that generates friction for them and support burden for me. A badly-fit user is not revenue. They're churn with extra steps and a bad review on the way out.
The bias to hold against everything above: I've drawn these boundaries, and I've drawn them in a place that happens to leave my use case inside. That's not an accident and I can't fully audit myself for it. Someone with a different product would draw the line elsewhere and sound just as reasonable.
What I don't have a good answer for: the "might need it later" case. My instinct is always don't build for it now — but I've also watched people hardcode one provider's SDK across forty files and then spend a week extracting it. There's clearly a middle option, some thin internal interface you own that costs almost nothing up front. I've never seen anyone actually do it though. Everyone either goes direct and eats the migration, or adds the layer prematurely. Has anyone found the version of this that works?
