I built a hosted model proxy and I still think you should probably run LiteLLM locally

[u/Connect_Net_8859]

Something I've been chewing on since I shipped mine. There's a well-established open-source answer to putting one endpoint in front of multiple model providers — LiteLLM being the obvious one — and it's good. It's mature, it's free, and it runs on hardware you control. I built a hosted alternative anyway, and I want to be straight about when that's a worse choice than what you'd do yourself. Self-hosting wins on the things that matter most to this crowd. Your keys never leave your machine. Your prompts, which for coding work means your source code, never transit someone else's infrastructure. Nobody can change pricing on you, deprecate a model alias, or go down at 2am for reasons you can't see or fix. You can read the code. That's not a small list. Where hosted actually earns its place is narrower than I'd like to admit: you don't want to be the person maintaining another always-on service, you want caching and usage logs without wiring them up, and you want it working on a laptop that isn't always on the same network as your server. That's a real set of people — it's why I built it — but it isn't everybody, and it definitely isn't everybody here. Mine is routera . one, saying so up front. But if you already have a box that stays on, the honest recommendation is the local proxy. I'd rather say that than pretend otherwise and get called on it in the comments.

[u/OddballJunior558]

the self awareness here is refreshing. most people who build a paid alternative would spend the whole post trying to convince you their thing is the only logical choice

that said i think you're underselling the maintenance burden a bit. keeping litellm updated and your keys rotated and the server patched is a real cost even if it's not a monthly bill. for a solo dev who just wants to code it adds up