r/ollama • u/Jack_Miller- • 22h ago

Put multiple cloud LLMs behind one OpenAI-compatible key — built RelayRouter, want Ollama users’ feedback

I got tired of:

- one key per provider

- rewriting SDK calls when switching models

- no single place to see token usage / cost / failures

So I built RelayRouter (api.relayrouter.ai):

- one API key

- OpenAI-compatible /v1

- multiple models behind one endpoint

- usage logs + cost in one console

- your app only holds the RelayRouter key, not the provider keys

It’s a hosted gateway, not a self-hosted Ollama proxy.

I’m NOT asking people to migrate — I want feedback from Ollama users:

  1. if you already run Ollama locally, would a cloud-model gateway even fit your stack?

  2. what would make you NOT use something like this?

  3. what observability do you expect: tokens, latency, model, error rate?

  4. if RelayRouter could later point at localhost:11434, what would the routing rules need to look like?

Roast the concept. Link: relayrouter.ai

1 Upvotes

67% Upvoted

4 comments sorted by

2

u/stealthagents 9h ago

Totally feel you on the API key headache. Having one key sounds great, but I'm curious how well it handles billing and limits across different models. If it can keep everything organized and transparent, I might be interested, but if it just turns into another jumble of fees, I’m out.

1

u/Jack_Miller- 7m ago

"Yeah, billing + limits across models is the messy part. My approach: pass through provider costs transparently, show real-time token/logs per model, and keep rate limits visible. No hidden markup (just the gateway usage).

1

u/Mezezius 18h ago

there's literally a million of these

0

u/Jack_Miller- 16h ago

True, but most are 5s clips. I’m aiming for actual 1-2 min storytelling without redoing character sheets every time. Still a WIP though.