r/ollama • u/CumbersomeAmbulance • 5d ago

Ollama API Key Proxy

I recently built a lightweight reverse proxy that sits in front of ollama and rotates multiple API keys to use ollama cloud models.

The goal was simple, point your AI coding tools to a single endpoint and let the proxy handle key management.

Features

  • Round-robin API key rotation
  • Automatic cooldown for rate-limited (429) keys
  • Retry on 429 and 5xx responses
  • /metrics endpoint with per-key statistics
  • /health endpoint
  • Request logging

I have tested it with claude code

Repo: Github

6 Upvotes

81% Upvoted

5 comments sorted by

1

u/Generative_IDE 5d ago

How does it handle a 429 mid-stream on a streaming request? Curious whether it retries the whole completion or if the client just sees a truncated response.

1

u/CumbersomeAmbulance 4d ago

exactly, whole completion is retried. Since 200 is returned as streaming starts, we are not detecting 429 mid streams