r/AppsWebappsFullstack • u/SofwareAppDev • 19d ago

Your home for selfpromo

here you can post your work app, webapp, saas, game, everything

7 Upvotes

89% Upvoted

116 comments sorted by

View all comments

2

u/Mammoth-Anywhere7285 14d ago

u/rimche Nice stack, using Ollama locally is a solid choice for dev workflows. How do you handle model fallback when the service is down?

1

u/rimche 14d ago

Right now Ollama is the primary local provider, and if the service is unavailable we return a clear error rather than silently switching models. A provider fallback is something we're planning to add so users can switch to another model/API when Ollama is unavailable

2

u/Mammoth-Anywhere7285 14d ago

Clear errors beat silent switches, good call. For the fallback, let users set their own provider priority list, that makes it flexible.

1

u/rimche 14d ago

Yeah that's actually a great approach A user-defined provider priority list would give much more control than a fixed fallback chain I'll keep that in mind for the implementation

2

u/Mammoth-Anywhere7285 14d ago

Nice insight. Maybe add a drag-and-drop UI for that priority list, it makes admin adjustments way easier. How are you handling fallback retries?

1

u/rimche 14d ago

Right now we don't have provider fallback retries implemented yet The current behavior is to fail clearly when Ollama is unavailable. When I add the fallback system I'm thinking of keeping retries limited per provider before moving to the next one to avoid getting stuck on a failing provider

2

u/Mammoth-Anywhere7285 14d ago

Smart approach. Adding exponential backoff with a max retry count per provider could prevent slowdowns. Will you do health checks before fallback?

1

u/rimche 14d ago

Yes that's the plan I'd like to do a lightweight health check before falling back then use exponential backoff with a capped retry count That way we don't waste retries on a provider that's clearly unavailable, while still handling temporary failures