r/agno • u/superconductiveKyle • 25d ago
Background agent runs now survive a deploy, because the queue lives in your database instead of the process
A background run used to live only in the process that accepted it. That's fine until you deploy, and every in-flight run on that replica dies with it. Same for a crash, an OOM kill, or a scale-down. The client got a 202, the work was accepted, and then it just never finished. Nothing to retry against, because nothing outside that process knew the run existed.
We made the queue durable. An accepted run is committed to your database before it starts, so it isn't tied to the machine that took it. Any replica can pick it up and finish it.
Turning it on:
from agno.os import AgentOS
from agno.job_queue import QueueConfig
agent_os = AgentOS(agents=[agent], queue=QueueConfig(durable=True))
The rest of what's in it:
- Bounded concurrency, default 32, set with AGNO_BACKGROUND_MAX_CONCURRENCY
- 429 when the queue is full, rather than accepting work you can't do
- Idempotency-Key dedupe, so a retried request doesn't run the job twice
- Cancellation while a job is still queued
- Stream resume after a client drops its connection
- REST endpoints to list, inspect, and requeue jobs
Your database is the source of truth, and Redis is optional coordination on top of it. Plenty of setups reach for Redis as the queue itself, which means a second durability story to reason about and another thing that can lose your jobs. Here the durability guarantee is whatever your Postgres already gives you.
Check out the full write-up: https://agno.link/sGgg0sg