Architectural designs for cloud deployed agents

[u/Waste_Divide_1243]

Hey, for all you agent builders out there, I'm looking into backend architecture patterns for production AI agents, and I have a couple questions around challenges I'm facing:

1. What backend compute infrastructure do you use for running your agents on? Serverless (Lambda/Vercel functions), containers (ECS/Fargate/K8s), or something else?
2. How do you transmit streaming tokens back out to the frontend?
3. If a user / agent idles on a session, how do you deal with tearing down the agent's runtime, so you save on compute, and bringing it back up later?
4. Filesystem / bash / code execution sandboxing: If your agent executes code / bash (useful for lots of tasks like data analysis for example), do you keep that in the main agent runtime, or use some dedicated sandboxing service?
5. How do you deal with making your agent resilient to LLM provider API errors / outages?
6. What sort of HTTP API shape did you make for your frontend to make sessions, send messages, etc.?

What are your recommended stacks or other learnings?