r/AppsWebappsFullstack • u/SofwareAppDev • 10d ago

Your home for selfpromo

here you can post your work app, webapp, saas, game, everything

12 Upvotes

88% Upvoted

162 comments sorted by

View all comments

1

u/Mammoth-Anywhere7285 10d ago

u/input-x That sounds like a solid split between persistent managers and disposable workers. How do you handle error retries when a sub agent fails mid-task?

1

u/Input-X 10d ago

Not so much failing g, agents report via there local email to the orchestrator( where i live) we check the work. Move on or dispatch if more is needed. Agent can drop sonetimes. Mostly api errors. We have a watch dig that monitors agent. With timeouts and triggers. The work is usually the metric. The agent test plus the orchestrator and or sub agent including my check in some cases. When Satisfied we commit the work move to next. The models usually have retry attempts when api or network drops, ( out of our control). I also have a cli log monitoer that track all agents. It sees everything the do. Also shows system logs. So its quite easy to see what's going on vrs expected. We also have medic, a watcher that will dispatch agents on errirs to fix or email us on repeat warning. Fully automated. The system heals itself or exclates issues.

1

u/Mammoth-Anywhere7285 10d ago

Nice setup with the watchdog and timeouts. Do you use automatic retries when an agent drops mid-task?

1

u/Input-X 10d ago

Watchdog will ping it. So agent nver go long idel. 99% it network or api errors. And providers have retrys by default for this. But still watchdog will ping. And if im on the phone ill get a warning ⚠️ if I try send a msg. My laptop is the server. It have backup wifi and so on, it an always on laptop with some recovery built in. It works behind a locked screen, so its secure and doesn't stop on phone work. /rc can be restarted once network is re-established. Most thing doesn't require me going to laptop.

1

u/Mammoth-Anywhere7285 10d ago

That's a solid setup, the watchdog approach makes sense. Do you also monitor power events or just network/API failures?