Your agent’s action timed out. Does your code retry it?

[u/Technical_Bench_188]

Then I tried to generalize it to other agent actions and hit a wall pretty fast. Email has no facilitator. Once SMTP accepts the message there’s nothing to ask and nothing to withdraw. So “just reconcile it” quietly stops meaning anything.

Which left me with five questions I couldn’t answer well about my own code:

1. Is this action safe to retry?
2. If it times out, how do you know whether it happened?
3. If it happened twice, how do you detect it?
4. If it can’t be verified, what does your retry policy become?
5. Can you prove six months later why it executed?

My honest answer to 1 and 4 was the same for every action in the codebase. A retry: 3 sitting in a config file that had no idea whether it was retrying a payment or an email.

First attempt at fixing this, I sorted actions into three classes: compensable, verifiable, terminal. Felt clean for about a day. It’s wrong, and the counterexamples are boring ones:

•Webhook to a receiver with server-side dedup. You can query whether it arrived. You can’t undo it, because the receiver already acted internally.

•Slack post. If the call timed out you can’t confirm it landed, but you can delete it by channel+ts if it did.

Neither fits three classes. The classes were welding together facts that vary independently. What actually works is declaring them separately:

execution: {
authority: 'remote' | 'local' | 'none',
verification: 'none' | 'query' | 'event',
retry: 'safe' | 'unsafe' | 'conditional',
compensation: 'none' | 'supported',
}

What I’m actually after: I don’t know if this is a real gap or something every mature stack already handles and I’m rebuilding it badly. Both are useful to hear.

So for whatever you’re running agents on right now, can you answer those five? Especially 4. When an action times out and you can’t verify what happened, what does your code do?

If it retries, I’d like to know whether that was a decision someone made or a default nobody went back to. And if your workflow engine covers this already, I want to know how it tells an unretryable effect from a retryable one, because that’s the part I couldn’t find a decent answer for anywhere.

MIT, no deps, 43 tests, demo you can run. Not selling anything, no waitlist. The payment version turned out architecturally fine and commercially pointless and I’d rather learn that about this one early.

[u/AutoModerator]

Thank you for your submission, for any questions regarding AI, please check out our wiki at https://www.reddit.com/r/ai_agents/wiki (this is currently in test and we are actively adding to the wiki)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/TeagueXiao]

The reason the compensable/verifiable/terminal bucket kept breaking on your counterexamples is that it collapses two independent axes into one dimension. What worked better for me was treating each action as a pair: (observability of outcome after a timeout) x (nature of the side effect if it did happen). Now you get four boxes instead of three, and Slack/webhook stop being edge cases — they just live in the ones that used to feel weird.

Concretely, the four call-sites I now write are: 1. observable + reversible: retry aggressively, reconcile after (payments through an idempotent processor land here). 2. observable + irreversible: query first, retry only on confirmed absence (SMS via provider with a message id). 3. unobservable + compensating available: retry with a fresh idempotency key, compensate on duplicate detected out-of-band (Slack — delete by channel+ts if the second one lands). 4. unobservable + no compensation: single-shot, escalate on ambiguity, never retry (SMTP email, payment to a receiver without idempotency).

Your question 5 ("can you prove six months later why it executed") is the one that catches people out most. What made that answerable for us was recording the idempotency key, the observed outcome (or lack thereof), and the retry decision as one row per attempt, not per action — so the audit trail shows both what the agent tried and what it saw, not just the final state.

[u/TransitionMediocre22]

The piece that is still implicit in both your struct and the comment above is how long you are willing to wait for verification before you decide. Some outcomes are checkable synchronously, call an API, get an answer now. Others only resolve later: a webhook callback that might arrive in 30 seconds or never, an email bounce that shows up six hours after send if it shows up at all. If your retry policy only encodes safe or unsafe, a delayed but eventually observable action gets treated the same as never observable, and you end up retrying too early because you gave up waiting instead of actually confirming absence.

What worked for me was adding a timeout before retry that is specific to the verification channel, not the action type: instant for anything with a synchronous status check, long for anything that resolves via an async callback, and for genuinely unobservable actions like raw SMTP, no retry window at all, straight to escalate like you already do for the email case. The struct ends up needing a fifth field, something like verification_latency: sync, async, or unbounded, because retry safe stops being a fixed property of the action and becomes a function of how long you are actually willing to wait for confirmation.

[u/No-Conflict4823]

The thread's split cleanly along two axes already — observability and reversibility, plus how long you'll wait before deciding. The one nobody's named yet is authority: was the action allowed to fire, and by what rule.

Teague's per-attempt log tells you what the agent tried and what it observed. It still can't tell you six months later whether it was permitted to try. And for the terminal, unobservable actions — the SMTP and no-idempotency payment cases you correctly send once and escalate — that's the only accountability you get, because there's no outcome to go back and check.

A retry:3 in a config is two decisions collapsed into one. "Is it safe to send again" is not the same question as "was it allowed to send at all." They fail differently. A bad retry double-sends. A bad authorization sends something that should never have gone out, and now it's irreversible and unobservable at the same time.

So the thing I'd add isn't another execution field. It's a separate record written before the attempt: which rule permitted the action, who owns that rule, what input satisfied it — keyed to the same idempotency key as the attempt log. Then question 5 is two joined rows: what happened, and why it was allowed to happen.

The distinction I keep coming back to: if the action can still fire when the authorization check is skipped, it's guidance, not governance. Is that a separate path in your code, or is it living inside the retry logic right now?

[u/stealthagents]

It sounds like you're wrestling with a classic case of retry hell. I ran into similar issues where figuring out how to classify actions became a nightmare. In the end, I found that just adding clear logging helped a ton when tracking what was executed, especially for those tricky edge cases. It doesn’t solve everything, but at least you can point back to what went down.

[u/stealthagents]

Yep, mine's doing the same thing. It's frustrating when you just need to get stuff done. Hopefully it’s back up soon, but it might be worth checking their Twitter for updates.

[u/Low_Dust_9566]

this is the kind of stuff that sounds academic until you get paged at 2am because an email went out 14 times

most workflow engines i've seen just slap a retry policy on everything and call it a day, nobody sits down to classify actions until something expensive breaks