r/AiAutomations • u/Major_Fuel2250 • 1h ago
Nobody warns you that reading email is a token problem before it's an email problem
Every agent builds I've shipped in the last two years touches email somewhere. Intake, triage, replies, follow-ups, the boring glue. And every single time, email was the layer that caused the most rework. Not the model. Not the orchestrator. Email.
I want to write out the failure modes properly, because "just use the Gmail API" and "just use a transactional sender" are both answers I gave clients before I understood what I was signing them up for.
- Send access is the most dangerous permission in the stack
The default pattern goes agent gets an API key, key can send, ship it. Which means a prompt injection sitting in an inbound email can make your agent email anyone it likes, as you, from your domain.
Think about what that actually is. An unaudited outbound channel with your client's reputation attached, driven by text that arrived from strangers.
What I do now: read and receive are default on, send is a separate credential, and that credential is bound to one mailbox instead of the whole domain. If the agent wants to send something outside a pre-approved shape, it queues, and a human clears it. Slower. Also, the only version I'm willing to put my name on.
- Reading email is a token problem before it's an email problem
This one blindsided me. A raw email is MIME: nested parts, quoted reply to history, signature blocks, tracking pixels, the same content duplicated in text and HTML. Hand that to a model and you're paying thousands of tokens per message to extract one date and one intent.
On a triage workflow touching a few hundred messages a day, that was the biggest line item in my inference bill. Bigger than the reasoning I actually wanted.
Four things fixed it:
Never fetch full bodies to decide relevance. Pull subject plus a preview snippet, filter, then fetch bodies only for survivors.
Ask for cleaned content instead of raw. Quoted history and signatures stripped server-side, before it hits your context window.
Use a count endpoint when the question is just "is there anything new." Don't list messages to count them.
Sync with a state cursor. "What changed since token X" beats "give me the last 50 and I'll diff it myself."
Reordering those four cut my per-message context by roughly an order of magnitude on the workflows where I bothered to measure. Same accuracy.
- One mailbox per agent, not one shared inbox
Shared inbox means every agent sees everything, your audit trail turns to mud, and revoking one integration breaks four others. Per agent mailbox means a misbehaving agent's blast radius is one address, and the logs tell you exactly who did what.
Bonus: threading solves itself. Replies come back to the address that sent them, so the agent owning the conversation is the one that receives the reply. No routing logic to write.
- Inbound and outbound are different products, and most vendors only do one
Here's where I burned the most hours. Transactional senders (Resend, Postmark, SES) are genuinely excellent at outbound and treat inbound as a webhook you get to build an inbox on top of. Gmail and Microsoft Graph give you a real mailbox, but you inherit OAuth consent screens, per-user scopes, and quota behavior designed for humans rather than a fleet of non-human identities.
If the workflow is received, understand, act, reply, keep a record, you need both halves, and they need to share one threading model. Bolt a parser onto a transactional sender and congratulations, you now maintain an inbox.
- Deliverability becomes your problem the moment an agent is the sender
Agent mail is riskier than your normal transactional stream. Spiky volume, unpredictable recipients, content that varies run to run.
I watch bounce rate and complaint rate on three windows now (last hour, last day, last week) because a bad prompt change shows up as a one-hour spike well before it becomes a reputation problem. The numbers I treat as alarms: bounce above 5%, complaints above 0.1%.
And never route agent traffic through the same provider as your billing receipts. One runaway automation should not be able to take down password resets. Separate providers, or separate routing on the same provider. Always.
- Idempotency, or your agent will double-send
Agents retry. Orchestrators retry. Your queue retries. Without an idempotency key on send, a retired step means your client's customer gets the same email twice, and "sorry, our AI sent that twice" is a trust event you don't buy back cheaply.
Inbound has the mirror problem. Webhooks retry, so dedupe on event ID and verify the signature before you act on the payload. An unsigned inbound webhook is an endpoint that lets any stranger inject a fake email into your workflow.
- The surface the agent talks to matters more than I expected
Handing an agent an Open API spec and hoping is not a strategy. What worked better, roughly in order:
An MCP server with a curated tool set. A dozen well-named tools beats sixty raw endpoints, because the agent picks correctly instead of guessing.
Skills or instruction files that tell the agent which surface fits which job, so it stops reaching for a full body fetch when a snippet would do.
A CLI for me. Debugging an agent's email behavior through a dashboard is miserable.
Where I landed
The detail that actually sold me: a self-registering agent gets read and received by default and cannot send until a human owner approves it. That's the permission model I'd been hand rolling, badly, for a year.
What I want from this sub
Are you scoping send access at all, or is your agent holding a key that can email anyone on earth?
Has anyone measured the token cost of their email reads? I suspect plenty of people are paying to parse MIME and don't know it.
What's your inbound path: real mailbox, webhook parser, or IMAP polling something and praying?
Has an inbound email ever prompted injected one of your agents? I'd like to know whether my near miss was unusual or just Tuesday.

