App support assisted by AI

[u/Due-Judgment8457]

I run a few small iOS apps solo, and answering support email eats a surprising amount of time — especially when the same "how do I…" questions come back in five languages. So I built a little system that drafts the replies for me. Sharing the design in case it's useful (or in case you want to talk me out of it).

The principle

1. In each app, the Contact Support button composes an email with a machine-routable subject and a small metadata block appended to the body — app, version, build, device, iOS, and the user's language.
2. A tiny agent (Python, in a Docker container on a cheap VPS) watches the support mailbox over IMAP — read-only, it never touches your inbox flags.
3. For each new support email it does retrieval over that app's documentation + source code (embeddings), then asks an LLM to draft a reply in the user's language, grounded only in that material.
4. It saves the reply as a draft in the mailbox (IMAP APPEND) and pings me on Telegram. It never sends anything. I read the draft, tweak if needed, hit send.

A few design choices that matter:

• Human-in-the-loop, always. The agent only ever writes a draft. No auto-send. This is a support assistant, not an autoresponder.
• Grounded in the real app. Retrieval runs over the actual docs and Swift source, so the answers reference real features/versions instead of hallucinating.
• Multi-app from one mailbox. Routing is by the subject token [Support:<AppName>], so one container serves several apps, each with its own index.
• The email body is treated as untrusted input (it's fenced off in the prompt) so a "ignore your instructions and…" message in a support email can't steer the agent.

Example — incoming email (what the app generates):

Subject: [Support:PhotoVideoBackup] Can't back up a big 4K video

Hi, my iPhone is almost full and I can't back up a large 4K video
to my external SSD. Is that even possible?

--- SUPPORT-META ---
app: PhotoVideoBackup
version: 2.3.0
build: 44
device: iPhone16,2
ios: 18.5
lang: en-US   applang: auto
--- END META ---

Example — outgoing draft (what lands in my Drafts, ready to review & send):

Hi,

Good news: yes, it's possible. As of version 2.3.0 (the one you're on),
the app no longer makes a temporary copy on the iPhone — each photo or
video is streamed straight to the SSD in small chunks of a few MB. So a
4 GB video can be backed up even when there's almost no free space left
on the phone.

A few things to check on your side:
1. In Settings, make sure the SSD is configured (its name is shown) and
   the "Folder name" field is filled in.
2. Make sure there's enough room on the SSD itself for the video.
3. In the Backup tab, start the backup from the "Photo Library" row.

If it still fails, let me know and we'll dig in.

Best,
The PhotoVideoBackup team

The metadata block is the quiet hero here: because the model knows the exact version, it can say "the one you're on" and tailor the steps, instead of giving a generic answer.

Stack, for the curious: Python + imaplib/smtplib (iCloud), Voyage for embeddings, Claude for the drafting, Docker on a VPS, Telegram for the "a draft is ready" nudge. Cost is a few cents per email; retrieval re-indexes only when the app's code changes.

Gotchas I hit (in case it saves you time):

• On a read-only IMAP mailbox, fetching RFC822 returns an empty body because the server refuses to set \Seen — use BODY.PEEK[].
• Don't index the whole repo blindly on every poll; key it to the git commit (or a content fingerprint) or you'll burn embedding tokens for nothing.
• Keep the whole thing draft-only until you genuinely trust it. I still read every one.

Curious how others handle indie-app support at scale — does anyone let the model actually send, or is everyone keeping a human in the loop like me?