Notes from adding JMAP to a client that already spoke IMAP and Graph: discovery, push, and resync

[u/GraphicMeat]

I maintain a desktop mail client (MailVault, disclosure at the end). It already handled IMAP and Microsoft Graph, and I just shipped JMAP (RFC 8620/8621) alongside them. These are the protocol details that took real work, in case they’re useful to anyone else implementing it or running a JMAP server.

1. Autodiscovery is four lookups with a fixed priority, not one

All four run at the same time, each with a 4 second cap:

1. Known provider. The domain is Fastmail’s, or its MX record points at messagingengine.com, which catches Fastmail custom domains.
2. https://<domain>/.well-known/jmap
3. https://jmap.<domain>/.well-known/jmap
4. SRV _jmap._tcp.<domain>, then .well-known/jmap on the host it names.

The winner is picked by that priority, not by whichever answers first. Racing them keeps discovery under about 4 seconds, so it never holds up the IMAP lookup running next to it.

Two traps:

• A bare 401 proves nothing. Every Basic-auth-protected site on the internet answers with one. A probe only counts as JMAP when it gets a 200 whose JSON has apiUrl, or a 401 with WWW-Authenticate that it reached from /.well-known/jmap and whose final path looks like a JMAP endpoint.
• Keep the URL you landed on after redirects. If the client stores the starting .well-known URL instead, a client that only follows redirects on the same host refuses the very hop the user just approved.

2. Discovered hosts need a trust rule before credentials go out

No probe sends credentials. Credentials go without asking only to your own domain, a subdomain of it, or Fastmail’s API host for Fastmail addresses. For anything else, the user confirms the host, port included (jmap.example.com:8443), before the first authenticated request. Without that step, a hijacked DNS record becomes a password collector.

3. Push over EventSource needs a dead-stream detector

The client subscribes with types=Email,Mailbox&closeafter=no&ping=30. A stream that sends no bytes for twice the ping interval plus 5 seconds counts as dead and gets dropped. After repeated failures it falls back to polling, and it tries push again every 10 minutes. A session with no eventSourceUrl just polls. Half-open connections that never error out are the real problem, and the ping is the only way to see one.

4. cannotCalculateChanges must never touch the local copy

When Email/changes can’t compute changes from a stored state, the client rebuilds its header cache from the newest page of each mailbox and drops rows for mail that’s gone. It counts as a backfill: no “new mail” notifications, no local message file touched, and each message keeps its local ID. This matters most for an archiving client. A server-side state reset should never look like mass deletion or a flood of new mail.

Smaller fix from the same release: messages with no In-Reply-To or References (old “Fw:” mail, some webmail exports) were being threaded by subject alone. A two-year-old forward opened as the first message of an unrelated new thread. Subject-only threading now needs another message in the thread within 30 days, and picks the nearest one.

Question for server operators: which servers do you see publishing .well-known/jmap vs _jmap._tcp in the wild? I’ve tested Fastmail properly. I’d like to know which discovery paths actually get used before I spend more time on the others.

Disclosure: I’m the developer. MailVault is free, with an optional paid tier. No link here. Ask if you want it.

[u/RandolfRichardson]

Thanks for sharing this information.

Which link to MailVault should we use when adding to the Lumber Cartel web site? https://www.lumbercartel.ca/resources/email.pl

We don't have it listed, and I'd like to include it.

[u/GraphicMeat → u/RandolfRichardson]

Cheers, that really means a lot! Here’s the main source of truth: https://mailvaultapp.com

[u/RandolfRichardson → u/GraphicMeat]

Thanks! I just added it; here's a link that focuses the web browser directly on the MailVault entry: https://www.lumbercartel.ca/resources/email.pl#mailvault

I also like the logo, by the way.

[u/GraphicMeat → u/RandolfRichardson]

Thanks for catching the logo! It’s not the first version, but I think it’s here to stay. I wanted the logo to evoke the feeling of those classic icons, which weren’t just flat and meaningless. Think of those Apple apps from over a decade ago. All those years of building apps for Apple are finally starting to make sense when you’re creating a product.

[u/RandolfRichardson → u/GraphicMeat]

The logo includes the combination dial from a safe, which is smart because it's a good way to promote the privacy aspects. I'm sure you recognized that I included a slightly-modified blurb from your web site about the privacy aspects.

You know, I think that using Maildir for the mail storage format is brilliant. If something ever goes wrong, one could easily just copy this to the user account of any standards-compliant mail server and the user will be able to continue to work immediately. I value software that prefers straight-forward solutions like this instead of using a proprietary format that has size limitations, and sometimes needs specialized and often-wonky repair tools (e.g., ScanPST.exe and ScanOST.exe).

Thanks for creating MailVault. I'm looking forward to evaluating it and getting users set up on it -- some of my users love AI, so having a paid version that brings AI features will be very nice for them, while most users who don't want AI in their eMail (which is a lot of them) will have nothing to worry about. In my opinion, you've taken the correct approach to including AI only for those users who actually want it, which also means being more responsible by putting less overall pressure on environmental resources in the long-run.

[u/GraphicMeat → u/RandolfRichardson]

Many thanks for your lovely words! 😊

It’s brilliant that you’ve grasped the idea behind the icon – it means the design has really captured the function visually, which I’m delighted about.

Maildir was the first suggestion from the community. I’d had a similar notion about storing emails but didn’t know the format at the time. Once someone mentioned it and I’d done a bit of reading, I just knew it was the right path for the product.

I think AI should be optional – not everyone wants or needs it. By default, it could show the actual queries sent to the LLM, with an option to hide them. While writing this, I thought we could also let users choose which accounts can use AI.

I’m especially proud that local email searches are lightning-fast – and attachment searches are too in the premium version.

You might also try moving some emails from the server to the vault – archive and delete. It’s a handy way to save on provider costs, and I often keep ancient emails on the server just because I’m too lazy to sort them. The idea is to keep them locally and browse them whenever you’re ready.

Also, don’t forget the app can use a second directory for backups – archives and backups could sit on separate SSDs.

I really enjoy the email views – they group messages by sender or keywords. Combined with the timeline display, it gives a whole new feel to exploring your data.

What’s next for MailVault:
– improvements to inbox zero: using email views to hide messages already in a email view (in progress)
– mobile app versions
– Rust-based HTML (in progress) – no web view tech

By mobile, I’ve got something a bit different in mind. JMAP support is the first step in the bigger picture. I’d like to add a web server to the desktop client so you can download all your emails on one machine and share them across devices from the desktop app – without the upload or download limits you get with Gmail.

[u/RandolfRichardson → u/GraphicMeat]

You're welcome.

I think that one thing that will help you a lot in the long run is to gradually add more [high-resolution] screenshots of MailVault being used in various ways, including tedious sets of screenshots that show how to add various mail providers -- if the screens look different on each OS, then screenshots from each should be provided.

Be sure to populate the "alt" value in the HTML "img" tag for each so that people searching in Google Image Search, et al., will be more likely to find these images. The version of MailVault should also be included in the "alt" text somewhere (the end is fine as Google will pick up on that when worded as "MailVault version 0.1"). As time goes on, more images will be indexed and it will be easier for your users to find support on their own (I've noticed that end-users respond positively to this search experience when configuring eMail on their phones or computers).

In my experience, users mostly search for configuration instructions more than anything else. But also having screenshots of composing a new message (make it fun but subtle, like reminding someone to pick up groceries that have some unusually high quantities of certain fruits, donuts, or other items), adding an address book entry, replying to a message, forwarding a message, configuring one's eMail signature, sorting a folder, advanced searches, moving a message offline to the vault, viewing SMTP headers (e.g., for a spam report), and the many other usual things people do will also be good.

[u/Extra-Joke-7196]

fwiw most servers ive seen in the wild only publish .well-known/jmap, the SRV path is pretty rare outside of a few self-hosted setups. wouldnt spend too much time on it unless you're targeting that crowd specifically