Skip to content

feat: improve watch by fetching only new/changed messages instead of fetching everything (Mirador)#43

Open
kylefeng28 wants to merge 1 commit intopimalaya:masterfrom
kylefeng28:dev-idle-sync
Open

feat: improve watch by fetching only new/changed messages instead of fetching everything (Mirador)#43
kylefeng28 wants to merge 1 commit intopimalaya:masterfrom
kylefeng28:dev-idle-sync

Conversation

@kylefeng28
Copy link
Copy Markdown

@kylefeng28 kylefeng28 commented Feb 24, 2026

This PR improves Mirador's watch feature by avoiding fetching all messages on every idle loop, but instead only fetching new messages and changed/expunged messages.

@soywod I know you're pretty busy doing a major IO-free refactor of pimalaya/io-imap, so feel free to hold off on reviewing this until the refactor is done. (Just was excited to send this out since I was playing around with Mirador and finally got this working). I can rebase on the new code whenever it is released, looking forward 😁

(There's also some stuff I wasn't sure about, like if exec_hooks() should handle changed messages or not, but maybe we can discuss that later as well.)

This basically implements the basic sync algorithm from RFC 4549 - Synchronization Operations for Disconnected IMAP4 Clients, particularly particularly section 4.3.1. This doesn't make of the CONDSTORE/CHANGEDSINCE/HIGHESTMODSEQ features directly (which would be the improved sync algorithm from section 6.1), but this should be an improvement beyond the existing Mirador watch feature which fetches all messages every loop:

  1. Client discovers new messages from the server by doing FETCH <lastseenuid+1>:* (where is the latest UID of the latest message fetched from the last sync)
  2. Client discovers changes and expunged messages using UID FETCH 1:<lastseenuid> (FLAGS)
    • Messages that were previously fetched but not in the updated fetch command are expunged

This should partially address some other issues related to Mirador watch functionality:

Note: Even though though this PR doesn't make use of CONDSTORE explicitly, it seems that some IMAP servers (e.g. Gmail) will return both new and updated messages after a FETCH <lastseenuid+1>:*. I didn't see this documented in the RFC, but maybe this is some IMAP behavior related to the intersection of a persistent IMAP connection + IDLE + CONDSTORE?

email_lib::imap::ImapClient:

  • Add new methods fetch_flags() and fetch_flags_map()
  • Change fetch_all_envelopes() to use fetch_all_envelopes("1:*") instead of fetch_envelopes_by_sequence("1:*") to use UIDs instead of sequence IDs

@kylefeng28 kylefeng28 changed the title feat: improve watch by fetching only new messages and changed flags instead of fetching all messages feat: improve watch by fetching only new/changed messages instead of fetching everything Feb 24, 2026
@kylefeng28 kylefeng28 changed the title feat: improve watch by fetching only new/changed messages instead of fetching everything feat: improve watch by fetching only new/changed messages instead of fetching everything (Mirador) Feb 24, 2026
@soywod
Copy link
Copy Markdown
Member

soywod commented Feb 24, 2026

Thanks for diving into this. Indeed I would advise to wait for the refactor and see how these changes could be integrated. Can't wait to try new IMAP lib inside Mirador and other mail-related CLIs 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants