Skip to content

fix(emlx): preserve and expose RFC Message-ID - #816

Merged
wesm merged 2 commits into
kenn-io:mainfrom
salmonumbrella:fix/issue-815-message-id
Sep 11, 2026
Merged

wesm merged 2 commits into
kenn-io:mainfrom
salmonumbrella:fix/issue-815-message-id

Conversation

@salmonumbrella

Copy link
Copy Markdown
Contributor

What changed

  • Preserve RFC Message-ID during raw email import and repair missing IDs when re-importing Apple Mail archives.
  • Link replies to a unique parent in the same source, including parents imported later and interrupted imports.
  • Expose rfc822_message_id in analytics SQL and show-message --json, including daemon connections. Rebuild caches with the added column.

Why

Apple Mail imports kept Message-ID in raw MIME but left the stored identifier empty. That prevented SQL queries and JSON clients from using it to identify messages or construct Mail.app links.

Usage

Re-import the same directory and source identifier to fill missing IDs, then rebuild the cache:

msgvault import-emlx --no-resume me@example.com ~/Mail/INBOX.mbox/
msgvault build-cache --full-rebuild
msgvault show-message 123 --json
msgvault query "SELECT id, rfc822_message_id FROM messages LIMIT 10"

New and repaired IDs use the original case without angle brackets. Existing nonempty IDs stay unchanged. Wrap and URL-encode the ID for a Mail.app link, such as message://%3CCase-ID%40example.test%3E.

Closes #815

@roborev-ci

roborev-ci Bot commented Sep 9, 2026

Copy link
Copy Markdown

roborev: Combined Review (4621830)

Verdict: Changes require fixes for 1 finding.

Medium

  • internal/mime/message_id.go:31-37: ParseMessageIDs passes the entire In-Reply-To header to a single-ID normalizer, so RFC-valid values with comments or multiple message IDs normalize to empty and cannot be linked. Parse In-Reply-To as a message-ID list, ignoring comments and folding whitespace, resolve valid candidates using the unique-parent rule, and add coverage for comments and multiple IDs.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 7s | Total: 14m46s

@wesm

wesm commented Sep 10, 2026

Copy link
Copy Markdown
Member

looking

@wesm wesm self-assigned this Sep 10, 2026
Apple Mail imports kept Message-ID in raw MIME but left the stored ID
empty. Preserve it for message links, analytics, and JSON clients, and
repair missing IDs on re-import without replacing archived content.

Resolve unique reply parents within the same source after import so
out-of-order messages and interrupted imports retain their reply links.

Give the detail-query test its SQLite connection and exercise the path
used on Windows, where DuckDB's SQLite scanner is disabled.

Generated with Codex
Co-authored-by: Codex <noreply@openai.com>
@wesm
wesm force-pushed the fix/issue-815-message-id branch from 4621830 to e065aa8 Compare September 10, 2026 21:48
@roborev-ci

roborev-ci Bot commented Sep 10, 2026

Copy link
Copy Markdown

roborev: Combined Review (e065aa8)

Verdict: Changes require fixes for 2 findings.

High

  • internal/store/email_headers.go:115-120: The reply-page query uses ? placeholders directly with PostgreSQL, which requires $1, $2, etc.; EMLX imports will fail during reply resolution. Run the query through s.Rebind(...) or a store query wrapper, and add PostgreSQL coverage for the resolver.

    Reported by: codex

Medium

  • internal/importer/ingest.go:145-150; internal/store/messages.go:1150: The generic message upsert unconditionally assigns excluded.rfc822_message_id, so upserting an existing message can clear or replace a previously nonempty identifier instead of preserving it. Make the conflict assignment fill-only: retain the stored nonempty ID and use the incoming value only when the stored value is NULL or empty.

    Reported by: codex


Reviewers: 2 done | Synthesis: codex, 8s | Total: 10m5s

Keep existing RFC Message-IDs when ordinary imports or syncs upsert a
message. Explicit provider repairs still replace an incorrect identifier
along with the rest of the snapshot.

Move incoming reply links to the survivor before hiding duplicate messages
so purging deduplicated copies does not fail a foreign-key check. Index the
parent link so each duplicate does not scan the whole archive.

Generated with Codex
Co-authored-by: Codex <noreply@openai.com>
@roborev-ci

roborev-ci Bot commented Sep 10, 2026

Copy link
Copy Markdown

roborev: Combined Review (7f49463)

No issues found.


Reviewers: 2 done | Synthesis: codex | Total: 10m28s

@wesm
wesm merged commit 6c62a15 into kenn-io:main Sep 11, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

import-emlx leaves rfc822_message_id empty; no way to read a message's Message-ID (analytics cache, show-message)

2 participants