Skip to content

Hash expansion call sites position-independently so macro-expanded code survives shift edits - #98

Draft
xmakro wants to merge 1 commit into
perf/anchor-respanfrom
perf/expn-anchor
Draft

Hash expansion call sites position-independently so macro-expanded code survives shift edits#98
xmakro wants to merge 1 commit into
perf/anchor-respanfrom
perf/expn-anchor

Conversation

@xmakro

@xmakro xmakro commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Stacked on #99 and #97 (the fused def_anchor stack), which this depends on. Reworked from the earlier revision that sat on the closed #91's channel; the expn_call_site_position position-fold query from that revision is replaced by expn_anchor, the expansion-side twin of def_anchor.

ExpnData is the remaining position carrier after #97: its call_site and def_site spans hashed absolutely, so editing text above a macro invocation re-fingerprinted the expansion and everything expanded from it, even though relative spans and the content-addressed source_span anchor were already position-free. This PR hashes both spans position-independently (stable_hash_span and stable_hash_anchor_span become one implementation differing only in whether the absolute arm covers the offset; parented spans were already relative). Identity between textually identical invocations rests on the existing disambiguator mechanism, plus one lowering fix: mark_span_with_reason now parents desugaring call sites the way rustc_expand already parents macro invocation spans, so same-kind desugarings in different definitions are distinct by parent rather than by session-order disambiguation. That also turns a previously order-dependent typeck_root invalidation in tests/incremental/hashes/function_interfaces.rs deterministic; the test's annotations are updated.

The anchor

With positions out of the expansion fingerprint, a renderer that lands on a parentless collapsed call site (debuginfo for macro-generated definitions under #[collapse_debuginfo], #[track_caller] whose expansion cause is an item-level invocation) needs an explicit dependency, exactly as definition-extent renderings need def_anchor. That dependency is expn_anchor(ExpnHash): the same content as def_anchor, the file id plus SourceFile::line_extent_hash over the call-site span, so it covers the rendered line index, column and extent line structure of the call site and nothing else. A byte-shifting edit above the invocation that moves no line break stays green; an edit that changes any rendered value goes red. The previous revision paired an opaque position fold with a separate line-table dependency at each site; here it is one node with the precision of the rendered values, the same fusion #99 applies on the definition side.

The dep-node key is the ExpnHash itself: it packs losslessly into the key fingerprint, so this PR reintroduces the small KeyFingerprintStyle::SelfHash recovery style (three hunks) that the bucketed design had used, now scoped to this one key type. Forcing a node for a foreign expansion resolves it through the defining crate's metadata; expn_hash_to_expn_id becomes fallible so a vanished crate or expansion keeps the zero sentinel and re-executes dependents instead of ICEing, while the on-disk-cache decode path (whose references can only have been green if the expansion still exists) unwraps loudly.

Wiring

TyCtxt::walk_chain_collapsed_tracked replaces the untracked hygiene walk wherever the collapsed result feeds a cached artifact. When the collapse lands on a parentless call site it records expn_anchor and registers its extent for #99's line-observation hook; otherwise the caller anchors through the span's parent or the instance as before. A collapsed rendering that escapes both anchors trips the hook's conservative fallback and costs reuse rather than staleness.

Expansion anchors are recorded only where no definition-keyed anchor already covers the position. An ExpnHash is fragile across edits: inserting any macro invocation (the benchmark suite's println! patch, for instance) shifts the disambiguators of later identical invocations, changing their hashes, and a dep node whose key cannot be recovered can never be marked green, so every dependent goes red. Definition-keyed anchors are stable under the same edit, so they take priority and the ExpnHash-keyed nodes are reserved for positions nothing else covers (this took ripgrep's patched rebuild from seven modules re-codegened to one, against the baseline compiler's five). Bang invocations hash their call-site extent; derive and attribute invocations, whose recorded call sites are just the path, stay file-granular. Wired at the cg_ssa debuginfo funnels, cg_clif's get_span_loc, cg_llvm's file_metadata_from_def_id, and span_as_caller_location via the new expansion_cause_with_expn.

tests/run-make/incr-macro-static-decl-line pins the edge: a #[collapse_debuginfo] macro generates a static, and the two byte-length-identical versions swap the invocation line with a padding line, so nothing re-fingerprints and only the expn_anchor dependency refreshes the static's DW_AT_decl_line.

Performance

Same-day A/B on the setup from #99 (instructions:u, jemalloc, 81 cells). Marginal effect of this commit against the stack head (#99 + #97):

scenario marginal vs stack
full +0.05%
incr-full +0.04%
incr-unchanged -0.86%
incr-patched -3.31%
all cells -1.28%

The largest single contributor in the stack. The patched win is where expansion-heavy code lives: serde incr-patched improves about -18% across profiles, and cargo opt incr-unchanged improves -18.6% (this commit's anchors give the caller-location and collapsed-debuginfo classes their precise invalidation channel, which #99 standalone leaves to conservative re-execution). The cost side is a broad set of small incr-unchanged cells at +0.28% to +0.85%, the marking cost of the expn_anchor nodes; the coverage-first recording rule above keeps their count down.

Cumulative for the full reworked stack (#99 + #97 + #98) against the merge-base (fresh base run), 81 cells: full -0.15%, incr-full +0.07%, incr-unchanged -0.47%, incr-patched -4.03%, all cells -1.48%, with 47 cells improved by at least 0.25% and 8 regressed; every check-profile cell improves (serde incr-patched -19.0% to -20.7%). The largest residual is cargo debug incr-unchanged +9.4%, with the rest at or below +1%. See #99 for the trust-tiered anchor design and the staleness bugs the fail-red rework surfaced.

@xmakro
xmakro force-pushed the perf/expn-anchor branch from fcd0df7 to 5e2f48a Compare August 5, 2026 15:52
@xmakro
xmakro force-pushed the perf/expn-anchor branch from 5e2f48a to 0aea640 Compare August 6, 2026 06:02
@xmakro
xmakro force-pushed the perf/anchor-respan branch from d645986 to 00c78e7 Compare August 6, 2026 06:02
@xmakro
xmakro force-pushed the perf/anchor-respan branch from 00c78e7 to 372c8ba Compare August 6, 2026 08:01
@xmakro
xmakro force-pushed the perf/expn-anchor branch 2 times, most recently from a1e523e to a1b2ce1 Compare August 6, 2026 11:58
@xmakro
xmakro force-pushed the perf/anchor-respan branch from 372c8ba to be63e55 Compare August 6, 2026 11:58
@xmakro
xmakro force-pushed the perf/anchor-respan branch from be63e55 to 2190174 Compare August 7, 2026 12:39
@xmakro
xmakro force-pushed the perf/anchor-respan branch from 2190174 to 118a0ee Compare August 7, 2026 13:34
@xmakro
xmakro force-pushed the perf/expn-anchor branch from a1b2ce1 to ea7a63f Compare August 7, 2026 14:21
@xmakro
xmakro force-pushed the perf/anchor-respan branch from 118a0ee to 96ce91c Compare August 7, 2026 17:20
@xmakro
xmakro force-pushed the perf/expn-anchor branch from ea7a63f to 3037ac2 Compare August 7, 2026 17:20
@xmakro
xmakro force-pushed the perf/anchor-respan branch from 96ce91c to 4b14f71 Compare August 7, 2026 21:13
@xmakro
xmakro force-pushed the perf/expn-anchor branch from 3037ac2 to 3792fd0 Compare August 7, 2026 21:13
@xmakro
xmakro force-pushed the perf/anchor-respan branch from 4b14f71 to 4a62588 Compare August 8, 2026 00:43
@xmakro
xmakro force-pushed the perf/expn-anchor branch from 3792fd0 to c065c20 Compare August 8, 2026 00:43
@xmakro
xmakro force-pushed the perf/anchor-respan branch from 4a62588 to b59ed1e Compare August 8, 2026 10:59
@xmakro
xmakro force-pushed the perf/expn-anchor branch from c065c20 to 8567411 Compare August 8, 2026 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant