Hash expansion call sites position-independently so macro-expanded code survives shift edits - #98
Draft
xmakro wants to merge 1 commit into
Draft
Hash expansion call sites position-independently so macro-expanded code survives shift edits#98xmakro wants to merge 1 commit into
xmakro wants to merge 1 commit into
Conversation
xmakro
force-pushed
the
perf/expn-anchor
branch
from
August 5, 2026 15:52
fcd0df7 to
5e2f48a
Compare
xmakro
force-pushed
the
perf/expn-anchor
branch
from
August 6, 2026 06:02
5e2f48a to
0aea640
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 6, 2026 06:02
d645986 to
00c78e7
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 6, 2026 08:01
00c78e7 to
372c8ba
Compare
xmakro
force-pushed
the
perf/expn-anchor
branch
2 times, most recently
from
August 6, 2026 11:58
a1e523e to
a1b2ce1
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 6, 2026 11:58
372c8ba to
be63e55
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 7, 2026 12:39
be63e55 to
2190174
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 7, 2026 13:34
2190174 to
118a0ee
Compare
xmakro
force-pushed
the
perf/expn-anchor
branch
from
August 7, 2026 14:21
a1b2ce1 to
ea7a63f
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 7, 2026 17:20
118a0ee to
96ce91c
Compare
xmakro
force-pushed
the
perf/expn-anchor
branch
from
August 7, 2026 17:20
ea7a63f to
3037ac2
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 7, 2026 21:13
96ce91c to
4b14f71
Compare
xmakro
force-pushed
the
perf/expn-anchor
branch
from
August 7, 2026 21:13
3037ac2 to
3792fd0
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 8, 2026 00:43
4b14f71 to
4a62588
Compare
xmakro
force-pushed
the
perf/expn-anchor
branch
from
August 8, 2026 00:43
3792fd0 to
c065c20
Compare
xmakro
force-pushed
the
perf/anchor-respan
branch
from
August 8, 2026 10:59
4a62588 to
b59ed1e
Compare
xmakro
force-pushed
the
perf/expn-anchor
branch
from
August 8, 2026 10:59
c065c20 to
8567411
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #99 and #97 (the fused
def_anchorstack), which this depends on. Reworked from the earlier revision that sat on the closed #91's channel; theexpn_call_site_positionposition-fold query from that revision is replaced byexpn_anchor, the expansion-side twin ofdef_anchor.ExpnDatais the remaining position carrier after #97: itscall_siteanddef_sitespans hashed absolutely, so editing text above a macro invocation re-fingerprinted the expansion and everything expanded from it, even though relative spans and the content-addressedsource_spananchor were already position-free. This PR hashes both spans position-independently (stable_hash_spanandstable_hash_anchor_spanbecome one implementation differing only in whether the absolute arm covers the offset; parented spans were already relative). Identity between textually identical invocations rests on the existingdisambiguatormechanism, plus one lowering fix:mark_span_with_reasonnow parents desugaring call sites the wayrustc_expandalready parents macro invocation spans, so same-kind desugarings in different definitions are distinct by parent rather than by session-order disambiguation. That also turns a previously order-dependenttypeck_rootinvalidation intests/incremental/hashes/function_interfaces.rsdeterministic; the test's annotations are updated.The anchor
With positions out of the expansion fingerprint, a renderer that lands on a parentless collapsed call site (debuginfo for macro-generated definitions under
#[collapse_debuginfo],#[track_caller]whose expansion cause is an item-level invocation) needs an explicit dependency, exactly as definition-extent renderings needdef_anchor. That dependency isexpn_anchor(ExpnHash): the same content asdef_anchor, the file id plusSourceFile::line_extent_hashover the call-site span, so it covers the rendered line index, column and extent line structure of the call site and nothing else. A byte-shifting edit above the invocation that moves no line break stays green; an edit that changes any rendered value goes red. The previous revision paired an opaque position fold with a separate line-table dependency at each site; here it is one node with the precision of the rendered values, the same fusion #99 applies on the definition side.The dep-node key is the
ExpnHashitself: it packs losslessly into the key fingerprint, so this PR reintroduces the smallKeyFingerprintStyle::SelfHashrecovery style (three hunks) that the bucketed design had used, now scoped to this one key type. Forcing a node for a foreign expansion resolves it through the defining crate's metadata;expn_hash_to_expn_idbecomes fallible so a vanished crate or expansion keeps the zero sentinel and re-executes dependents instead of ICEing, while the on-disk-cache decode path (whose references can only have been green if the expansion still exists) unwraps loudly.Wiring
TyCtxt::walk_chain_collapsed_trackedreplaces the untracked hygiene walk wherever the collapsed result feeds a cached artifact. When the collapse lands on a parentless call site it recordsexpn_anchorand registers its extent for #99's line-observation hook; otherwise the caller anchors through the span's parent or the instance as before. A collapsed rendering that escapes both anchors trips the hook's conservative fallback and costs reuse rather than staleness.Expansion anchors are recorded only where no definition-keyed anchor already covers the position. An
ExpnHashis fragile across edits: inserting any macro invocation (the benchmark suite'sprintln!patch, for instance) shifts the disambiguators of later identical invocations, changing their hashes, and a dep node whose key cannot be recovered can never be marked green, so every dependent goes red. Definition-keyed anchors are stable under the same edit, so they take priority and theExpnHash-keyed nodes are reserved for positions nothing else covers (this took ripgrep's patched rebuild from seven modules re-codegened to one, against the baseline compiler's five). Bang invocations hash their call-site extent; derive and attribute invocations, whose recorded call sites are just the path, stay file-granular. Wired at the cg_ssa debuginfo funnels, cg_clif'sget_span_loc, cg_llvm'sfile_metadata_from_def_id, andspan_as_caller_locationvia the newexpansion_cause_with_expn.tests/run-make/incr-macro-static-decl-linepins the edge: a#[collapse_debuginfo]macro generates a static, and the two byte-length-identical versions swap the invocation line with a padding line, so nothing re-fingerprints and only theexpn_anchordependency refreshes the static'sDW_AT_decl_line.Performance
Same-day A/B on the setup from #99 (instructions:u, jemalloc, 81 cells). Marginal effect of this commit against the stack head (#99 + #97):
The largest single contributor in the stack. The patched win is where expansion-heavy code lives: serde incr-patched improves about -18% across profiles, and cargo opt incr-unchanged improves -18.6% (this commit's anchors give the caller-location and collapsed-debuginfo classes their precise invalidation channel, which #99 standalone leaves to conservative re-execution). The cost side is a broad set of small incr-unchanged cells at +0.28% to +0.85%, the marking cost of the
expn_anchornodes; the coverage-first recording rule above keeps their count down.Cumulative for the full reworked stack (#99 + #97 + #98) against the merge-base (fresh base run), 81 cells: full -0.15%, incr-full +0.07%, incr-unchanged -0.47%, incr-patched -4.03%, all cells -1.48%, with 47 cells improved by at least 0.25% and 8 regressed; every check-profile cell improves (serde incr-patched -19.0% to -20.7%). The largest residual is cargo debug incr-unchanged +9.4%, with the rest at or below +1%. See #99 for the trust-tiered anchor design and the staleness bugs the fail-red rework surfaced.