Skip to content

feat(fast-inbox): make the inbox message rollback content-aware - #25398

Open
spalladino wants to merge 9 commits into
spl/inbox-reorg-prune-proposed-chainfrom
spl/inbox-content-aware-rollback
Open

feat(fast-inbox): make the inbox message rollback content-aware#25398
spalladino wants to merge 9 commits into
spl/inbox-reorg-prune-proposed-chainfrom
spl/inbox-content-aware-rollback

Conversation

@spalladino

@spalladino spalladino commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Context

When an L1 reorg touched a block whose Inbox messages the node had already consumed, the archiver rewound the messages from that block on and pruned every proposed block built on them, before it knew what the canonical chain would hand back. An everyday re-mine, where the same transactions land in a replacement block in the same order, therefore cost the proposer its slot even though every leaf and every rolling hash was unchanged and L1 would still have accepted the checkpoint.

Approach

The rollback now fetches the canonical messages before it writes anything, compares them against what it holds, and commits the swap in a single transaction.

  • The fetch runs in the same whole-L1-block batches the regular retrieval uses and stops at the first batch that reaches the last index the node holds, so a node that fell far behind an orphaned bucket does not pull everything since into one transaction. It waits for a bucket that spans co-timestamped L1 blocks to close before stopping, and checks every batch tail against the live chain, the final one included. A failed check writes nothing.
  • Equal rolling hashes at an index mean equal leaves through it, so one walk finds the lowest index the canonical chain no longer backs: either its leaf changed, or it closed a bucket that the canonical chain now carries on past, which takes away a boundary a block ended on. Leaves, bucket snapshots and proposed blocks are then dropped only from there. A re-mine that keeps the partition drops nothing and only rewrites the bucket metadata derived from the L1 block; a bucket whose tail was replaced keeps the blocks that stopped below the replaced leaf; a split only adds a boundary no block ever used, so it drops nothing either.
  • The swap, the snapshot rewrite, the sync point and the proposed-chain prune commit together, and the finalized marker is never advanced past the sync point the swap commits.
  • Before publishing, the proposer checks that its own archiver still holds the checkpoint's last block under the hash it was built with. The end-of-slot prune and the prune on conflict with a checkpoint seen on L1 can still take those blocks while attestations are collected, and publishing them would put a checkpoint on L1 that the node cannot serve. A node configured not to push its proposed blocks skips the check.

Stacked on #25361

Fixes A-1907

…nical one

Adds a single-transaction replace that drops leaves only from a given index, deletes the bucket
snapshots above the last canonical bucket, re-delivers the canonical messages and moves the
syncpoint, so a rollback can commit what the canonical chain holds rather than only what it removes.
The snapshot deletion is factored out of the post-removal rewind, since a re-delivery that renumbers
or merges buckets is rejected as incomplete while the stale snapshots are still in place.
…n prune together

Wraps the store swap and the prune of the proposed blocks built on the leaves it changed in one
transaction, the way the rewind already does, so a crash cannot leave a proposed chain standing on
messages the store replaced.
The rollback to the last canonical Inbox bucket now fetches the canonical messages for the range it
is about to replace before writing anything, compares them against the ones it holds, and prunes
leaves and proposed blocks only from the first leaf whose value actually changed. A reorg that
re-mines the same messages in another L1 block costs the bucket metadata and nothing else, and one
that replaces the tail of a bucket keeps the blocks that consumed the part of it that survived.

The fetch stops at the first batch that covers the last index we hold, so a node that fell far behind
does not pull the whole canonical chain into one transaction, and every batch tail it did read is
compared against the live chain before the swap commits; a reorg landing mid-fetch throws the
messages away without writing. The finalized marker moves with the syncpoint only when it is at or
below it, so the rolling-hash fallback can never skip an unsynced range.
…kpoint's blocks

Attestation collection takes seconds, and the archiver can prune the proposed chain in that window:
a slot that closed without a checkpoint, or a checkpoint seen on L1 that conflicts with the local
blocks. The publish path now reads the checkpoint's last block back from the archiver and compares
its hash, rather than publishing a checkpoint this node cannot serve.

The test doubles gain an archiver that serves back what the proposer pushed to it, and the mock
checkpoint builder no longer hands back seeded blocks it was never asked to build.
…e end to end

Withholds the propose the way the pruning test does, then rewinds L1 and replays both the message and
the propose: the message keeps its leaf and index, so the bucket only moves to the replacement L1
block, the proposed chain that consumed it survives, and the withheld checkpoint still publishes and
promotes the very blocks it was built on.
…no slot

The inbox rollback is content-aware now, so immediate consumption only loses a slot when a reorg
actually reorders or drops a message.
…y between the reorg paths

Both message fetches walk the same whole-L1-block batches, both updater writes commit alongside the same proposed-chain prune, and both reorg e2es read blobs out of a replayed tx the same way.
…locks to the archiver

A node running with skipPushProposedBlocksToArchiver or in fisherman mode keeps its proposed blocks out of the local archiver on purpose, so their absence at publish time says nothing about a prune.
…d away

A block ends on an Inbox bucket boundary, and world state, the prover and the
next checkpoint's bundle selection all derive what it consumed from the buckets
that boundary belongs to. A reorg that re-mines two buckets' messages into a
single L1 block leaves every leaf where it was, so the content comparison found
nothing to prune, but the boundary between the two buckets stopped existing and
the block that ended on it could no longer be replayed.

The swap now treats a boundary the canonical chain took away as a difference,
one-sidedly: a merge prunes the blocks that ended on the lost boundary, while a
split only adds a boundary no block ever ended on and prunes nothing. The walk
starts at the last canonical bucket's own last message, since that bucket can
keep its L1 blocks and still absorb the re-mined messages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant