diff --git a/barretenberg/.claude/skills/gate-counts/SKILL.md b/barretenberg/.claude/skills/gate-counts/SKILL.md index 82f4e64ac8db..a94cd08391fe 100644 --- a/barretenberg/.claude/skills/gate-counts/SKILL.md +++ b/barretenberg/.claude/skills/gate-counts/SKILL.md @@ -220,11 +220,13 @@ AZTEC_GENERATE_TEST_DATA=1 \ yarn workspace @aztec/prover-client test regenerate_rollup_sample_inputs ``` -Regenerates `Prover.toml` for `rollup-block-root-first-empty-tx`, -`rollup-block-root-first`, `rollup-block-root-first-single-tx`, -`rollup-block-root`, `rollup-block-root-single-tx`, `rollup-block-root-msgs-only`, -`rollup-block-merge`, `rollup-checkpoint-root`, `rollup-checkpoint-root-single-block`, -`rollup-checkpoint-merge`, `rollup-tx-merge`, `rollup-root`. +Regenerates the `Prover.toml` of every block-root variant plus the block-merge, +checkpoint-root, checkpoint-merge, tx-merge and root circuits. Read the exact +inventory off the `dump` fields of the `scenarios` array in +`yarn-project/prover-client/src/test/regenerate_rollup_sample_inputs.test.ts` +rather than from a list here: each scenario declares which circuits it is +responsible for, and the set changes whenever a block-root variant is added or +removed. **Private-kernel + transaction-base circuits — e2e prover full test** (spins up an L1/anvil sandbox): diff --git a/barretenberg/.claude/skills/update-prover-toml/SKILL.md b/barretenberg/.claude/skills/update-prover-toml/SKILL.md index 0298d57ff1ce..76df525d882c 100644 --- a/barretenberg/.claude/skills/update-prover-toml/SKILL.md +++ b/barretenberg/.claude/skills/update-prover-toml/SKILL.md @@ -31,7 +31,7 @@ Two commands cover the protocol-circuit tomls, split by whether the sample needs AZTEC_GENERATE_TEST_DATA=1 yarn workspace @aztec/prover-client test regenerate_rollup_sample_inputs ``` -Regenerates: `rollup-block-root-first-empty-tx`, `rollup-block-root-first`, `rollup-block-root-first-single-tx`, `rollup-block-root`, `rollup-block-root-single-tx`, `rollup-block-root-msgs-only`, `rollup-block-merge`, `rollup-checkpoint-root`, `rollup-checkpoint-root-single-block`, `rollup-checkpoint-merge`, `rollup-tx-merge`, `rollup-root`. The scenario list lives in the `scenarios` array in that test; each scenario's `dump` field names the tomls it owns. +Regenerates every block-root variant plus the block-merge, checkpoint-root, checkpoint-merge, tx-merge and root tomls. Take the exact inventory from the `scenarios` array in that test rather than from a list here: each scenario's `dump` field names the tomls it owns, and the set changes whenever a block-root variant is added or removed. ### Private-kernel and transaction-base circuits — e2e prover full test diff --git a/docs/docs-developers/docs/aztec-nr/framework-description/ethereum_aztec_messaging.md b/docs/docs-developers/docs/aztec-nr/framework-description/ethereum_aztec_messaging.md index e035578649bd..9963a6b7a92d 100644 --- a/docs/docs-developers/docs/aztec-nr/framework-description/ethereum_aztec_messaging.md +++ b/docs/docs-developers/docs/aztec-nr/framework-description/ethereum_aztec_messaging.md @@ -31,7 +31,7 @@ Use the `Inbox` contract's `sendL2Message` function: #include_code deposit_public l1-contracts/test/portals/TokenPortal.sol solidity :::note Message availability -L1 to L2 messages are not available immediately. The proposer batches messages from the Inbox and includes them in the next L2 block. You must wait for this before consuming the message on L2. +L1 to L2 messages are not available the moment the L1 transaction is mined. Messages the proposer's node has observed on L1 are eligible for the next L2 block it builds, subject to the per-block and per-checkpoint message caps; near the end of a checkpoint, consumption stops at an Inbox bucket boundary and later messages wait for the next checkpoint. You must wait for a block containing the message before consuming it on L2. Public functions can consume a message in the block that inserts it; private functions prove membership against a block header the wallet has synced, so they additionally wait for that block to reach the wallet. See [Inbox](../../foundational-topics/ethereum-aztec-messaging/inbox.md#how-messages-reach-l2) for how consumption works. ::: ### Consume the message on L2 diff --git a/docs/docs-developers/docs/foundational-topics/ethereum-aztec-messaging/inbox.md b/docs/docs-developers/docs/foundational-topics/ethereum-aztec-messaging/inbox.md index d3d818b629c5..7ff151c1e3f2 100644 --- a/docs/docs-developers/docs/foundational-topics/ethereum-aztec-messaging/inbox.md +++ b/docs/docs-developers/docs/foundational-topics/ethereum-aztec-messaging/inbox.md @@ -1,6 +1,6 @@ --- title: Inbox -description: Learn about the inbox mechanism in Aztec portals for receiving messages from L1. +description: Learn about the inbox mechanism in Aztec portals for receiving messages from L1, and how L2 blocks consume them. tags: [portals, contracts] references: ["l1-contracts/src/core/interfaces/messagebridge/IInbox.sol"] --- @@ -20,7 +20,9 @@ Sends a message from L1 to L2. | Recipient | [`L2Actor`](./data_structures.md#l2actor) | The recipient of the message. The recipient's version **MUST** match the inbox version and the actor must be an Aztec contract that is **attached** to the contract making this call. If the recipient is not attached to the caller, the message cannot be consumed by it. | | Content | `field` (~254 bits) | The content of the message. This is the data that will be passed to the recipient. The content is limited to a single field for rollup purposes. If the content is small enough it can be passed directly, otherwise it should be hashed and the hash passed along (you can use our [`Hash`](https://github.com/AztecProtocol/aztec-packages/blob/#include_aztec_version/l1-contracts/src/core/libraries/crypto/Hash.sol) utilities with `sha256ToField` functions). | | Secret Hash | `field` (~254 bits) | A hash of a secret used when consuming the message on L2. Keep this preimage secret to make the consumption private. To consume the message the caller must know the pre-image (the value that was hashed). Use [`computeSecretHash`](https://github.com/AztecProtocol/aztec-packages/blob/#include_aztec_version/yarn-project/stdlib/src/hash/hash.ts) to compute it from a secret. | -| ReturnValue | `(bytes32, uint256)` | The message hash (used as an identifier) and the leaf index in the tree. | +| ReturnValue | `(bytes32, uint256)` | The message hash (used as an identifier) and the message's leaf index in the L1-to-L2 message tree. | + +The leaf index is the Inbox's cumulative message count at insertion: the first message ever sent gets index 0, the next one index 1, and so on, with no padding between checkpoints. The same index is emitted in the `MessageSent` event as `message.index`, and is what you pass to `consume_l1_to_l2_message` on L2 together with the secret. #### Edge cases @@ -28,16 +30,66 @@ Sends a message from L1 to L2. - Will revert with `Inbox__VersionMismatch(uint256 expected, uint256 actual)` if the recipient version doesn't match the inbox version. - Will revert with `Inbox__ContentTooLarge(bytes32 content)` if the content is larger than the field size (~254 bits). - Will revert with `Inbox__SecretHashTooLarge(bytes32 secretHash)` if the secret hash is larger than the field size (~254 bits). +- Will revert with `Inbox__WouldOverwriteUnconsumedBucket(uint64 evictedBucketSeq)` if accepting the message would open a bucket whose ring slot still holds a bucket the proven chain has not consumed (see [ring headroom](#ring-headroom-and-back-pressure)). Sends resume once an epoch proof advances the proven consumption. + +## `MessageSent` event + +Every accepted message emits: + +```solidity +event MessageSent(bytes32 indexed hash, bytes32 inboxRollingHash, uint256 bucketSeq, DataStructures.L1ToL2Msg message); +``` + +- `hash` is the message hash (the L1-to-L2 tree leaf), the value the return of `sendL2Message` also gives you. +- `inboxRollingHash` is the Inbox's consensus rolling hash after absorbing this message (see below). +- `bucketSeq` is the sequence number of the L1 bucket the message was absorbed into. +- `message` is the full [`L1ToL2Msg`](./data_structures.md#l1tol2msg), including `message.index`, the leaf index. The event carries the whole message so that a portal sending through internal calls can still recover every field from the receipt. + +## How messages reach L2 + +The Inbox does not batch messages into per-checkpoint trees. It keeps a single **rolling hash** over every message it has ever accepted, `rollingHash' = sha256ToField(DOM_SEP__INBOX_ROLLING_HASH ‖ rollingHash ‖ leaf)`, starting from zero. Messages arriving in the same L1 block are grouped into a **bucket**: a snapshot of the rolling hash, the cumulative message count and the L1 timestamp at the end of that block. A bucket holds at most `MAX_L1_TO_L2_MSGS_PER_BLOCK` (256) messages; further messages in the same L1 block open a new bucket. Buckets are numbered by a dense sequence and stored in a ring of 4096 slots. + +An L2 proposer consumes messages as soon as its node observes them in a mined L1 block: + +- **Every ordinary L2 block consumes the messages the node has observed so far**, up to 256 per block and `MAX_L1_TO_L2_MSGS_PER_CHECKPOINT` (1024) per checkpoint. There is no waiting for a confirmation depth, no minimum message age, and no requirement that a block ends exactly at a bucket boundary: an intermediate block may end at any prefix of the message sequence, including in the middle of an L1 bucket. Selection uses the local view alone while the block's end stays at or below 768 messages into the checkpoint; a step above that line, and every final block, first resolves a live bucket end from the Inbox, so the proposer cannot consume past the last endpoint its remaining capacity can reach. Nothing is remembered between blocks: each one decides again from where the previous one ended. +- **Only the checkpoint's final message position must coincide with a live bucket end.** The checkpoint header commits to the rolling hash at that position (`inboxRollingHash`), and `Rollup.propose` checks it against the bucket the proposer names in the unsigned `bucketHint` calldata argument. The proposer resolves that endpoint with one `getBucketAtOrBeforeTotal` call and simulates the Rollup's header and Inbox checks (`Rollup.validateCheckpointHeaderAndInbox`) at the last L1 slot of the target L2 slot, once before gossiping the checkpoint and again immediately before publishing it. The pre-gossip run stands in for a pipelined parent that has not landed yet, so its verdict is conditional on that parent landing. The pre-publication run drops that override, because the parent is confirmed by then, but it is still not a simulation against unmodified L1 state: when a prune is due at the target slot it keeps the build's pinned proven tip, making the verdict conditional on the epoch proof landing by that slot. That is the same assumption the build made, and `propose` itself enforces it. Neither run is a full `propose` either: attestation-signature verification and the data-availability checks are out of scope. A preflight that passes means the header, settlement, cap and censorship checks hold under those assumptions, not that the transaction will land on L1. +- **Mandatory consumption.** A checkpoint may not leave unconsumed any bucket opened at or before the cutoff for its slot (one L1 slot before the previous L2 slot started), unless consuming it would exceed the 1024-message cap. This is what stops a committee from censoring L1-to-L2 messages. +- **Validators check content, not L1 bucket layout.** A block proposal carries a signed reference to the message prefix it consumed through (its rolling hash; the count is in the block header). Validators compare it against their own view of the Inbox and read the consumed messages by count. A validator whose view is behind or disagrees retries after a bounded L1 sync; a disagreement that persists is treated as a local-view problem, never as proposer misconduct. + +A message is therefore eligible for the next block the proposer builds after its node observes the L1 block carrying it, and normally enters it, rather than waiting one to two checkpoints. Inclusion can still be deferred: by a backlog larger than the per-block and per-checkpoint caps, by a checkpoint whose remaining capacity cannot reach past the bucket end it completes at, or by a checkpoint that fails to be built or published. + +### Public and private consumption + +A block's public (AVM) functions can read the messages inserted in that same block: the block's L1-to-L2 tree snapshot already includes them. Private functions prove message membership against a historical block header the wallet has synced, so a private `consume_l1_to_l2_message` has to wait until a block containing the message is available to the wallet. Both paths use the same leaf index and secret. + +### Accepted limitations + +Consuming messages immediately couples the proposed L2 chain to the L1 head. The design accepts the following consequences rather than adding a confirmation delay: + +- An L1 reorg that re-mines the same messages under different bucket boundaries changes nothing for intermediate blocks. If it removes the bucket end a **completed** checkpoint relied on, that checkpoint cannot be published: the node abandons it instead of re-signing a different one for the same slot. +- A reorg that changes what the Inbox holds makes the node roll its message log back to the newest message it can still find on L1 near the height it recorded for it, and drop the not-yet-checkpointed blocks that consumed anything past that point. The rollback happens before the missing messages are re-fetched, so a message the node cannot place is dropped even when L1 still has it unchanged, and blocks that consumed it are dropped with it even though the same content comes straight back. Already published checkpoints are never removed this way. +- Committee attestations prove that the checkpoint's message content matches the committee's view of L1, not that the checkpoint is publishable at that moment. The proposer's preflight and `Rollup.propose` are the authority on settlement, cap and censorship rules. +- An L1 reorg that becomes visible only after the `propose` transaction was submitted can revert it, spending L1 gas even though the preflight passed. +- Node-side reorg recovery inherits a shortcut that trusts a locally stored message as final once its recorded L1 height is at or below the finalized block the node last recorded. Recorded heights are refreshed only when the node re-fetches and reinserts a matching message (during forward ingestion, including the refetch that follows a rollback); a same-content reorg that leaves the Inbox position at the head unchanged is taken as agreement without re-fetching, so the message keeps its old height and the shortcut can later trust an unfinalized replacement. This is a known, deliberately deferred limitation of the node's recovery, not of the protocol. + +### Ring headroom and back pressure + +The ring of 4096 buckets is a fixed deployment parameter. When the ring would wrap onto a bucket the **proven** chain has not consumed yet, `sendL2Message` reverts rather than overwriting it, so no message is ever lost and `propose` can always resolve the bucket it needs. In exchange, a long proving stall halts L1-to-L2 sends for every portal until the next epoch proof advances the proven consumption. `getRingHeadroom()` reports how many buckets can still be opened; at the natural cadence of one bucket per message-bearing L1 block, the ring covers roughly 4096 L1 blocks, or about 13.6 hours, of consumption stall. ## View functions These functions allow you to query the current state of the Inbox. -| Function | Returns | Description | -| -------------------------- | ----------------- | ------------------------------------------------ | -| `getState()` | `InboxState` | Returns the current inbox state (rolling hash, total messages inserted). | -| `getTotalMessagesInserted()` | `uint64` | Returns the total number of messages inserted into the inbox. | -| `getFeeAssetPortal()` | `address` | Returns the address of the Fee Juice portal. | +| Function | Returns | Description | +| ---------------------------------------- | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `getState()` | `InboxState` | The live position of the Inbox in one read: `rollingHash`, `totalMessagesInserted` and `currentBucketSeq`. | +| `getTotalMessagesInserted()` | `uint64` | The total number of messages inserted into the inbox, which is also the leaf index the next message will get. | +| `getCurrentBucketSeq()` | `uint64` | The sequence number of the bucket currently accumulating messages. | +| `getBucket(uint256 seq)` | `InboxBucket` | The bucket with the given sequence number: `rollingHash`, cumulative `totalMsgCount`, opening `timestamp` and `msgCount`. Reverts with `Inbox__BucketOutOfWindow(seq, current)` if `seq` is ahead of the current bucket or has been evicted from the ring. | +| `getBucketAtOrBeforeTotal(uint64 total)` | `(uint64 seq, InboxBucket)` | The live bucket with the greatest cumulative message total at or below `total`, and its sequence number. Proposers use it to resolve a checkpoint's final message count to a bucket end. Reverts with `Inbox__NoBucketAtOrBeforeTotal(upperBound, oldestLiveTotal)` when even the oldest retained bucket ends past `total`. | +| `getProvenConsumedBucketSeq()` | `uint64` | The newest bucket the proven chain has consumed; buckets at or below it may be evicted when the ring wraps. | +| `getRingHeadroom()` | `uint256` | How many buckets can still be opened before `sendL2Message` reverts to protect an unconsumed bucket. Counts bucket openings, not messages. | +| `getFeeAssetPortal()` | `address` | The address of the Fee Juice portal. | ## Related pages diff --git a/docs/docs-developers/docs/foundational-topics/ethereum-aztec-messaging/index.md b/docs/docs-developers/docs/foundational-topics/ethereum-aztec-messaging/index.md index 5d40de70160a..385e772cfbac 100644 --- a/docs/docs-developers/docs/foundational-topics/ethereum-aztec-messaging/index.md +++ b/docs/docs-developers/docs/foundational-topics/ethereum-aztec-messaging/index.md @@ -38,7 +38,7 @@ With the aforementioned restrictions taken into account, cross-chain messages ca As an illustration, suppose a private function adds a cross-chain call. In such a case, the private function would not have knowledge of the result of the cross-chain call within the same rollup (since it has yet to be executed). -Similarly to the ordering of private and public functions, we can also reap the benefits of intentionally ordering messages between L1 and L2. When a message is sent from L1 to L2, it has been "emitted" by an action in the past (an L1 interaction), allowing us to add it to the list of consumables at the "beginning" of the block execution. This practical approach means that a message could be consumed in the same block it is included. In a sophisticated setup, rollup $n$ could send an L2 to L1 message that is then consumed on L1, and the response is added already in $n+1$. However, messages going from L2 to L1 will be added as they are emitted. +Similarly to the ordering of private and public functions, we can also reap the benefits of intentionally ordering messages between L1 and L2. When a message is sent from L1 to L2, it has been "emitted" by an action in the past (an L1 interaction), allowing us to add it to the list of consumables at the "beginning" of the block execution. This practical approach means that a message could be consumed by a public function in the same block it is included. The proposer inserts the L1-to-L2 messages its node has observed into every L2 block it builds, not only the first block of a checkpoint, so a message normally becomes consumable on L2 shortly after the L1 block carrying it is mined (see [Inbox](./inbox.md#how-messages-reach-l2) for the rules and their limitations). In a sophisticated setup, rollup $n$ could send an L2 to L1 message that is then consumed on L1, and the response is added already in $n+1$. However, messages going from L2 to L1 will be added as they are emitted. :::info Because everything is unilateral and async, application developers must explicitly handle failure cases so users can gracefully recover. Token bridges are a prime example: it would be very inconvenient if funds are locked on one domain but never minted or unlocked on the other. @@ -80,7 +80,7 @@ The rollup contract has a few very important responsibilities. The contract must To ensure that _state transitions_ are performed correctly, the contract will derive public inputs for the **rollup circuit** based on the input data, and then use a _verifier_ contract to validate that inputs correctly transition the current state to the next. All data needed for the public inputs to the circuit must be from the rollup block, ensuring that the block is available. For a valid proof, the _rollup state root_ is updated and it will emit an _event_ to make it easy for anyone to find the data. -As part of _state transitions_ where cross-chain messages are included, the contract must "move" messages along the way, e.g., from "pending" to "ready". +As part of _state transitions_ where cross-chain messages are included, the contract must "move" messages along the way, e.g., from "pending" to "ready". For L1 to L2 messages this means checking, when a checkpoint is proposed, that the rolling hash the checkpoint header commits to matches an Inbox bucket end, that consumption never rewinds or exceeds the per-checkpoint cap, and that no bucket old enough to be mandatory was skipped. ### Kernel Circuit diff --git a/docs/docs-developers/docs/resources/migration_notes.md b/docs/docs-developers/docs/resources/migration_notes.md index 182039212abb..fad58c5a826b 100644 --- a/docs/docs-developers/docs/resources/migration_notes.md +++ b/docs/docs-developers/docs/resources/migration_notes.md @@ -9,6 +9,74 @@ Aztec is in active development. Each version may introduce breaking changes that ## TBD +### [L1 Contracts] `Inbox.MessageSent` emits the full message and the consensus rolling hash + +The streaming Inbox (Fast Inbox) no longer ties a message to the checkpoint that will consume it, so the `MessageSent` event changed shape. The old event carried the target checkpoint number, the leaf index and a truncated `bytes16` sync hash; the new one carries the whole `L1ToL2Msg` (whose `index` field is the leaf index), the `bytes32` consensus rolling hash after the message, and the sequence number of the L1 bucket that absorbed it: + +```diff +- event MessageSent(uint256 indexed checkpointNumber, uint256 index, bytes32 indexed hash, bytes16 rollingHash); ++ event MessageSent(bytes32 indexed hash, bytes32 inboxRollingHash, uint256 bucketSeq, DataStructures.L1ToL2Msg message); +``` + +**Migration:** + +```diff + const [log] = parseEventLogs({ abi: InboxAbi, eventName: 'MessageSent', logs: receipt.logs }); +- const leafIndex = log.args.index; ++ const leafIndex = log.args.message.index; +``` + +**Impact**: Any code that decodes `MessageSent` from an L1 receipt (portals, bridges, indexers) must read the leaf index from `args.message.index`. There is no per-message checkpoint number any more: which checkpoint includes a message is decided when the proposer builds its blocks, not when the message is sent (see the next entry for how to check readiness). The index is still the message's global leaf index in the L1-to-L2 message tree, but its derivation changed: it used to be `(checkpointNumber - initial) * subtreeSize + positionInSubtree`, one padded subtree per checkpoint, and is now the compact cumulative Inbox message count at insertion. Remove any checkpoint-based index arithmetic and read the index the call returns or the event emits. This change shipped with the Fast Inbox umbrella and was previously undocumented here. + +### [Aztec.js] `getL1ToL2MessageCheckpoint` replaced by `getL1ToL2MessageIndex` + +Because a message is no longer bound to a checkpoint when it is sent, `AztecNode.getL1ToL2MessageCheckpoint(messageHash)` (JSON-RPC `aztec_getL1ToL2MessageCheckpoint`), which returned the `CheckpointNumber` that would include the message, is gone. Its replacement `getL1ToL2MessageIndex(messageHash)` (JSON-RPC `aztec_getL1ToL2MessageIndex`) returns the message's leaf index as a `bigint`, or `undefined` if the node has not seen the message on L1 yet. A message is consumable at a chain tip once that tip's L1-to-L2 message tree has grown past the index, which is what `waitForL1ToL2MessageReady` and `isL1ToL2MessageReady` in `@aztec/aztec.js/messaging` now check: + +```diff +- const checkpoint = await node.getL1ToL2MessageCheckpoint(messageHash); +- const ready = checkpoint !== undefined && checkpoint <= (await node.getCheckpointNumber()); ++ const index = await node.getL1ToL2MessageIndex(messageHash); ++ const block = await node.getBlockData('latest'); ++ const ready = index !== undefined && block !== undefined && index < BigInt(block.header.state.l1ToL2MessageTree.nextAvailableLeafIndex); +``` + +`waitForL1ToL2MessageReady(node, messageHash, { timeoutSeconds, chainTip })` accepts an optional `chainTip` (default `'latest'`); pass the tip the consuming PXE syncs to (for example `'proven'`) so readiness is answered at the block the transaction simulation will anchor to. This change shipped with the Fast Inbox umbrella and was previously undocumented here. + +### [L1 Contracts] Inbox and Rollup views for streaming message consumption + +Additive. The Inbox exposes its bucket ring: `getCurrentBucketSeq()`, `getBucket(seq)` (reverts `Inbox__BucketOutOfWindow(seq, current)` for a future or evicted bucket), `getBucketAtOrBeforeTotal(total)` returning `(seq, bucket)` for the live bucket with the greatest cumulative message total at or below `total` (reverts `Inbox__NoBucketAtOrBeforeTotal(upperBound, oldestLiveTotal)` when the ring no longer holds any such bucket), `getProvenConsumedBucketSeq()` and `getRingHeadroom()`. `getState()` returns `{ rollingHash, totalMessagesInserted, currentBucketSeq }`. `sendL2Message` reverts with `Inbox__WouldOverwriteUnconsumedBucket(evictedBucketSeq)` when the 4096-bucket ring would wrap onto a bucket the proven chain has not consumed. + +The Rollup gains `validateCheckpointHeaderAndInbox(CheckpointPreflightArgs calldata) returns (uint64 bucketHint)`, the proposer's pre-publication simulation of `propose`: it derives the effective parent checkpoint from the Rollup's own storage at the simulated `block.timestamp` (`Rollup__UnexpectedParentCheckpoint(expected, actual)` if it is not the one the caller expected), runs the shared header checks, resolves `expectedTotal` to a live bucket (`Rollup__InboxTotalNotAtBucketBoundary(expected, actual)` if the total is not a bucket end) and applies the same settlement, cap and censorship rules as `propose` (`Rollup__InvalidInboxRollingHash`, `Rollup__InboxBucketStillMutable`, `Rollup__InboxConsumptionBehindParent`, `Rollup__TooManyInboxMessagesConsumed`, `Rollup__UnconsumedInboxMessages`). `propose` is unchanged: it still takes the unsigned `bucketHint` in `ProposeArgs` and checks the bucket against the signed header's `inboxRollingHash`. Portals need none of this; it documents what proposers call. + +### [Aztec Node] The archiver's L1-to-L2 message API is addressed by message count, not by Inbox bucket + +The node stores the Inbox as an ordered message log and no longer keeps L1's bucket partition. On `L1ToL2MessageSource` (the archiver, `AztecNode` consumers of it and the archiver JSON-RPC schema): + +- Removed: `getLatestInboxBucketAtOrBefore`, `getInboxBucket`, `getInboxBucketByTotalMsgCount`, `getL1ToL2MessagesBetweenBuckets`. +- `getL1ToL2MessagesBetweenLeafCounts(start, end)` keeps its name but the bounds are now plain cumulative message counts (L1-to-L2 tree leaf counts) that need not be bucket boundaries. An invalid range rejects with `Error(/Invalid Inbox leaf count range/)`; a range the archiver has not fully synced rejects with `InboxMessageRangeNotSyncedError(startLeafCount, endLeafCount, detail)` (`@aztec/archiver`), which replaces `InboxBucketBoundaryNotSyncedError` and `InboxBucketNotSyncedError`. An empty result always means the range holds no messages. +- New: `getMessagePosition(totalMessageCount)` returning `InboxMessagePosition = { totalMessageCount: bigint; rollingHash: Fr }` (zero count resolves to the zero hash; a count past the synced tip resolves to `undefined`), `getSyncedMessagePosition()`, and `getL1ToL2MessageRange(start, end)` returning `InboxMessageRange = { messages: Fr[]; start; end }` read from one store snapshot, so `end.rollingHash` authenticates exactly `messages`. Both types and their zod schemas are exported from `@aztec/stdlib/messaging`. + +In `@aztec/stdlib/messaging`, `InboxBucket`, `InboxBucketSchema`, `isInboxConsumptionSufficient` and `getInboxCutoffTimestamp` are removed, and `InboxBucketRef` is renamed `InboxMessagePrefixRef`: the signed one-field reference a block proposal carries is the rolling hash over the message prefix the block consumed through, interpreted together with the block header's `state.l1ToL2MessageTree.nextAvailableLeafIndex`; it may name a position inside an L1 bucket. `BlockProposal.bucketRef` and `CheckpointProposal.lastBlock.bucketRef` are renamed `inboxPrefixRef`, and the encoding shrinks with the rename: the old `InboxBucketRef` serialized a bucket sequence (`uint64`), a bucket timestamp (`uint64`) and the rolling hash (`Fr`) as 48 bytes, while `InboxMessagePrefixRef` serializes the rolling hash on its own as 32 bytes. Proposals therefore do not round-trip between old and new peers. Every node that gossips, signs or validates proposals on a network has to run a matching version, so the upgrade has to be coordinated across the peer set; there is no supported rolling upgrade in which both encodings circulate at once. `L2BlockSink.addBlock(block, inboxPrefixRef)` now requires the reference and rejects, inside the insertion transaction, a block whose reference does not match the sink's own messages (`InboxPrefixMismatchError`), whose consumed range the sink does not hold (`InboxPrefixNotSyncedError`), whose parent is missing (`ProposedBlockParentNotFoundError`) or whose consumption rewinds (`InboxConsumptionRewindsError`). + +In `@aztec/ethereum/contracts`, the top-level `MessageSentLog.l1BlockTimestamp` field is removed, since the log's args now carry the whole message and nothing derives a bucket cutoff from the L1 block's timestamp any more. The `InboxContract` helper that fetched those timestamps was private, so its removal changes no public API. `InboxContract.getBucketAtOrBeforeTotal(upperBound)` (resolving to `undefined` on `Inbox__NoBucketAtOrBeforeTotal`) and `RollupContract.validateCheckpointHeaderAndInbox(l1TxUtils, args, { time, stateOverrides, from })` are added. `MIN_BLOCKS_FOR_INBOX_CATCHUP`, the floor on a network's `maxBlocksPerCheckpoint`, drops from 7 to 4, since a block can now carry any 256 messages regardless of bucket layout. + +**Migration:** + +```diff +- const bucket = await source.getInboxBucketByTotalMsgCount(parentCount); +- const messages = await source.getL1ToL2MessagesBetweenBuckets(bucket.seq, endBucket.seq); ++ const { messages, end } = await source.getL1ToL2MessageRange(parentCount, endCount); ++ // end.rollingHash is the hash a block ending at endCount signs as its InboxMessagePrefixRef +``` + +**Impact**: Only node-internal and tooling consumers are affected; `getL1ToL2MessageIndex`, `getL1ToL2MessageMembershipWitness` and the Aztec.nr consumption functions are unchanged. Operators upgrading a node: see the node changelog for the archiver store reset. + +### [Aztec Node] Sequencer and validator streaming-Inbox internals + +For consumers wiring these packages directly. `Sequencer` takes an `InboxContract` constructor argument (after `rollupContract`) and `AutomineSequencerDeps` gains `inboxContract`; completion uses it to resolve a checkpoint's final message position to a live bucket. `NodePublicCallsSimulator` deps drop `l1Client` and `useAutomineSequencer`. `SequencerPublisher.validateCheckpointHeader` is replaced by `validateCheckpointHeaderAndInbox(header, { expectedTotal, expectedParentCheckpointNumber }, simulationOverridesPlan?)`, which returns the `bucketHint` to pass to `propose`. `@aztec/sequencer-client` no longer exports `InboxBucketConfirmationTracker`, `InboxBucketEligibility`, `L1BlockReader`, `immediateEligibility`, `ConsumedBucketCursor`, `InboxBucketSelection`, `InboxBucketSource`, `SelectInboxBucketInput` or `selectInboxBucketForBlock`; it exports `InboxConsumptionCaps`, `PROTOCOL_INBOX_CONSUMPTION_CAPS`, `InboxEndpointResolver`, `StreamingMessageSource`, `selectOrdinaryMessageEnd`, `selectSafeLocalEnd`, `getOrdinaryCeiling`, `mustQueryEndpoint`, `getEndpointUpperBound` and `resolveEndpoint` instead. These helpers are stateless: nothing is retained between blocks. `selectOrdinaryMessageEnd` returns the greedy end of the locally observed messages under the per-block and checkpoint caps; `getOrdinaryCeiling` is the threshold one bucket's worth of messages below the checkpoint cap, and `selectSafeLocalEnd` holds the greedy end down to it. `mustQueryEndpoint` decides, from that prospective end, whether the block has to resolve a live L1 bucket end at all, and `resolveEndpoint` performs the single Inbox lookup (bounded by `getEndpointUpperBound`) and authenticates the resolved end against the local message log in one snapshot. + +Validator result reasons: the checkpoint reason `inbox_consumption_insufficient` is removed (only the proposer's preflight and L1 assess censorship now), and the block reasons `bucket_unknown`, `bucket_hash_mismatch`, `parent_bucket_unresolved` and `bucket_moves_backwards` are replaced by `inbox_prefix_unavailable`, `inbox_prefix_mismatch` and `consumption_moves_backwards`. `inbox_prefix_unavailable` and `inbox_prefix_mismatch` exist for both block and checkpoint proposals, are recorded as `unvalidated`, are retried through a bounded local L1 sync, and never produce a slashing event, a peer penalty or an invalid-proposal-slot marker. + ### [Aztec.js] Contract artifacts preserve the names of `#[abi(tag)]` globals Globals exported from a Noir contract with `#[abi(tag)]` now keep their names in the artifact: entries in `ContractArtifact.outputs.globals` are `{ name, value }` objects as emitted by the compiler, where the names used to be stripped on load. This lets TypeScript read contract constants by name instead of duplicating their values: diff --git a/docs/docs-developers/docs/tutorials/js_tutorials/aave_bridge.md b/docs/docs-developers/docs/tutorials/js_tutorials/aave_bridge.md index 029ec76fcbe3..9fd95032f437 100644 --- a/docs/docs-developers/docs/tutorials/js_tutorials/aave_bridge.md +++ b/docs/docs-developers/docs/tutorials/js_tutorials/aave_bridge.md @@ -96,7 +96,7 @@ sequenceDiagram Portal->>Aave: withdraw(aTokenAmount) Aave-->>Portal: underlying + yield Portal->>Inbox: sendL2Message(content) - Note over Inbox: Wait for an L2 block to include the message (12-30s) + Note over Inbox: Wait for the next L2 block to include the message User->>Bridge: claim_public(amount_with_yield) Bridge->>Inbox: consume_l1_to_l2_message Bridge->>Token: mint_to_public(user, amount_with_yield) @@ -441,7 +441,9 @@ Extract the message leaf index: #include_code get_claim_leaf_index /docs/examples/ts/aave_bridge/index.ts typescript -On the local network, L2 blocks are only produced when transactions are submitted. An L1-to-L2 message can only be consumed once an L2 block includes it, and the local sandbox includes it as soon as it sees it. This utility deploys two dummy contracts (with random salts for unique addresses) to force block production. On devnet or testnet, blocks are produced continuously and this step is unnecessary, but there the sequencer first waits for the L1 block carrying the message to gain a child, which takes one more L1 block (around 12 to 14 seconds): +On devnet or testnet, blocks are produced continuously. A proposer consumes the messages its node has already observed in a mined L1 block into the next L2 block it builds: there is no confirmation depth and no wait for the L1 block to gain a child. Observing a message is not the same as inserting it, though, and an inserted message is only usable once the chain tip your claim is simulated against has grown past the message's leaf index. Inclusion can still be deferred by a backlog above the per-block or per-checkpoint message caps, or by a checkpoint that fails to build or publish, so treat any particular delay as an observation rather than a guarantee. Instead of sleeping for a fixed interval, wait on readiness with `waitForL1ToL2MessageReady(node, messageHash, { chainTip })` from `@aztec/aztec.js/messaging`, passing the tip the consuming transaction anchors to (`'latest'` by default, or `'proven'` if that is what your PXE syncs to). + +On the local network, blocks are produced only when transactions are submitted, so nothing will insert the message until something else is sent. This utility deploys two dummy contracts (with random salts for unique addresses) to force block production: #include_code mine_blocks /docs/examples/ts/aave_bridge/index.ts typescript @@ -501,7 +503,7 @@ If `claim_public` reverts, ensure you called `set_minter(l2Bridge.address, true) ### L1→L2 message not found — claim reverts after mining blocks -An L1-to-L2 message becomes consumable once an L2 block includes it, which takes 12 to 30 seconds after the L1 transaction. Make sure `mine2Blocks` runs before the claim. If the issue persists, verify the `messageLeafIndex` extracted from the `MessageSent` event is correct. +A message emitted by `MessageSent` on L1 is not yet consumable. It becomes consumable once an L2 block has inserted it and the chain tip your claim is simulated against has grown past its leaf index, and there is no fixed delay that guarantees either step. On the local network, make sure `mine2Blocks` runs before the claim so a block is produced at all. On any network, wait on readiness rather than on a timer: call `waitForL1ToL2MessageReady` (or poll `isL1ToL2MessageReady`) against the tip the claim will anchor to. If readiness never arrives, verify the `messageLeafIndex` extracted from the `MessageSent` event is correct. ## Next Steps diff --git a/docs/docs-developers/docs/tutorials/js_tutorials/token_bridge.md b/docs/docs-developers/docs/tutorials/js_tutorials/token_bridge.md index 230cc542274b..ffbf22bf0265 100644 --- a/docs/docs-developers/docs/tutorials/js_tutorials/token_bridge.md +++ b/docs/docs-developers/docs/tutorials/js_tutorials/token_bridge.md @@ -443,7 +443,7 @@ Use viem to extract this information: #include_code get_message_leaf_index /docs/examples/ts/token_bridge/index.ts typescript -This extracts the logs from the deposit and retrieves the leaf index. You can now claim it on L2. However, a message can only be claimed once an L2 block includes it. On a live network the sequencer first waits for the L1 block carrying the message to gain a child, which takes one more L1 block (around 12 to 14 seconds), so expect 15 to 30 seconds of latency; the local sandbox consumes the message as soon as it sees it. If you called `claim` on the L2 contract immediately, it would return "no message available". +This extracts the logs from the deposit and retrieves the leaf index. You can now claim it on L2. However, a message can only be claimed once an L2 block includes it. On a live network the proposer normally includes the message in the next L2 block it builds after its node observes the L1 block carrying it, within a few seconds of that block being mined, though near the end of a checkpoint it may wait for the next one; the local sandbox consumes the message as soon as it sees it. If you called `claim` on the L2 contract immediately, it would return "no message available". On a local network blocks are only produced when transactions are submitted, so add a utility function that forces a couple of blocks (it deploys a contract with a random salt): diff --git a/docs/docs-developers/docs/tutorials/js_tutorials/uniswap_swap.md b/docs/docs-developers/docs/tutorials/js_tutorials/uniswap_swap.md index c921285d639a..4cfb0c562e41 100644 --- a/docs/docs-developers/docs/tutorials/js_tutorials/uniswap_swap.md +++ b/docs/docs-developers/docs/tutorials/js_tutorials/uniswap_swap.md @@ -309,7 +309,7 @@ Bridge WETH from L1 to L2: When depositing from L1 to L2, we use a secret/secret-hash pattern: generate a random secret on the client, send only the hash to L1 (in the deposit transaction), then later reveal the secret on L2 to claim the tokens. This prevents **front-running attacks**: a malicious sequencer (the node that orders and processes L2 transactions) cannot observe the L1 deposit and claim the tokens themselves because they don't know the secret. Only someone who knows the preimage can claim. ::: -Before claiming, we need to mine a couple of L2 blocks. An L1-to-L2 message is not available the moment it is sent -- it becomes consumable only once an L2 block includes it. The local sandbox includes it as soon as it sees it; on a live network the sequencer first waits for the L1 block carrying it to gain a child, which takes one more L1 block (around 12 to 14 seconds). We use a helper that deploys throwaway contracts to force these blocks: +Before claiming, we need to mine a couple of L2 blocks. An L1-to-L2 message is not available the moment it is sent -- it becomes consumable only once an L2 block includes it. The local sandbox includes it as soon as it sees it; on a live network the proposer normally includes it in the next L2 block it builds after observing the L1 block carrying it (near the end of a checkpoint it may wait for the next one). We use a helper that deploys throwaway contracts to force these blocks: #include_code mine_blocks /docs/examples/ts/example_swap/index.ts typescript diff --git a/docs/docs-operate/operators/reference/changelog/index.md b/docs/docs-operate/operators/reference/changelog/index.md index f73aa4071179..b2a0cef51192 100644 --- a/docs/docs-operate/operators/reference/changelog/index.md +++ b/docs/docs-operate/operators/reference/changelog/index.md @@ -11,6 +11,23 @@ This changelog documents all configuration changes, new features, and breaking c ## Version history +### [v6.0.0](./v6.md) + +The streaming Inbox (Fast Inbox) consumes L1-to-L2 messages into the next L2 block the proposer builds, with an archiver store reset on upgrade. + +**Key changes:** +- Archiver store format changed (`ARCHIVER_DB_VERSION` 10, no migration): the node resets and resyncs its archiver on first start. This is the only store the streaming Inbox changes; other v6 work bumps other stores independently +- Messages are eligible for the next L2 block once observed on L1; only a checkpoint's final message position must be an Inbox bucket end +- Validators retry, and never penalize, local Inbox prefix disagreements (`inbox_prefix_unavailable`, `inbox_prefix_mismatch`) +- L1 reorg recovery rolls the message log back to an authenticated anchor before refetching, dropping messages past it (and the proposed blocks that consumed them) even when L1 still holds the same content +- `maxBlocksPerCheckpoint` floor lowered from 7 to 4; no configuration changes required + +**Migration difficulty**: Low + +[View full changelog →](./v6.md) + +--- + ### [v5.2.0](./v5.2.md) Node-operator hardening release: peerless nodes stop acting, slashing votes against your own validators are surfaced, and the RPC server gains timeout and CORS configuration. diff --git a/docs/docs-operate/operators/reference/changelog/v6.md b/docs/docs-operate/operators/reference/changelog/v6.md new file mode 100644 index 000000000000..e1407301e32a --- /dev/null +++ b/docs/docs-operate/operators/reference/changelog/v6.md @@ -0,0 +1,71 @@ +--- +title: v6.0.0 +description: "Operator-facing changes in the v6.0.0 release: the streaming Inbox consumes L1-to-L2 messages as they are observed, with an archiver store reset on upgrade." +displayed_sidebar: operatorsSidebar +--- + +## Overview + +**Migration difficulty**: low. No configuration changes are required. The archiver data store format changed (`ARCHIVER_DB_VERSION` 10) with no migration path, so on first start the node resets its archiver store and resyncs L2 state from L1. Plan for the same downtime as an initial sync. Every other setting keeps its previous meaning. + +## Breaking changes + +### Archiver store reset on first start + +The archiver now stores L1-to-L2 messages as a plain ordered log (compact index, leaf, L1 block number, consensus rolling hash) and no longer records L1's Inbox bucket partition. There is no migration from the previous layout. When the node opens a store written by an earlier version it logs that the schema version changed, resets the archiver data directory and syncs L2 state from L1 again, exactly as a fresh node would. Nothing needs to be deleted by hand; if you prefer to clear the directory yourself, follow the procedure under [SYNC_BLOCK Failed Error](/operate/operators/operator-faq#sync_block-failed-error) in the FAQ. + +The archiver is the only store the streaming Inbox changes; world state follows the archiver through the block stream, and the L1 transaction state store is unaffected. Other v6 work bumps other stores on its own schedule: `PXE_DATA_SCHEMA_VERSION` moves from 13 to 14 in this release for unrelated reasons, so a PXE reset in v6 is not caused by the streaming Inbox. + +### `maxBlocksPerCheckpoint` floor lowered from 7 to 4 + +The network consensus configuration requires `maxBlocksPerCheckpoint` to be at least `MIN_BLOCKS_FOR_INBOX_CATCHUP`, so a checkpoint can always clear a cap-sized backlog of mandatory L1-to-L2 messages. Blocks now consume messages by count rather than by whole L1 bucket, so the floor is `ceil(1024 / 256) = 4` instead of 7. Existing configurations remain valid; networks that were forced to shorten their block duration to satisfy the old floor may relax it. + +## Behaviour changes + +### L1-to-L2 messages enter the next L2 block the proposer builds + +A proposer consumes the messages its archiver has observed in mined L1 blocks into the next ordinary L2 block it builds, up to 256 messages per block and 1024 per checkpoint. There is no confirmation depth, no message age requirement and no bucket-eligibility rule; intermediate blocks may end anywhere in the message sequence, including inside an L1 bucket. Only a checkpoint's final message position has to coincide with a live Inbox bucket end, which is what `Rollup.propose` checks against the header's `inboxRollingHash`. A block consults L1 only when it is the checkpoint's final block or when its prospective end would pass the checkpoint's start count plus 768 messages, where a greedy step could leave the last endpoint the cap allows behind. It then resolves one with a single `Inbox.getBucketAtOrBeforeTotal` call, bounded by the checkpoint cap on a non-final block and additionally by that block's own 256-message reach on the final one. A non-final block consumes toward the resolved end and may stop inside a bucket; the final block ends on it. Nothing is retained between blocks and consumption is never frozen: each block decides again, so a bucket that closes later in the slot can still be taken. If the sub-slot schedule runs out before the checkpoint ends on a bucket end, the proposer builds one extra transaction-less block to reach it, bounded by the last block build time its timetable budgets ahead of the checkpoint proposal receive deadline; past that time it still tries, and the proposal may arrive late. Messages beyond the per-block cap wait for a later block of the same checkpoint; messages past the selected end, or beyond the remaining checkpoint capacity, wait for a later checkpoint. + +Before gossiping a checkpoint and again immediately before publishing it, the proposer simulates the Rollup's header and Inbox checks through `Rollup.validateCheckpointHeaderAndInbox` at the last L1 slot of the target L2 slot; both runs deliberately skip the attestation-signature and data-availability checks, including the publication run that happens after attestations were collected. The pre-gossip run is conditional on the simulated parent it expects (an unpublished pipelined parent, if any); the pre-publication run uses the real Rollup state plus any invalidation the proposer bundles ahead of `propose`, except that while an epoch proof is still expected to land by the target slot it keeps assuming the proven tip the checkpoint was built against, so its verdict at an epoch boundary is conditional on that proof landing and `propose` itself remains the final check; the bucket hint it returns is the one sent with `propose`. A checkpoint whose blocks the local archiver no longer holds is not published. + +Log and metric reasons you may see from the proposer: + +- `inbox_completion_unresolved`: no live bucket end was reachable for the final block, or for the extra block that ends a checkpoint whose schedule ran out. It also covers the case where that extra transaction-less block was resolved but could not be built, so the reason on its own does not distinguish an endpoint problem from a build failure; the accompanying `phase` field does. +- `inbox_prefix_reorged`: an L1 reorg changed messages the checkpoint already consumed. +- `inbox_range_unavailable`: the archiver could not serve the consumed range. +- `header_validation_failed` and `header_validation_timeout`: the pre-gossip header and Inbox preflight was rejected, or did not return before the proposal send deadline. +- `publication_preflight_failed` and `publication_preflight_timeout`: the same preflight immediately before publication was rejected, or did not return before the L1 publish deadline. +- `checkpoint_blocks_pruned`: the local archiver no longer holds the blocks the checkpoint would publish. + +All of them abandon the slot without signing a conflicting checkpoint. + +### Validators no longer penalize local Inbox disagreement + +A block proposal carries a signed reference to the message prefix it consumed through. A validator whose own view of L1 cannot confirm that prefix (`inbox_prefix_unavailable`) or disagrees with it (`inbox_prefix_mismatch`) retries after a bounded archiver sync within the attestation deadline. Both outcomes are recorded as `unvalidated`; neither is a slashable offense, a peer penalty or an invalid-proposal-slot marker, even if the validator cannot catch up in time. The previous `bucket_unknown`, `bucket_hash_mismatch`, `parent_bucket_unresolved` and `inbox_consumption_insufficient` reasons are gone: validators check message content, not L1 bucket layout or censorship rules, which only the proposer's preflight and `propose` assess. + +### L1 reorg recovery rolls back to the newest message it can still find + +When the Inbox position at the L1 head disagrees with the local log, the archiver finds an anchor: a shorter canonical prefix matched by hash, or a stored message L1 still emits at the same index and hash within five L1 blocks of the height the node recorded for it, searched backwards with at most 32 event lookups per sync pass. It then rolls the log back to that anchor in one transaction, which deletes every message past it, rewinds the sync point and prunes the not-yet-checkpointed blocks that consumed more messages than the anchor retains (prune metric type `inbox_reorg`). The deleted messages are re-fetched by ordinary forward sync on the next pass. + +The rollback happens before the refetch, so it is conservative: a message whose lookup misses is dropped even if L1 still has it unchanged, and the proposed blocks that consumed it are dropped with it, even though the same content comes straight back. A reorg that moves messages far from the heights the node recorded, or an L1 provider that answers `eth_getLogs` with an empty result instead of an error, therefore costs locally proposed work that was never invalid. Published checkpoints are never deleted by this path, and L1 remains the authority on what the log should contain. Recovery needs an L1 provider that can serve `eth_getLogs` for the Inbox back to the anchor; a block the provider cannot serve is reported as a failure, never treated as an absence of messages. + +While a rollback reaches below the checkpointed tip, the node withholds proposed-checkpoint data until the checkpointed tip's `inboxRollingHash` agrees with the message log again, so nothing builds on blocks whose messages the node cannot currently serve. + +### Accepted limitations + +These are consequences of consuming messages immediately, accepted by design rather than bugs to expect fixes for: + +- An L1 reorg that removes the bucket end a completed checkpoint relied on costs that checkpoint's publication; the node abandons the slot rather than re-signing a different checkpoint for the same duty. +- Committee attestations prove that the checkpoint's message content matches the committee's view of L1, not that it is publishable. The proposer's preflight and `propose` are authoritative for settlement, cap and censorship. +- An L1 reorg that becomes visible only after `propose` was submitted can revert it and spend L1 gas despite a passing preflight. +- Recovery inherits a shortcut that trusts a stored message as final once its recorded L1 height is at or below the finalized block the node last recorded on an agreeing sync. Recorded heights are refreshed only when a matching message is re-fetched and reinserted (forward ingestion, including the refetch after a rollback); a same-content reorg that leaves the Inbox position at the head unchanged is taken as agreement without re-fetching and leaves them stale, so a message re-mined above finality can later be trusted as final. This is documented in the archiver and deliberately deferred. + +## New L1 views + +Additive, for tooling and monitoring. On the Inbox: `getCurrentBucketSeq()`, `getBucket(seq)`, `getBucketAtOrBeforeTotal(total)`, `getProvenConsumedBucketSeq()` and `getRingHeadroom()`; on the Rollup: `validateCheckpointHeaderAndInbox(CheckpointPreflightArgs)`. New revert reasons: `Inbox__NoBucketAtOrBeforeTotal(upperBound, oldestLiveTotal)`, `Rollup__InboxTotalNotAtBucketBoundary(expected, actual)`, `Rollup__UnexpectedParentCheckpoint(expected, actual)`. + +`getRingHeadroom()` is worth alerting on. The Inbox holds a ring of 4096 buckets and refuses a `sendL2Message` that would overwrite a bucket the proven chain has not consumed (`Inbox__WouldOverwriteUnconsumedBucket`). A proving stall long enough to exhaust the headroom halts L1-to-L2 sends for every portal until the next epoch proof lands; at one bucket per message-bearing L1 block the ring covers roughly 13.6 hours of stalled consumption, less under heavy message traffic. Nothing is lost when this happens, and sends resume automatically. + +## Configuration + +No flags or environment variables were added, removed or given new defaults by this change. The `SEQ_INBOX_L1_CONFIRMATIONS` setting discussed during development never shipped. diff --git a/docs/docs-operate/operators/reference/glossary.md b/docs/docs-operate/operators/reference/glossary.md index db6a5c4cd5e0..e519f4931dbb 100644 --- a/docs/docs-operate/operators/reference/glossary.md +++ b/docs/docs-operate/operators/reference/glossary.md @@ -113,7 +113,7 @@ The process of calculating the expected gas cost for an Ethereum transaction bef ### Inbox -The L1 contract that receives messages sent from Ethereum to Aztec L2. The archiver monitors the Inbox for new L1-to-L2 messages. +The L1 contract that receives messages sent from Ethereum to Aztec L2. It commits to every message it accepts with a rolling hash, snapshotted per L1 block into a ring of buckets. The archiver monitors the Inbox for new L1-to-L2 messages, and the proposer includes the messages its node has observed in the L2 blocks it builds. ## J diff --git a/docs/sidebars-operate.js b/docs/sidebars-operate.js index 547fb9fea079..5c94de6569a0 100644 --- a/docs/sidebars-operate.js +++ b/docs/sidebars-operate.js @@ -234,6 +234,7 @@ const referenceItems = [ id: "operators/reference/changelog/changelog", }, items: [ + "operators/reference/changelog/v6", "operators/reference/changelog/v5.2", "operators/reference/changelog/v4.3", "operators/reference/changelog/v4.2", diff --git a/l1-contracts/test/RollupFieldRange.t.sol b/l1-contracts/test/RollupFieldRange.t.sol index 0c583d9e6684..f79276e1d62a 100644 --- a/l1-contracts/test/RollupFieldRange.t.sol +++ b/l1-contracts/test/RollupFieldRange.t.sol @@ -175,7 +175,9 @@ contract RollupFieldRangeTest is RollupBase { vm.blobhashes(this.getBlobHashes(full.checkpoint.blobCommitments)); - // Streaming Inbox: nothing is seeded here, so reference the genesis bucket (hash 0). + // Streaming Inbox: `_populateInbox` above did seed messages, but consuming them is optional at this timestamp, + // since their buckets are not yet past the censorship cutoff. The genesis bucket (hash 0, total 0) is therefore + // still a legal endpoint for this checkpoint. header.inboxRollingHash = bytes32(0); ProposeArgs memory args = diff --git a/noir-projects/fnd/noir-protocol-circuits/crates/rollup-lib/src/checkpoint_root/components/checkpoint_root_inputs_validator.nr b/noir-projects/fnd/noir-protocol-circuits/crates/rollup-lib/src/checkpoint_root/components/checkpoint_root_inputs_validator.nr index aa2dffe501ae..152d4ce9602c 100644 --- a/noir-projects/fnd/noir-protocol-circuits/crates/rollup-lib/src/checkpoint_root/components/checkpoint_root_inputs_validator.nr +++ b/noir-projects/fnd/noir-protocol-circuits/crates/rollup-lib/src/checkpoint_root/components/checkpoint_root_inputs_validator.nr @@ -81,10 +81,11 @@ impl CheckpointRootInputsVal /// values across the rest of the checkpoint. Do not drop or weaken any of them on the assumption that some flag /// still marks the first block — none does. /// - /// Together they also pin the first-block behaviour of the block roots themselves: a block absorbs the l1-to-l2 - /// tree root into its blob data exactly when its start sponge blob is still uninitialized, which the sponge blob - /// assert below allows only for the leftmost block (every block absorbs its block-end fields, so a mid-checkpoint - /// block always inherits a sponge blob with fields already absorbed). + /// Together they also pin the first-block behaviour of the block roots themselves. Every block absorbs its own + /// block-end fields, the l1-to-l2 message tree root among them, since any block may insert its own bundle; the + /// root is per-block, not checkpoint-constant. What distinguishes the leftmost block is therefore the *start* + /// sponge blob: only it may still be uninitialized, which the sponge blob assert below is what enforces, so a + /// mid-checkpoint block always inherits a sponge blob with fields already absorbed. fn validate_start_states(self) { let first_rollup = self.previous_rollups[0].public_inputs; diff --git a/noir-projects/fnd/noir-protocol-circuits/crates/types/src/constants.nr b/noir-projects/fnd/noir-protocol-circuits/crates/types/src/constants.nr index 1c68751e7200..56b554de5088 100644 --- a/noir-projects/fnd/noir-protocol-circuits/crates/types/src/constants.nr +++ b/noir-projects/fnd/noir-protocol-circuits/crates/types/src/constants.nr @@ -62,7 +62,9 @@ pub global NOTE_HASH_SUBTREE_ROOT_SIBLING_PATH_LENGTH: u32 = pub global NULLIFIER_SUBTREE_ROOT_SIBLING_PATH_LENGTH: u32 = NULLIFIER_TREE_HEIGHT - NULLIFIER_SUBTREE_HEIGHT; // Cap on L1-to-L2 messages bundled into a single L2 block. One quarter of the per-checkpoint -// cap: a checkpoint drains its Inbox consumption across up to four message-bearing blocks. +// cap, so four blocks at full bundles are the fewest that can drain a checkpoint's whole Inbox +// allowance. It is a floor, not a ceiling: blocks end at arbitrary message prefixes, so a +// checkpoint may spread the same allowance over any number of partially filled blocks. pub global MAX_L1_TO_L2_MSGS_PER_BLOCK: u32 = 256; // Cap on L1-to-L2 messages consumed by a single checkpoint. pub global MAX_L1_TO_L2_MSGS_PER_CHECKPOINT: u32 = 1024; diff --git a/spartan/environments/network-defaults.yml b/spartan/environments/network-defaults.yml index 98470816537f..323cee8232dc 100644 --- a/spartan/environments/network-defaults.yml +++ b/spartan/environments/network-defaults.yml @@ -225,9 +225,10 @@ networks: AZTEC_LAG_IN_EPOCHS_FOR_RANDAO: 1 # Single-validator dev network. AZTEC_TARGET_COMMITTEE_SIZE: 1 - # 3s blocks rather than the 6s of the 72s-slot networks: at 6s a 36s slot only fits 4 blocks per checkpoint, - # below MIN_BLOCKS_FOR_INBOX_CATCHUP, which leaves the network haltable by an adversarial L1-to-L2 message - # pattern no checkpoint can consume within its block budget. + # 3s blocks rather than the 6s of the 72s-slot networks: at 6s a 36s slot only fits 4 blocks per checkpoint. + # Four exactly meets MIN_BLOCKS_FOR_INBOX_CATCHUP (ceil(1024 / 256)), so it is legal but leaves no slack: every + # one of the four would have to carry a full 256-message bundle to drain a mandatory backlog within the slot, + # and any block opportunity lost to timing puts the checkpoint below the floor. 3s blocks give room to spare. SEQ_BLOCK_DURATION_MS: 3000 # Must equal what the default proposer budgets derive for a 36s slot / 3s block (see chain_l2_config.test.ts). MAX_BLOCKS_PER_CHECKPOINT: 9 diff --git a/yarn-project/archiver/README.md b/yarn-project/archiver/README.md index f11bdeb96678..51f6adcd6991 100644 --- a/yarn-project/archiver/README.md +++ b/yarn-project/archiver/README.md @@ -42,13 +42,22 @@ Two independent syncpoints track progress on L1: - `blocksSynchedTo`: L1 block number for checkpoint events - `messagesSynchedTo`: L1 block ID (number + hash) for messages +Message progress is actually two separate pointers, and the difference matters: +- The **scanned cursor** is the L1 block through which `MessageSent` logs were read into the store. It moves with every + validated batch and says only which L1 blocks were queried, not that their responses were complete. +- The **certified syncpoint** (`messagesSynchedTo`) is the L1 block at which the *whole* stored log was found equal to + the Inbox's own position at a canonical captured head. Only a syncpoint may answer "the node is synced to this head". + +A batch that has not been through such a comparison advances the scanned cursor and clears the syncpoint. Its messages +stay in the store and stay usable locally; what is withheld is the claim that the log matches L1 at that height. + ### L1-to-L2 Messages Messages are synced from the Inbox contract by `InboxMessageSynchronizer`. Each sync pass captures the L1 head, reads the Inbox's position at that head (cumulative message count and consensus rolling hash) and compares it with the local message log. 1. If the persisted message syncpoint already is the captured head (number and hash), there is nothing to do. 2. If the local position equals the Inbox's, only the syncpoint (and the finalized-block marker) is updated. -3. Otherwise `MessageSent` events are fetched forward from the syncpoint in bounded L1 block batches. Each batch is validated (contiguous compact indices, unbroken rolling-hash chain) and committed together with the syncpoint covering it, so completed batches are usable at once and a later RPC failure leaves them in place. Oversized log ranges are bisected at block boundaries; a single block the provider cannot serve is reported as a failure, never as an absence of messages. +3. Otherwise `MessageSent` events are fetched forward from the scanned cursor in bounded L1 block batches. Each batch is validated (contiguous compact indices, unbroken rolling-hash chain) and committed together with the scanned cursor that covers it, so completed batches are usable at once and a later RPC failure leaves them in place. None of these intermediate batches has been compared with the Inbox's position, so none of them moves the syncpoint: they clear it instead. The batch reaching the captured head is committed with that head as the syncpoint only once the position after it equals the captured one, which certifies the intermediate batches along with it. Oversized log ranges are bisected at block boundaries; a single block the provider cannot serve is reported as a failure, never as an absence of messages. 4. After the fetch the local position is compared with the captured one again. A disagreement means an L1 reorg changed messages the node already holds, and recovery starts. Recovery is pinned to the captured head and bounded per pass: @@ -90,7 +99,7 @@ The `blocksSynchedTo` syncpoint is updated: Note that the `blocksSynchedTo` pointer is NOT updated during normal sync when there are no new checkpoints. This protects against small L1 reorgs that could add a checkpoint on an L1 block we have flagged as already synced. -The `messagesSynchedTo` pointer is always advanced to the current L1 block on success. If a rolling hash mismatch or post-download inconsistency is detected, the pointer rolls back to the last common message and the operation retries. The rolling hash chain and pre/post-sync consistency checks provide the primary reorg protection. +On the message side the scanned cursor advances with every committed message batch, while `messagesSynchedTo`, the certified syncpoint, only moves to the captured L1 head once the whole local log agrees with the Inbox's position at that head; an uncertified batch clears it. On a disagreement the recovery transaction described above rewinds the cursor to just before the anchor's L1 block and clears the syncpoint, so the syncpoint is never ahead of content the node has actually compared with the Inbox, and the messages it no longer certifies are the ones the conservative rollback has already deleted. The rolling hash chain and the pre/post-sync position comparison provide the primary reorg protection. ### Block Queue @@ -124,7 +133,7 @@ Use checkpointed queries when the result must reflect L1 state (e.g., determinin Both message and checkpoint sync detect L1 reorgs by comparing local state against L1. When detected, they find the last common ancestor and rollback. -**Messages**: Each stored message includes its rolling hash. During sync, if the local last message's rolling hash doesn't match L1, the archiver walks backwards through local messages, querying L1 for each one, until it finds a message with a matching rolling hash. Everything after that message is deleted, and the syncpoint is rolled back. +**Messages**: Each stored message includes its rolling hash. During sync, if the local position doesn't match the Inbox's at the captured L1 head, the archiver finds an anchor (a shorter canonical prefix matched by hash, or a stored message L1 still emits at the same index and hash near its recorded height) and rolls the log back to it, dropping every message past the anchor and every proposed block that consumed one, before ordinary forward sync re-fetches the canonical suffix. Unchanged messages the bounded lookup cannot place are dropped and re-fetched too; a re-mine the node can still place changes nothing. See "L1-to-L2 Messages" above for the bounded, per-pass procedure. **Checkpoints**: When the archiver queries the Rollup contract for the archive root at the local pending checkpoint number, and it doesn't match the local archive root, the local checkpoint is no longer in L1's chain. The archiver walks backwards through local checkpoints, querying `archiveAt()` for each, until it finds one that matches. All checkpoints after that are unwound. diff --git a/yarn-project/archiver/src/errors.ts b/yarn-project/archiver/src/errors.ts index 1ac09cc66293..1b64961a7565 100644 --- a/yarn-project/archiver/src/errors.ts +++ b/yarn-project/archiver/src/errors.ts @@ -124,6 +124,10 @@ export class InboxMessagePrefixChangedError extends Error { * Thrown when a cumulative Inbox message-count range is not fully backed by the messages this archiver has synced, * either because it reaches past the synced tip or because the store is missing a message the range needs. * Distinguishes "not available locally, retry once L1 sync catches up" from a genuinely empty range. + * + * The distinction is only available in process. Across JSON-RPC this arrives as a generic `Error` carrying the + * message text alone, so remote callers cannot recover the class or the `startLeafCount`/`endLeafCount` fields and + * instead treat any range failure as unavailability. Do not add behaviour that depends on `instanceof` surviving. */ export class InboxMessageRangeNotSyncedError extends Error { constructor( diff --git a/yarn-project/aztec-node/src/aztec-node/node_public_calls_simulator.ts b/yarn-project/aztec-node/src/aztec-node/node_public_calls_simulator.ts index c5afa85dc68b..0cd41b8ffa87 100644 --- a/yarn-project/aztec-node/src/aztec-node/node_public_calls_simulator.ts +++ b/yarn-project/aztec-node/src/aztec-node/node_public_calls_simulator.ts @@ -91,9 +91,10 @@ export interface NodePublicCallsSimulatorDeps { * applying the same `SimulationOverridesPlan` the sequencer applies so the simulated mana min fee * matches what the sequencer will write into the block header. * - * Either way it also predicts the L1-to-L2 message bundle the next block would consume and appends it + * Either way it also estimates the L1-to-L2 message bundle the next block would consume and appends it * to the fork, so a transaction consuming a message that is in the Inbox but not yet in a block - * simulates against the state it will actually run in. + * simulates against something close to the state it will run in. That estimate is best effort and can + * differ from what the block actually consumes, as `appendPredictedL1ToL2Messages` describes. */ export class NodePublicCallsSimulator { private readonly blockSource: L2BlockSource; @@ -241,22 +242,30 @@ export class NodePublicCallsSimulator { } /** - * Appends the L1-to-L2 messages the next block would consume to the simulation fork, so a transaction consuming - * a message that has reached the Inbox but no block yet simulates against the state it will run in. Runs the same - * local-only part of the sequencer's selection: every message the archiver has observed, up to the per-block cap - * and the threshold one bucket below the checkpoint cap. + * Appends the L1-to-L2 messages the next block is expected to consume to the simulation fork, so a transaction + * consuming a message that has reached the Inbox but no block yet simulates against something close to the state + * it will run in. Runs the same local-only part of the sequencer's selection: every message the archiver has + * observed, up to the per-block cap and the threshold one bucket below the checkpoint cap. * - * This is a lower bound, not the sequencer's choice. Above that threshold, and on a checkpoint's final block, the - * sequencer's end depends on a live L1 bucket end it reads from the Inbox and this node does not, so the - * prediction stops where the local log alone is authoritative. + * The result is best effort, and neither an upper nor a lower bound on what the next block takes. Above that + * threshold, and on a checkpoint's final block, the end comes from a live L1 bucket boundary the sequencer reads + * from the Inbox and this node does not, and that boundary can sit below the local estimate. With a cursor of 0, + * 400 messages observed and live buckets ending at 200 and 400, this appends 256 while a final block lands on + * 200: a public call consuming message index 220 simulates successfully and then fails when it runs for real. + * Callers that need certainty check inclusion at an L2 tip that already exists, with `isL1ToL2MessageReady` from + * `@aztec/aztec.js/messaging`. * - * Best-effort. Any failure, such as messages not synced yet or a torn archiver snapshot, leaves the fork at the - * tip state, which is what the transaction sees if the next block consumes nothing. + * Any failure, such as messages not synced yet or a torn archiver snapshot, leaves the fork at the tip state, + * which is what the transaction sees if the next block consumes nothing. */ private async appendPredictedL1ToL2Messages( fork: MerkleTreeWriteOperations, opts: { - /** Last block of the checkpoint the next block extends; undefined when the next block opens a checkpoint. */ + /** + * Last block of the *parent* checkpoint, the one the in-progress checkpoint starts after, whose L1-to-L2 leaf + * count is the origin of the per-checkpoint cap. It is not a block of the checkpoint being extended. Undefined + * when the next block opens a checkpoint, in which case the tip is the origin. + */ checkpointStartBlock: BlockNumber | undefined; }, ): Promise { diff --git a/yarn-project/aztec.js/src/utils/cross_chain.ts b/yarn-project/aztec.js/src/utils/cross_chain.ts index 1f1a813ddcc7..83ee624929dd 100644 --- a/yarn-project/aztec.js/src/utils/cross_chain.ts +++ b/yarn-project/aztec.js/src/utils/cross_chain.ts @@ -4,7 +4,10 @@ import type { BlockTag } from '@aztec/stdlib/block'; import type { AztecNode } from '@aztec/stdlib/interfaces/client'; /** - * Waits for the L1 to L2 message to be ready to be consumed. + * Waits for the L1 to L2 message to be ready to be consumed, that is, for a block that already exists to have + * inserted it into the message tree at the given tip. This is a check on the chain as it stands, not a promise about + * a future block: a node simulating public calls may optimistically include messages that are still only in the + * Inbox, and the block that eventually inserts them can stop short of the one being waited for. * @param node - Aztec node instance used to obtain the information about the message * @param l1ToL2MessageHash - Hash of the L1 to L2 message * @param opts - Options @@ -31,7 +34,10 @@ export function waitForL1ToL2MessageReady( } /** - * Returns whether the L1 to L2 message is ready to be consumed. + * Returns whether the L1 to L2 message is ready to be consumed: whether a block at `chainTip` has already inserted + * it into the message tree. A message that is in the Inbox but not yet in any block is not ready, even though a node + * may already simulate public calls against it; use this rather than a successful simulation when a caller needs to + * know the message is really there. * @param node - Aztec node instance used to obtain the information about the message * @param l1ToL2MessageHash - Hash of the L1 to L2 message * @param chainTip - Chain tip to evaluate readiness against. Defaults to `'latest'`. Pass the tip the consuming PXE diff --git a/yarn-project/end-to-end/src/single-node/cross-chain/streaming_inbox_buckets.test.ts b/yarn-project/end-to-end/src/single-node/cross-chain/streaming_inbox_buckets.test.ts index 00e855241e3a..6ac7103ea4a6 100644 --- a/yarn-project/end-to-end/src/single-node/cross-chain/streaming_inbox_buckets.test.ts +++ b/yarn-project/end-to-end/src/single-node/cross-chain/streaming_inbox_buckets.test.ts @@ -23,11 +23,16 @@ jest.setTimeout(900_000); // Streaming Inbox coverage of what happens at Inbox bucket boundaries. Buckets are keyed by L1 block timestamp, // and Ethereum's strictly increasing timestamps close a bucket the moment its block is mined, so on a real chain a -// bucket is observed whole. anvil is the exception: two blocks can share a timestamp and therefore a bucket, which -// is the only way a test can make one bucket span two L2 blocks or grow a bucket after a checkpoint selected its end -// as the final endpoint. Both cases here mine such co-timestamped blocks by hand with L1 interval mining paused for -// well under an L2 slot (the sequencer stops building once the archiver's synced slot falls more than one slot behind -// the wall clock). +// bucket is observed whole. anvil is the exception: two blocks can share a timestamp and therefore a bucket, so a +// bucket can still grow after the node observed it. +// +// A bucket split across L2 blocks does not by itself need that exception: blocks end at arbitrary message prefixes, +// so an ordinary block can stop in the middle of a bucket it observed whole, whether because the per-block cap cut +// it short or because the next block picked up the rest. What the co-timestamped blocks buy these two scenarios is a +// bucket that is still open across the split, so the second half arrives after the first L2 block was built, and a +// bucket that grows past an endpoint a checkpoint had already selected as its final position. Both scenarios mine +// such co-timestamped blocks by hand with L1 interval mining paused for well under an L2 slot (the sequencer stops +// building once the archiver's synced slot falls more than one slot behind the wall clock). // // Same environment as streaming_inbox.test.ts (36s slots / 6s blocks, message-only blocks allowed), plus a prover // node so the checkpoint containing a split bucket has to be accepted by an actual prover, not just marked proven. diff --git a/yarn-project/kv-store/src/interfaces/common.ts b/yarn-project/kv-store/src/interfaces/common.ts index 45584336b0ad..ff4264dd6c03 100644 --- a/yarn-project/kv-store/src/interfaces/common.ts +++ b/yarn-project/kv-store/src/interfaces/common.ts @@ -5,9 +5,9 @@ export type Value = NonNullable; /** A range of keys of arbitrary type. */ export type CustomRange = { - /** The key of the first item to include */ + /** The key to start iterating at, inclusive */ start?: K; - /** The key of the last item to include */ + /** The key to stop iterating at, exclusive: an entry with exactly this key is not returned */ end?: K; /** Whether to iterate in reverse */ reverse?: boolean; diff --git a/yarn-project/sequencer-client/src/sequencer/automine/automine_sequencer.ts b/yarn-project/sequencer-client/src/sequencer/automine/automine_sequencer.ts index fa8d1e3a9e55..65db37ad5695 100644 --- a/yarn-project/sequencer-client/src/sequencer/automine/automine_sequencer.ts +++ b/yarn-project/sequencer-client/src/sequencer/automine/automine_sequencer.ts @@ -479,7 +479,8 @@ export class AutomineSequencer { // Streaming Inbox: automine builds a single-block checkpoint, so its one block is the checkpoint's final block // and has to land on a live bucket end: bound the consumed total by what the archiver holds and the caps, // resolve the live L1 bucket end at or below it with one Inbox call, and authenticate the range to it against - // the local log. The parent total is the fork's L1-to-L2 leaf count (compact indexing). + // the local log. Landing on that boundary can leave observed messages behind, so the block may consume fewer + // than the caps allow. The parent total is the fork's L1-to-L2 leaf count (compact indexing). const parentTotalMsgCount = (await fork.getTreeInfo(MerkleTreeId.L1_TO_L2_MESSAGE_TREE)).size; const cursor = await this.deps.l1ToL2MessageSource.getMessagePosition(parentTotalMsgCount); if (cursor === undefined) { diff --git a/yarn-project/sequencer-client/src/sequencer/inbox_message_selection.test.ts b/yarn-project/sequencer-client/src/sequencer/inbox_message_selection.test.ts index 844c8b01c4c5..cc8774bebfe2 100644 --- a/yarn-project/sequencer-client/src/sequencer/inbox_message_selection.test.ts +++ b/yarn-project/sequencer-client/src/sequencer/inbox_message_selection.test.ts @@ -117,6 +117,43 @@ describe('resolveEndpoint', () => { }); }); +// The local-only step is what a node reuses to guess the next block's bundle when simulating public calls. It is an +// estimate in both directions: a final block lands on a live bucket boundary, which can be behind that step. +describe('local selection against the final block that has to land on a bucket boundary', () => { + const caps = PROTOCOL_INBOX_CONSUMPTION_CAPS; + + it('ends a final block below the local step when the last boundary in reach is behind it', async () => { + const messageSource = mock(); + const inbox = mock(); + const streamingInbox = mockStreamingInbox(messageSource, inbox); + // Cursor at the checkpoint start, 400 messages observed, live buckets ending at 200 and 400. + streamingInbox.set( + Array.from({ length: 400 }, (_, i) => new Fr(i + 1)), + [200n, 400n], + ); + const selection = { cursorCount: 0n, localSyncedCount: 400n, checkpointStartCount: 0n, caps }; + + const localEnd = selectSafeLocalEnd(selection); + const upperBound = getEndpointUpperBound({ ...selection, isFinalBlock: true }); + const resolved = await resolveEndpoint({ + inbox, + messageSource, + cursor: streamingInbox.positionAt(0n), + upperBound, + }); + + // The local step takes a full block; the final block stops at the boundary below it. + expect(localEnd).toEqual(256n); + expect(upperBound).toEqual(256n); + const finalBlockEnd = resolved.ok ? resolved.endpoint.totalMessageCount : undefined; + expect(finalBlockEnd).toEqual(200n); + // A public call using message index 220 runs against a message the local step covers and the block never inserts. + const messageIndex = 220n; + expect(messageIndex < localEnd).toBe(true); + expect(messageIndex >= finalBlockEnd!).toBe(true); + }); +}); + describe('ordinary message selection', () => { const caps = PROTOCOL_INBOX_CONSUMPTION_CAPS; // 1024 - 256: the last position from which one block always reaches the end of the bucket the cursor sits in. diff --git a/yarn-project/sequencer-client/src/sequencer/inbox_message_selection.ts b/yarn-project/sequencer-client/src/sequencer/inbox_message_selection.ts index 5bf975782f4b..8f9ff4cc160c 100644 --- a/yarn-project/sequencer-client/src/sequencer/inbox_message_selection.ts +++ b/yarn-project/sequencer-client/src/sequencer/inbox_message_selection.ts @@ -86,8 +86,13 @@ export function selectOrdinaryMessageEnd(input: { /** * The furthest a block may advance on the local log alone without risking the checkpoint's last legal endpoint: the - * greedy end held down to the threshold. A block whose endpoint lookup fails or resolves short of this still takes - * it, since ending at or below the threshold always leaves one bucket of checkpoint capacity in reserve. + * greedy end held down to the threshold. A non-final block whose endpoint lookup fails or resolves short of this + * still takes it, since ending at or below the threshold always leaves one bucket of checkpoint capacity in reserve. + * + * Reused outside block building to guess what the next block will consume, this is an estimate and not a bound in + * either direction. A checkpoint's final block has to land on a live L1 bucket boundary, which can be below this: + * from a cursor of 0, with 400 messages observed and live buckets ending at 200 and 400, this returns 256 while the + * final block ends at 200. */ export function selectSafeLocalEnd(input: { cursorCount: bigint; diff --git a/yarn-project/stdlib/src/messaging/l1_to_l2_message_source.ts b/yarn-project/stdlib/src/messaging/l1_to_l2_message_source.ts index 117126c960cb..c45c77bfdacb 100644 --- a/yarn-project/stdlib/src/messaging/l1_to_l2_message_source.ts +++ b/yarn-project/stdlib/src/messaging/l1_to_l2_message_source.ts @@ -85,6 +85,11 @@ export interface L1ToL2MessageSource { * valid and returns equal positions. The range contract is that of `getL1ToL2MessagesBetweenLeafCounts`: an * invalid range, or one the source cannot serve whole (including its starting position), throws rather than * returning a partial or empty result. + * + * An in-process archiver throws its own typed error for a range it has not synced, but that class does not survive + * a JSON-RPC hop: a remote source raises a generic `Error` carrying only the message. Callers therefore have to + * treat any throw from this method as "not available from this source right now" and must not branch on the error's + * class or on `instanceof`. * @param startLeafCount - The cumulative Inbox message count the range starts at, inclusive. * @param endLeafCount - The cumulative Inbox message count the range ends at, exclusive. */ diff --git a/yarn-project/validator-client/src/proposal_handler.ts b/yarn-project/validator-client/src/proposal_handler.ts index 7ae3f2c25999..718e48c9e668 100644 --- a/yarn-project/validator-client/src/proposal_handler.ts +++ b/yarn-project/validator-client/src/proposal_handler.ts @@ -1307,11 +1307,6 @@ export class ProposalHandler { return { accepted: true, messages }; } - /** - * Runs {@link readCheckpointConsumedMessages}, waiting out a local sync lag the way the per-block checks do: an - * unavailable or mismatching prefix forces an archiver sync and re-reads until it resolves or the attestation - * deadline passes. Neither outcome is proposer misconduct, so a timeout keeps the nonpunitive reason. - */ /** * Reads the blocks of a slot as one snapshot for checkpoint validation, locating the checkpoint's last block by the * signed archive. Undefined when the archive is not among the slot's blocks or the blocks do not chain onto each @@ -1334,6 +1329,11 @@ export class ProposalHandler { return contiguous ? { blocks, lastBlockIndex } : undefined; } + /** + * Runs {@link readCheckpointConsumedMessages}, waiting out a local sync lag the way the per-block checks do: an + * unavailable or mismatching prefix forces an archiver sync and re-reads until it resolves or the attestation + * deadline passes. Neither outcome is proposer misconduct, so a timeout keeps the nonpunitive reason. + */ private async awaitCheckpointConsumedMessages( slot: SlotNumber, checkpointStartTotal: bigint,