Skip to content

perf(beacon): precompute the epoch transition off the import path - #652

Draft
MegaRedHand wants to merge 2 commits into
beacon-chain-integrationfrom
perf/beacon-epoch-precompute
Draft

MegaRedHand wants to merge 2 commits into
beacon-chain-integrationfrom
perf/beacon-epoch-precompute

Conversation

@MegaRedHand

Copy link
Copy Markdown
Collaborator

Motivation

The first block of every epoch runs process_epoch inline on the chain actor (fork_choice::on_block, then stf::state_transition, then process_slots), followed by the post-epoch rehash the state-root check needs. In the import replay such blocks take ~1.5 s against ~0.58 s for an ordinary block (measured on the replay).

The transition depends only on the parent, which is known by the end of the previous epoch's last slot, so it can run ahead of time on a blocking worker, off the import path.

Design

Part Behaviour
Trigger: head An import leaves a block at the last slot of epoch E as head: precompute (E+1, head_root). Covers a late last-slot block.
Trigger: timer Three quarters into the last slot of E, derived from the configured slot duration. The once-per-slot tick arms a delayed self-message on last slots only (send_after). Starts a worker only if none is running and the key is not cached. Covers a skipped last slot, where the head is an earlier block.
Gate Beacon only. Skipped while the sync tracker says syncing, and for a head more than one slot behind the wall clock (the tracker updates once per slot, so a follower replaying history would otherwise precompute every epoch).
Work spawn_blocking: clone the head state, process_slots to the first slot of E+1, flush pending writes, hash_tree_root so tree-node hashes are memoized, then message the state back. At most one worker in flight; deduped by (epoch, root) through the cache.
Store CacheKey::CheckpointState { epoch: E+1, root: head_root }. checkpoint_state derives exactly that value for the checkpoint (the checkpoint block's post-state advanced to the epoch's first slot), so attestation targets for E+1 hit it too. A unit test asserts equal hash_tree_root.
Consume on_block: when the parent's slot is before the block epoch's first slot and the block is at or after it, look up CheckpointState { block_epoch, parent_root }. A hit clones the entry, advances any remaining skipped slots, and applies the block with the new stf::apply_block (the part of state_transition after process_slots; process_slots rejects a target equal to the current slot, so a boundary block cannot go through state_transition).
Miss / race Today's path, unchanged. A block that arrives while the worker runs imports inline; the late result is stored and unused by that block.

Forks: process_slots performs the fork upgrade at the boundary, so nothing here is fork-specific.

New metrics (documented in docs/metrics.md): lean_beacon_epoch_precompute_lookups_total{result} (epoch-crossing imports only), lean_beacon_epoch_precompute_seconds, lean_beacon_epoch_precompute_started_total{trigger}.

Not in this PR

  • Progressive balances (O(1) justification, and a cheaper compute_pulled_up_tip).
  • Holding the block when it races the worker (it computes inline instead, so no ordering dependency is added).
  • Advancing the head every slot; only epoch boundaries are advanced.

Testing

  • ethlambda-storage lib, ethlambda-state-transition lib, ethlambda-blockchain lib, beacon spec suite (mainnet), clippy and cargo check -p ethlambda: see counts below.
  • New unit tests: resuming from a precomputed state gives the inline post-state (block on the boundary and with 1 and 3 skipped slots, compared with the block's committed state root); the lookup applies only to epoch-crossing imports; the worker's state equals checkpoint_state's for the same key; trigger decisions as a pure function (head, timer, syncing, stale head, in-flight, already cached) and the three-quarter delay.
  • There is no actor-level test: the beacon actor's test harness drives handlers without a Context, so the trigger decision is tested as a pure function instead.
  • Not measured: no benchmark was run for this PR.

Results: storage lib 138 passed (1 ignored); state-transition lib 327 passed (16 ignored); blockchain lib 217 passed; beacon spec suite (mainnet) 5734 passed, 4 failed (the 4 expected gossip matched_fixture_cases on the empty gossip dir), 152 ignored; clippy (workspace, and state-transition with beacon-spec-tests) clean; cargo check -p ethlambda clean.

The first block of every epoch ran process_epoch inline on the chain actor,
then paid the post-epoch rehash for its state-root check. The transition
depends only on the parent, which is known before the block arrives.

A blocking worker now clones the head state, advances it to the first slot of
the next epoch, flushes and hashes it, and the actor stores it under
CheckpointState { epoch, root }, the key fork_choice::checkpoint_state already
derives for that checkpoint. Triggers: an import that makes a last-slot block
head, and three quarters into a last slot (covers a skipped last slot). Both
are skipped while syncing; at most one worker runs.

fork_choice::on_block resumes from the cached state for an epoch-crossing
block and applies the block with the new stf::apply_block. A miss, including
a block that races the worker, takes the previous path unchanged.

Adds lean_beacon_epoch_precompute_{lookups_total,seconds,started_total}.
@MegaRedHand MegaRedHand added the beacon Ethereum Beacon Chain client label Oct 2, 2026
MegaRedHand added a commit that referenced this pull request Oct 2, 2026
…636-638-gloas-live

Conflicted files:
- crates/blockchain/src/lib.rs
- crates/blockchain/state_transition/src/beacon/fork_choice.rs
- crates/blockchain/state_transition/src/metrics.rs
- docs/metrics.md

Gloas: the precompute stays enabled, because the base state cannot differ.
In this branch a gloas block's post-state never holds its own payload: the
child applies it (process_parent_execution_payload) inside process_block,
after process_slots, and on_execution_payload_envelope only records and
checks, it stores no second state. The store keeps one state per root and
both on_block (get_state(parent_root)) and checkpoint_state read it, and the
worker clones get_state(head_root) and caches the result under
CheckpointState { E+1, head_root }, so the entry is process_slots of the very
state an import of a child of that root clones. The fulu-to-gloas boundary
needs no case either: the upgrade runs inside process_slots on both paths.
A new unit test (a_gloas_precompute_starts_from_the_state_an_import_would)
checks, for a gloas parent, that the worker's result equals
checkpoint_state's for the key, that an import finds it, and that advancing
the rest of a skipped first slot from it equals plain process_slots.

Other adaptations:
- transition_block and stf::apply_block take the ActiveBalanceCache that
  #648 added to state_transition; on_block passes the store's own cache.
- on_block's gloas-specific code between the parent read and the
  transition does not use the cloned state, so removing that early clone
  (as #652 does) needed nothing else.
- lib.rs: the NewBlock handler keeps the envelope settling that follows the
  import and triggers the precompute after it, so a head moved by a settled
  envelope's block is seen.
- metrics.rs and docs/metrics.md: both sides' metrics kept.
- Test fixtures of #652 follow this branch's BlockInputs (operations) and
  the extra argument.
- docs/beacon_stf.md notes why gloas shares the key.
MegaRedHand added a commit that referenced this pull request Oct 5, 2026
@MegaRedHand MegaRedHand mentioned this pull request Oct 5, 2026
4 tasks
MegaRedHand added a commit that referenced this pull request Oct 5, 2026
…36-638-gloas-live

Brings gloas validator duties (produceBlockV4, envelope publication, PTC
duties and payload attestations, gloas attestation data and aggregates, VC
gloas support) onto the deployment branch, keeping every behavior of #626,
#633, #636, #638, #646, #647-#652, #656, #658-#660 and the sync-committee and
liveness endpoints.

Conflict resolutions keep both sides: the attestation pool stays in Store
(tmp) while the payload attestation pool is threaded through P2P and the RPC
handles (feature); the aggregate endpoints keep tmp's liveness recording and
attesting indices and add the feature's fork-header check and gloas pooling;
the VC tests and fake execution client serve both fulu blobs and gloas V6.

Semantic fixes:
 a. POST /eth/v2/beacon/blocks (gloas) calls publish_beacon_block(block,
    Vec::new()): gloas columns travel with the envelope. RecordingNetwork
    implements publish_beacon_block(block, sidecars) and both new methods.
 b. produceBlockV4 appends client versions to the graffiti exactly like
    produceBlockV3 (graffiti::execution_client_version run alongside the
    payload build, with_client_versions, Extension<OwnVersion>) and logs it.
 c. Attestation data, aggregate_attestation keep require_execution_client and
    require_validated for gloas slots; payload_attestation_data now applies
    the same two rules (503 without an execution client, or when the voted
    block's payload is unvalidated).
 d. Proposer duties v1 and v2 serve gloas epochs from a fulu or gloas state's
    proposer_lookahead; nothing refuses gloas any more; v2 keeps its
    dependent root.
 e. The VC's per-validator ProposerSettings apply to gloas proposals (graffiti
    in the BlockRequest, fee recipient compared with the bid's); a test pins
    the graffiti. VC tests updated to the ProposerSettings constructors.
 f. gloas production reads the attestation pool from Store and calls the
    stf with the ActiveBalanceCache the perf work added; fulu production and
    pack_operations are untouched (gloas blocks carry no pooled operations).
 g. Chain events are emitted by the chain actor only, so nothing on the RPC
    publish paths needed to move; gloas imports reach it unchanged.
 h. Cargo.lock unchanged; cargo check --locked passes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

beacon Ethereum Beacon Chain client

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant