Skip to content

fix(beacon): verify a block's signature before it waits for its parent or columns - #629

Draft
MegaRedHand wants to merge 4 commits into
beacon-chain-integrationfrom
fix/beacon-verify-block-signature-before-waiting
Draft

MegaRedHand wants to merge 4 commits into
beacon-chain-integrationfrom
fix/beacon-verify-block-signature-before-waiting

Conversation

@MegaRedHand

Copy link
Copy Markdown
Collaborator

A beacon block that waits, for its parent's post-state and then for its custody columns, is kept unjudged until what it waits for arrives. A held one also has its columns asked for again every slot until finality evicts it. Mainnet gossip carries altered copies of real blocks (a blob transaction re-encoded after signing, the original signature kept): nobody signed their root, and no peer has columns for them.

lambdaclass/ethlambda_private#38 moved the signature check for a parentless block into gossip, but that check has two gaps:

  • It reads the head state only through Store::cached_state, and queues the block unchecked when it finds none. After a resume nothing caches the head state before the first import.
  • Range sync and by-root fetches never pass through gossip validation at all.

What it cost

eth-5, keep-DB restart, 2026-09-25 19:41Z. Slot 15295105 had two gossip blocks from proposer 2128692, 2 ms apart: the altered copy 9b320e3a (917 KB, first) and the real b4a78feb (367 KB). The parent was held for columns, so gossip returned Queue(ParentNotReady), and with no head state cached yet the altered copy went to the chain actor unchecked. It was held for columns and logged Data column fetch failed after max retries 13 times, once per slot.

In the first ~45 s after that restart Prometheus shows 3 queue/parent_not_ready, 0 accept, and no reject/bad_signature until the first import.

Change

Every path a beacon block takes into waiting now requires a verified proposer signature first.

gossip beacon_block, parent has no post-state
  │
  ├─ head state cached?  no ─────────────────────▶ Ignore(signature_unverified)   [was: Queue]
  └─ precheck_block(Recent(head))
       ├─ Ok ────────────────────────────────────▶ Queue ─▶ chain actor
       ├─ BadSignature ──────────────────────────▶ Reject
       └─ UnknownProposer ───────────────────────▶ Ignore(signature_unverified)   [was: Queue]

chain actor (gossip, range sync, by-root)
  ├─ parent has no post-state ─▶ precheck_block(Recent(head))   ─▶ Err: refuse, drop parked subtree
  └─ columns missing          ─▶ precheck_block(Parent(parent)) ─▶ Err: refuse
Where Before After
gossip::block::queue_if_signed (was queue_unless_forged) Queue unless the signature is shown forged Queue only once it verifies; otherwise Ignore(SignatureUnverified) (new IgnoreReason)
precheck_block, Reference::Recent, proposer not in state Ok(()) Err(UnknownProposer), so Ok always means the signature verified
Validated::forward a block is forwarded on every outcome forwarded only on Accept / Queue, so an Overloaded block is dropped too
Chain actor, before parking for a parent no check precheck_before_waiting(Wait::Parent): signature against the head's post-state; on failure also discard_pending_subtree
Chain actor, before holding for columns no check precheck_before_waiting(Wait::Columns): every precheck rule against the parent's post-state; returns ImportError::Precheck
Resume (fetch_initial_beacon_state, both resuming returns) head state not cached Store::cache_head_state()

A fresh checkpoint sync already caches its anchor through insert_state, so the resume path was the only one missing it.

New metric: lean_beacon_blocks_refused_before_waiting_total{reason, wait}, where reason is the PrecheckError label and wait is parent or columns.

Docs (separate commit): beacon_wire.md and metrics.md no longer say a block is forwarded to the chain actor on every outcome or when Overloaded. They also now list the signature_unverified ignore reason and the new counter.

Trade-offs

  • A real block can now be dropped from gossip: when no head state is cached, when its proposer is newer than the head state, or when the validation pool is overloaded. It still arrives through a child's by-root fetch or range sync once its parent has imported. The code comment on spawn_stateful_checks notes that no block was measured Overloaded on the mainnet followers over the 24 hours before this change.
  • One BLS verification per waiting block in the chain actor, on top of the one in its eventual state transition. This applies only to blocks that wait, not to blocks imported straight away.

Tests

Check Result
New: an_unknown_parent_by_a_proposer_the_head_state_lacks_is_dropped passes
New: only_an_accepted_or_queued_block_is_forwarded passes
New: a_resumed_store_caches_its_head_state_on_request passes
Updated: a_block_whose_parent_was_never_seen_is_dropped_with_no_state_to_check_it (was ..._is_queued) passes
Updated: a_proposer_missing_from_the_reference_state_is_unknown_to_both_references passes
--lib: blockchain 140 passed, 2 failed (see below)
--lib --bins: state-transition, p2p, storage, ethlambda pass
make lint, cargo fmt --check clean

Not yet run on a follower.

Known failures (why this is a draft)

Test Failure Cause
releasing_once_every_custody_column_arrives_clears_the_hold panic at precheck_before_waiting: "the reference block has a post-state" The release goes through on_block, and the fixture's parent (ZERO) has no state, so the parent-wait precheck runs. The head (ZERO) has no state either, so the expect panics.
a_childs_fan_out_does_not_livelock_a_held_parent blocks_awaiting_columns.contains_key(&parent_root) is false fulu_block is unsigned and bare_state() has no validators, so the columns-wait precheck refuses the block with UnknownProposer instead of holding it.

Neither test is wrong: they hold unsigned blocks, and holding now requires a valid signature. The signing helpers (secret_key_for, with_validators_at) are #[cfg(test)] pub(crate) in ethlambda-state-transition, so the blockchain crate cannot reach them. Options:

  • A. Expose test_state behind a test-utils feature and give these tests keyed states and signed blocks.
  • B. Add a test-only switch that skips precheck_before_waiting. This is smaller, but these tests would then stop covering the precheck path.
  • C. Separately: decide whether a missing reference state should refuse the block instead of panicking.

Moved

  • Moved from lambdaclass/ethlambda_private#58, now that beacon-chain-integration lives on this repo.
  • Based on beacon-chain-integration @ c79fabd5, merged into the branch. One conflict, in verdict.rs tests: both sides added a test at the same spot, so both are kept.

…t or columns

A beacon block that waits, for its parent's post-state and then for its
custody columns, is kept unjudged until what it waits for arrives, and a
held one has its columns asked for again every slot until finality evicts
it. Mainnet gossip carries altered copies of real blocks (a blob
transaction re-encoded after signing, the original signature kept): nobody
signed their root and no peer has columns for them.

Gossip's queue path checked the signature only against a cached head
state, and queued the block anyway when it found none. After a resume
nothing caches the head state before the first import, so in that window
every parentless block reached the chain actor unchecked, and an altered
one sat held for columns until finality. Range sync and by-root fetches
never pass through gossip validation at all.

- Gossip queues a parentless block only once the head state verifies its
  signature. One it cannot judge is ignored as `signature_unverified`, and
  only an accepted or queued block is forwarded, so an overloaded one no
  longer reaches the actor unjudged.
- `precheck_block` against a recent state reports an unknown proposer
  instead of passing it, so `Ok` always means the signature verified.
- The chain actor runs `precheck_block` before parking a block for its
  parent (signature, against the head state) and before holding it for
  columns (every rule, against the parent's post-state).
- Resuming from the DB caches the head's state, so gossip can judge a
  parentless block from the first message on.
The previous commit stopped forwarding a block gossip could not verify the
signature of, `Overloaded` included, so the "still forwarded" and "either
way" passages no longer hold. It also added an ignore reason and a counter
nothing documented yet.
…ix/beacon-verify-block-signature-before-waiting

# Conflicts:
#	crates/net/p2p/src/beacon/verdict.rs
@MegaRedHand MegaRedHand added the beacon Ethereum Beacon Chain client label Oct 1, 2026
@MegaRedHand MegaRedHand mentioned this pull request Oct 5, 2026
4 tasks

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

beacon Ethereum Beacon Chain client

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant