You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 17af7c3
Browse filesBrowse the repository at this point in the historyBrowse files
perf(storage): borrow values on read, make has_state skip snapshot reads, keep states in blob files (#658)
## Motivation
On the Hoodi follower, the import guard's `has_state(parent)` check
costs 0-1 ms on most blocks but **p50 180 ms, p90 332 ms, max 536 ms**
on the block after each epoch's first block (48h of import timings,
10-03 to 10-05):
| slot % 32 | `guards` p50 | `guards` p90 |
|---|---|---|
| 0 | 0 ms | 1 ms |
| **1** | **180 ms** | **332 ms** |
| 2 | 0 ms | 177 ms (when slot 1 is empty) |
| 3..31 | 0 ms | 1-8 ms |
An epoch-crossing block's state is stored as a full snapshot only, and
`has_state` did `get(States)` on it: RocksDB read and copied the whole
~150 MB state to answer a yes/no question, although that state was still
in the store's state cache. On Plataberget the same function costs ~30
ms on every block, since `States` has no bloom filter and a miss still
reads a snapshot-sized data block per L0 file.
## Changes
1. **`has_state` is an existence check.** It consults the state cache
(`peek`, so the LRU order is untouched), then `StateDiffs` before
`States`, through `contains`, and never copies a value. A cached
`BlockState` always means the state is pending or persisted, and nothing
deletes a persisted state, so the answer is unchanged.
2. **`StorageReadView::read` is the one required lookup.** It lends the
value to a `&mut dyn FnMut(&[u8])` callback, which keeps the trait
usable as `dyn StorageReadView`. `get` and `contains` are default
methods on top of it, so both backends implement `read` only. RocksDB
uses `get_pinned_cf`.
3. **Decodes read from the backend's buffer.**
`StorageReadViewExt::read_with` decodes inside the borrow, and every
get-then-decode site uses it. Beacon state reconstruction folds its
deltas while the snapshot is still borrowed and hands the result over as
a `Cow`: a snapshot root decodes straight from RocksDB's buffer, and the
writer's parent-bytes path takes ownership without a second copy.
`get(..).is_some()` checks became `contains`. `get` stays where the
bytes outlive the read (the beacon walk's delta records, the
pending-column take) and for the stored config, which must not be
decoded before the version and preset checks.
4. **`States` and `StateDiffs` use RocksDB blob files.**
| setting | value | why |
|---|---|---|
| `min_blob_size` | 4 KiB | one default data block; anything larger gets
an oversized block anyway |
| blob compression | LZ4 | blob files default to none, while SST blocks
get Snappy; the values are raw SSZ |
| blob GC | off | nothing deletes or overwrites a state, so GC would
only relocate live blobs |
| blob cache | none | the store caches decoded states, and a
snapshot-sized entry would evict the whole block cache |
## Compatibility
No `DB_VERSION` bump. RocksDB applies the blob options to an existing
directory as it goes: new writes land in blob files, and inline values
move out as compaction rewrites their SSTs.
`a_directory_written_without_blob_files_still_reads` opens a directory
written with plain options and reads both old inline and new blob
values. `estimate_table_bytes` now adds `rocksdb.live-blob-file-size`,
which `estimate-live-data-size` leaves out, so the table-size metric
keeps counting state bytes.
## Testing
- New tests: `has_state` on cached, diff, snapshot-only and absent
roots, through a counting backend that proves the cached path reads
neither table; `read` on both backends (value, absent key, callback
error); a cold-cache beacon reconstruction across a whole snapshot
interval; blob placement and the pre-blob directory.
- `cargo test --workspace --profile release-fast --lib --bins`: 0
failures. Clippy `-D warnings` clean.
- Spec-test suites not run locally.
## Not covered
- A bloom filter on `States`: blob files make misses cheap here, but
other large-value tables still have none.
- The live effect on `guards`: worth re-measuring on a follower after
deploy.
0 commit comments