fix: audit follow-up — DB resilience, read-path perf, ingest/clustering safety, ETH failed-tx fee correction#117
Open
soad003 wants to merge 115 commits into
Open
fix: audit follow-up — DB resilience, read-path perf, ingest/clustering safety, ETH failed-tx fee correction#117soad003 wants to merge 115 commits into
soad003 wants to merge 115 commits into
Conversation
… clustering One-off path (run_clustering_spark): bulk token-range reads of raw.transaction and address_ids_by_address_prefix, zero-copy Arrow feed into the Rust union-find (process_transactions_arrow), connector bulk-write of fresh_address_cluster / fresh_cluster_addresses, then a fresh_cluster_stats (size + min_address_id) aggregate. Incremental delta clustering reads only address->cluster and stats for touched addresses, classifies each component new/join/merge, and reads full membership only for the absorbed side of a merge; per-partition fetch_size-paged reads (read_partitions_concurrent) replace the multi-partition IN that could ReadTimeout on large clusters. Adds fresh_cluster_stats table + transformed_utxo v1->v2 migration.
…read Harvest each batch's multi-input address-id sets from the txs the delta loop already holds and cluster per batch before persist, instead of re-reading the whole block range and re-resolving addresses after every run. last_synced_block then also marks clustering progress: a crash redoes the batch and re-clusters idempotently. run_fresh_clustering stays as the stand-alone range re-cluster for backfill/recovery and the regression tests. Adds Rust unit tests (arrow null/empty/accumulate, Clustering struct path), Python harvest-helper unit tests, and a one-off vs incremental partition-equivalence test across batch sizes.
… parity Seed a small hand-crafted raw UTXO dataset into Cassandra and assert the PySpark one-off (run_clustering_spark) and the incremental run_fresh_clustering produce the same address partition across block batches, reusing compare_cluster_partitions. Lives in the regression suite (needs live Cassandra + Spark); production code runs via current_venv subprocesses.
Cap the one-off clustering to transactions with block_id <= end_block so the chain can be clustered as of a given height. No start bound: clustering is transitive over the full history, so a sub-range start is meaningless. Threads end_block through multi_input_address_id_sets and run_clustering_spark and exposes it as the transformation cluster --end-block CLI option. Adds a manufactured-dataset regression asserting a capped one-off equals incremental over the same [0, end_block] range.
Add the legacy cluster table's stat columns (no_incoming/outgoing_txs, in/out_degree, first/last_tx_id, total_received/spent and their _adj variants) to fresh_cluster_stats via migration 2_to_3, so the table can carry the per-cluster stats the weekly recompute will populate. Columns are nullable; nothing writes them yet. size + min_address_id stay the live tier.
Aggregate the address-level tables (address + address_incoming/outgoing_ relations) through the fresh address->cluster membership into the full per- cluster stats: size, min_address_id, raw total_received/spent (element-wise currency sums), first/last_tx_id, and relation-derived in/out_degree and no_incoming/outgoing_txs (distinct neighbour clusters / summed tx-counts over external edges, self-edges dropped). Pure DataFrame transforms with synthetic- frame unit tests; recompute_fresh_cluster_stats is the Cassandra I/O wrapper. New CLI 'transformation recompute-cluster-stats' runs it under the transformed- keyspace lock (same lock the delta updater holds) and truncate-rebuilds for a self-healing weekly job. Membership is untouched; total_*_adj left unset (no delta-updater reference definition exists).
Cassandra has no ADD IF NOT EXISTS for columns, so re-applying an ALTER TABLE ... ADD raises a column-conflict rather than 'already exists'. Treat both as already-applied so a partially-applied migration can be re-run.
address_cluster_mapping_v2 plus best_cluster_tag_v2 / tag_count_by_cluster_v2 alongside the existing objects for the clustering cutover. Additive and idempotent (IF NOT EXISTS, unique indexes for REFRESH CONCURRENTLY); legacy views untouched.
…luster totals - one-off relabels every cluster to min(address_id); incremental elects the smallest-root survivor, so cluster_id == root == min(address_id) end-to-end - total_received_adj/total_spent_adj = summed external inter-cluster estimated_value (cluster totals minus intra-cluster flow) - fresh_cluster_stats.size counts every member via a left join - input-id resolution drops unresolvable (multisig) addresses instead of raising
- qualify CREATE TABLE/TYPE incl. IF NOT EXISTS; provision pyspark in the venv; strip JDWP/SPARK_HOME and pin worker python for the Spark subprocess; pass the setuptools_scm version to the docker build - split batches so a real two-cluster merge runs, and assert cluster_id == min(members) on both the one-off and incremental paths
The recompute is just the Spark job that rebuilds fresh_cluster_stats; the cadence is an infra/scheduling concern, not a property of the code.
GRAPHSENSE_TAGSTORE_FRESH_CLUSTERS routes the cluster-tag read path to the fresh-clustering address_cluster_mapping_v2 / *_v2 materialized views via a per-call resolver in queries.py. Independent of the write-side GRAPHSENSE_FRESH_CLUSTERING_ENABLED so v2 can be populated and validated while reads stay on the legacy relations; defaults off, instant rollback by unsetting.
…o_addresses Give the fresh tables the same id_group partitioning as the legacy address / cluster tables: fresh_address_cluster keyed (address_id_group, address_id), fresh_cluster_addresses (cluster_id_group, cluster_id, address_id), fresh_cluster_stats (cluster_id_group, cluster_id). Keeps partition count and Spark-scan chunks bounded at billion-row scale instead of one partition per id. Also renames fresh_cluster_stats.size -> no_addresses to match the legacy cluster column the tagstore mapping feeder reads. bucket_size (= configuration.bucket_size) is threaded into the one-off Spark writers, the delta-loop writes, and the fresh point-read helper, which gained optional composite-key (group_column) support. Manufactured parity regression uses a tiny bucket so the dense ids span several distinct buckets.
run_clustering_spark PHASE 4 now runs recompute_fresh_cluster_stats instead of the size+root-only backfill, so a one-off bootstrap leaves fresh_cluster_stats complete (totals, first/last tx, degrees, tx-counts, adjusted totals), not just no_addresses + min_address_id. Drop the now dead backfill_fresh_cluster_stats.
# Conflicts: # CHANGELOG.md # src/graphsenselib/deltaupdate/deltaupdater.py # tests/regressions/tests/clustering/ingest_runner.py
…GSTORE_FRESH_CLUSTERS The env var that already flips tagstore reads now also flips the feeder: Cassandra read source (fresh_address_cluster/fresh_cluster_stats), Postgres write target (address_cluster_mapping_v2), and the *_v2 MV refresh + staleness sample. Exclusive switch; first v2 fill needs a full --rerun.
…RESH_CLUSTERING_ENABLED Entity stats now read fresh_cluster_stats and cluster membership reads fresh_cluster_addresses when the flag is on (UTXO only), matching the address-level fresh_cluster_id path. Also fixes get_fresh_cluster_id to include the address_id_group partition key (was an unservable query).
Fresh clustering persists only multi-member clusters, so an address absent from fresh_address_cluster is its own singleton (cluster_id==address_id, 1 member). The UTXO feeder left-joined fresh_address_cluster and inserted the resulting NULL gs_cluster_id, violating NOT NULL. Now coalesces missing cluster_id to address_id (mirrors the account-model branch); no-op for the legacy address.cluster_id path. Verified end-to-end: acm_v2 maps both multi-member (cluster 33) and singleton (cluster 1) LTC addresses.
Fresh clustering persists only multi-member clusters, so singleton ids have no fresh_cluster_stats row. Under GRAPHSENSE_FRESH_CLUSTERING_ENABLED: - get_address_entity_id resolves the fresh cluster id (falls back to address_id) instead of the stale legacy address.cluster_id - get_entity synthesizes a one-address entity for singletons - list_entities restores singletons dropped from the bulk lookup, preserving input order and dropping only genuinely unknown ids
# Conflicts: # CHANGELOG.md
…0_to_1 edit transformed_utxo_0_to_1 shipped in v2.13.0 creating fresh_address_cluster / fresh_cluster_addresses with a single-column partition key. The partition-bucket reshape had edited that released migration in place, which is a silent no-op on any keyspace already past schema_version 1 (version-gated + CREATE IF NOT EXISTS) - all prod UTXO keyspaces are at v1 with the old shape, so the new bucketed reads would fail there. Revert 0_to_1 to its released bytes and deliver the reshape as a new transformed_utxo_3_to_4 (DROP + CREATE, since Cassandra cannot ALTER a primary key). Safe: the tables hold only regenerable derived data and are empty on every keyspace that has not run fresh clustering. schema.sql keeps the final bucketed shape for greenfield; both v1 and v0 paths converge to v4.
Under GRAPHSENSE_TAGSTORE_FRESH_CLUSTERS the feeder reads membership from fresh_address_cluster, which stores only multi-member clusters. A batch of only singleton addresses resolved to an empty get_cluster_ids and hit the early return DataFrame(), dropping the whole batch - and singletons are the majority under fresh clustering. Empty membership now maps every address to its own singleton cluster (cluster_id == address_id) via _as_singleton_clusters; legacy behavior (empty means not-found) is unchanged when the switch is off.
- Remove is_fresh_clustering_empty (analytics): no callers after the delta updater stopped gating on a first-run emptiness check. - read_partitions_concurrent: drop the unreachable non-bucketed branch (all callers pass group_column) and turn the bucket_size=None foot-gun (TypeError at k // bucket_size) into an explicit ValueError.
…st crate The shipped incremental path reads only touched addresses and classifies new/join/merge in Python; the rebuild-full-mapping-then-diff API is unused. Removes both pymethods, the snapshot field, and their tests; rebuilt crate.
fresh_cluster_stats no_incoming_txs/no_outgoing_txs summed per-edge no_transactions over address relations, over-counting a multi-input tx once per member address. Count distinct tx_id per cluster from address_transactions instead (new cluster_tx_counts); degrees and adjusted totals stay edge-based.
Counting per-address is_outgoing split double-counted the change pattern (spend from member A, change to member B of the same cluster) in BOTH directions. Legacy computeClusterTransactions sums signed value per (cluster, tx) and assigns one direction from the net sign. Sum the signed address_transactions.value per (cluster, tx) and count by net sign; net==0 counts in neither. Subsumes the earlier distinct-count fix.
parallel_find_all collected a Vec<(u32,u32)> then split it with a single-threaded into_iter().unzip() — an ~8 GB transient plus a serial O(n) pass on the driver at BTC scale (~1e9 ids). Fill cluster_ids in place in parallel by index and generate address_ids from the dense range.
…s recompute cluster_relation_stats inner-joins the anchor endpoint (result-identical: the base left-join drops singleton-anchored rows downstream) to shrink the degree/ value aggregation shuffles. recompute persists members once — it feeds three joins under different partitionings that ReuseExchange cannot dedupe.
…d Delta duplicates after a mid-chunk crash
…ustering + clamp union-find ids to avoid Rust panic
…upply inflation (forward-only fee-only sender addresses)
A resolved address.cluster_id whose stats row is absent (dangling reference, e.g. a tip cluster whose stats write lagged, or a not-yet written fresh_cluster_stats row) left no_addresses NaN after the merge; the unconditional .astype(int) then raised IntCastingNaNError, aborting the whole multiprocess recalc on a single bad row. Keep the real cluster_id and store the unknown size as NULL, not a fabricated 1 (gs_cluster_no_addr = 1 is the singleton sentinel in the cluster-tag views, so 1 would misroute a real cluster's tags).
When a resolved cluster_id yields no no_addresses, log each row and classify the cause: no row in the cluster table (no_cluster_row) vs a row whose no_addresses column is NULL (row_present_null_no_addresses), plus a per-batch summary. Lets a full recalc reveal the mechanism and prevalence without crashing.
The structural fold seeds a cluster from its members' pre-batch address rows (the delta clusters before the batch's address writes commit), so it misses every touched member's current-batch activity — not just existing members growing between mutations, but also newly-clustered addresses' own-batch activity. That made the money/tx columns only weekly-fresh, leaning on recompute-cluster-stats. Mirror the legacy cluster table: fold every touched address's per-batch EntityDelta onto its fresh cluster (from_activity_delta, membership-neutral) so pre-batch fold + this-batch propagation == the from-scratch member sum. Columns are now always-fresh; the recompute is only an optional backstop. Singletons are skipped (their pre-join history folds structurally on join), and a merge's absorbed side is guarded against resurrection. Cost: one fresh_address_cluster + fresh_cluster_stats read per batch over all touched addresses (not just co-spends). Verified by a new test that grows address rows across batches and asserts delta == one-off on final sums (new cluster, activity-only batch, merge, join).
The test job ran the full 4-way Python matrix (~6.5 min) and the build job depended on it, serializing it in front of every publish and roughly doubling wall-clock (~6 -> ~11-13 min). Tests still run via run_tests.yaml on pushes and PRs; publishing no longer waits on them.
Build the gs_clustering wheel before COPYing ./src so Python-only changes no longer invalidate the (expensive) Rust compile, and add BuildKit cache mounts for the cargo registry/git and crate target dir so the arrow/pyo3 dependency tree is compiled once and reused. The wheel is copied out to /wheels since cache mounts aren't part of the layer. The publish workflow persists those mounts across CI runs via actions/cache + buildkit-cache-dance. Measured locally: Rust compile 53.9s cold -> 3.7s when only crate code changes; a Python-only change skips the Rust layer entirely.
fresh_cluster_stats is an absolute row folded from its own stored value (stored + this batch's activity), so committing it ahead of the batch's commit point let a crash-and-replay fold the same activity onto itself twice. The structural writes alone were replay-safe -- on a redo the members are already assigned and the join no-ops -- so this only became reachable once activity propagation landed. Append the clustering changes to self.changes instead of applying them inline, so they ride the same WAL record and the same commit as the address rows and the bookkeeping that advances last_synced_block. This is the ordering TX mode already used. _apply_clustering_inputs stays for the standalone backfill, which re-reads committed rows and may commit per chunk. Guard tests cover the staging, the activity-only batch (no co-spend, which no test previously exercised), the untouched batch, and the fresh rows and their deletes through the WAL codec -- the fresh tables enter the WAL here for the first time.
# Conflicts: # Makefile
Conflicts were in the one-off clustering path, which both branches hardened independently. transformation/cli.py, transformation/clustering.py: resolved to feature/clustering2. It replaces the Cassandra-fed one-off (run_clustering_one_off_from_cassandra) with the Spark/Arrow one (run_clustering_spark), already holds the transformed-keyspace lock this branch was adding, and brings the recompute-cluster-stats command. Ported this branch's out-of-bounds id guard onto the new Arrow feed: the union-find is sized from summary_statistics.no_addresses, which can lag the real highest address id, and Rust indexes its node array directly (verified: feeding id 999 into a size-11 union-find panics at unionfind.rs:39). The bound check is a vectorized max over the chunk, so the zero-copy fast path is unaffected; only a chunk that trips it pays a Python fallback. flatten() is used rather than .values so a sliced ListArray does not read the parent buffer and mis-fire.
Add the adversarial folds the suite had no coverage for: chain merge inside one batch, a member absorbed while it transacts, a singleton's pre-join history, a clustered member with no address row -- plus seeded fuzz asserting after every batch that each cluster equals the from-scratch member sum. Verified non-vacuous: double-folding activity, dropping it, or leaking no_addresses from the activity contribution each fail all nine.
The suite assumed both servers share a key namespace, which holds only for two throwaway containers. Pointing BASELINE_SERVER at a real deployment needs a key that its gateway accepts, so add CURRENT_AUTH / BASELINE_AUTH overrides; unset, behaviour is unchanged. The manual suite sent no Authorization header at all, which a gateway-fronted baseline rejects.
A deployment behind an API gateway sees injected headers the bare local server never gets. X-Consumer-Groups in particular drives tag obfuscation, so without it every private-tag field reads as a difference. Add CURRENT_HEADERS / BASELINE_HEADERS (JSON); unset, behaviour is unchanged.
…istutils shim pyspark 3.5 imports distutils.version at runtime (hit by every createDataFrame(pandas_df)); on Python >= 3.12 distutils only exists via setuptools' shim and the python:3.13-slim image ships no setuptools. Dropping the runtime setuptools dep (0dcece3) therefore broke Spark jobs in the Docker image with ModuleNotFoundError: No module named 'distutils'. Dev/testing venvs were unaffected (dependency groups still pull setuptools), which is why tests never caught it. Remove again once pyspark >= 4.
UTXO relations pair net senders with net receivers, but list_links filtered on raw io membership, so an address netting to an outflow while appearing in the outputs produced a link with no relation edge. The relations pre-check correctly found no edge and returned an empty result, disagreeing with the full walk and with the neighbor list. Filter on the netted flows instead (and fetch only the netted direction of the second address), making links, neighbors and the pre-check consistent.
An address on both sides of a tx was counted in reg_in and reg_out, creating relations and stats the spark transform never writes: relations pair net senders with net receivers, and stats count each tx on the net side only. Net the flows first, matching computeAddressTransactions + splitTransactions in graphsense-spark.
The fresh multi-input harvest (delta updater, driver backfill, and Spark one-off) merged coinjoin co-spends that the legacy Scala clustering filters out. All three paths now respect the transformed keyspace's configuration.coinjoin_filtering (default true); a NULL coinjoin flag counts as not-coinjoin. The Scala-parity regression's from-scratch harvest skips flagged txs the same way.
The net-sender-in-outputs address must not be linked (no relation edge, consistent with the neighbor list); the pure-output receiver of the same tx stays a baseline-compared link. The invariant test intentionally fails against pre-netting servers.
make rest REF=<ref|url|local> CUR=<ref|url|local> SIZE=small|medium|large resolves each side (git ref/working tree -> docker build+serve, hosted URL -> direct), wires auth and gateway headers, and runs the manual, fuzz, and loki suites cumulatively by size. No args prints full help with runtime estimates. Refresh stale README.
DEPTH=quick|standard|full (old small|medium|large kept as aliases) communicates the runtime tradeoff directly. Loki runs with pytest-xdist (LOKI_WORKERS, default 8), cutting the full depth from hours to ~15-30 min; estimates in the help corrected accordingly.
BTC is at ~0.7 * 2**32 unique addresses and cluster ids are derived from address ids (root == min address id), so the 2**32 offset would collide within its remaining headroom. Raw ids in the fresh_* tables and tagstore v2 relations are unshifted, so no data migration is needed; shifted ids stay far below the 2**53 JS-safe-integer limit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Follow-up fixes from the 2026-07-07 codebase audit — the higher-review items that
were held back from
developand done together on this branch. Covers Cassandraconnection/retry resilience, read-path performance, ingest/clustering safety, and
one data-correctness fix (ETH failed-transaction gas-fee accounting).
Each item is a separate commit. All changes pass the full pre-commit suite (ruff,
ty, full pytest, codegen check).
Changes
DB resilience (async + sync layers)
fix(db): stop blocking the event loop on Cassandra reconnect(90664f37) —connect()was recursive and re-created the wholeClusteron the read path onNoHostAvailable, blocking the asyncio reactor. It is now a bounded, non-recursiveconnect loop (
connect_max_attempts, default 60), and the read path lets atransient
NoHostAvailablepropagate to the driver's own reconnection policyinstead of tearing the cluster down.
fix(db): ride out transient stalls in the batch read helpers(3c07bf3c) —execute_statements_asyncandexecute_batchdid not route through theapplication-side
_execute_with_backoff, so a cluster-wide stall failed themimmediately. They now ride out
TRANSIENT_DB_ERRORSon the calling thread(per the retry architecture in
CLAUDE.md— backoff never runs on the driverreactor). Reads only; write paths already retry at the apply layer.
Read-path performance (web)
perf(web): normalize_address_transactions bounded-concurrent(cd77544b) —per-row tx lookups were serial (up to
pagesizesequential Cassandra round-tripsper request); now processed with a bounded
asyncio.gather(cap 100). Sameresults, far less tail latency.
perf(web): list_links filters the fresh page + concurrent cluster ids(
fad0192c) — filtering now operates on the freshly fetched page instead of thegrowing accumulator, and cluster-membership checks resolve concurrently. Also
tightens
links?token_currency=…so only the requested token's rows are fetched(no native-asset leak into token-filtered results).
REST serialization safety (web + regenerated python client)
fix(web): make first_tx/last_tx Optional, guard first_tx_id == -1(
f869ea3f) — addresses/entities with the-1"no originating tx" sentinel(miner-reward addresses, and now ETH fee-only senders — see below) no longer 500.
Regenerates the OpenAPI python client (
Addressmodel).Ingest / clustering safety
fix(ingest): remove orphaned side-table rows on append resume(592e2f1f) —Delta append-mode writes side tables before the
blocktable, so a crash betweenthem left rows that a resume from
MAX(block_id)+1would re-append as duplicates.On resume we now delete
block_id > <block-table max>from the side tables once,under the ingest lock, before catching up.
fix(transformation): lock + clamp for one-off clustering(580b56c4) —one-off clustering now holds the transformed-keyspace lock (so it cannot overlap a
running delta-updater or Spark transform) and clamps any address id beyond the
union-find size out of the feed (defensive against a Rust out-of-bounds panic on a
stale snapshot), warning on drops.
Data correctness — ETH failed-tx gas fee (the one with data implications)
fix(deltaupdate): debit failed-tx gas fee to the sender(b4849d28) — onETH the miner is credited the gas fee for every transaction including reverted
ones, but the payer was debited only if already tracked in the batch. A sender that
appears only in failed transactions was untracked, so its debit was dropped and
the fee was minted from nowhere — total supply (Σ balances) drifts upward,
monotonically, forever. Such fee-only senders are now added to the address set
and get a zero-stat address row (
first_tx_id = -1, mirroring miner rewards) so thedebit lands and the fee balances. TRX is unaffected (no miner fee credit; failed
txs are pre-filtered).
Deployment considerations
Group A — safe to roll out as-is, no data implications
90664f37,3c07bf3c,cd77544b,fad0192c,f869ea3f.These are connection/retry resilience, read-path performance, and API serialization
safety. They do not change stored data and do not require any re-transform or
backfill.
cd77544b/fad0192cincrease concurrent Cassandra reads per request(bounded), so the only thing to watch is coordinator load under peak fan-out.
f869ea3fis a Web API model change → deploy the regenerated client alongsidegs-rest.
Group B — ingest / ops behavior changes (safe, but be aware)
592e2f1f(append resume dedup): only affects append-mode ingest, and onlyon a resume after a crash. It issues a
DELETEon the Delta side tables forblock_idabove the last fully committed block — i.e. exactly the rows that wouldotherwise become duplicates. It never touches data at or below the last committed
block, so already-correct keyspaces are unaffected. No migration; effective on next
resume.
580b56c4(clustering lock): one-off clustering will now block while atransform/delta-update holds the transformed-keyspace lock (intended — prevents a
torn read). No change to existing data.
Group C — ETH failed-tx fee (
b4849d28): forward-only correction, history needs a re-transformThis is the one item where "just roll it out" needs a caveat.
behavior to newly processed blocks immediately. It does not depend on the
keyspace already being re-transformed, and it introduces no dependency on the Spark
side. Safe to deploy on its own — no new inflation accrues after it lands.
failed-tx fees remain wrong until the canonical Spark fix + full re-transform
lands (tracked separately, scheduled by ops "eventually"). This PR is the lib half
only; it deliberately does not attempt to rewrite history.
no_transactions = 0/first_tx_id = -1". Consumers must tolerate this:f869ea3f). Deploy the gs-rest change togetherwith / before the delta-updater change so these addresses serialize without a
500 the first time one is queried.
worth a quick check before promoting to a user-facing keyspace.
after the current id space (normal delta-updater behavior). A later full
re-transform reassigns every id from scratch anyway, so there is no id-scheme
coordination to get right now.
Suggested rollout order: Group A + B anytime → deploy gs-rest (
f869ea3f) →deploy the delta-updater (
b4849d28). Schedule the Spark re-transform separately toclear the historical inflation.
Testing
connect()retry, batch-helper ride-out, append-resumeorphan cleanup, ETH fee-only entity delta + a supply-conservation property test
(nets to zero with the fix, inflates by exactly
gas_used × gas_pricewithout it).pytest,ruff, andtygreen (enforced by pre-commit on every commit).Not in this PR / follow-ups
(the other half of the ETH correction).
421fix and CHANGELOG/release consolidation already landed ondevelopseparately.