v0.0.4: reference-locality bucketing (runtime autotune), sparse chunk walk, foundations

[alejandrogzi]

Summary

Sync of the tested experimental tree (alejandrogzi/hspZ experimental @ ce1de86) into production, as v0.0.4. Two exact speed-ups plus the foundation repairs behind them; every arm reproduced the frozen digests on NVIDIA L4, RTX 4090 and AMD (ZLUDA).

• ref-loc-buckets (default feature) — stable in-chunk reference-window bucketing before the score gate, survivors restored to original order, output byte-identical. L4 whole-genome wall −6.5% (11,029 / 10,901 → 10,274 / 10,224 s, disjoint reversed pairs). On an RTX 4090 the pass costs +3.7%, so HSPZ_REF_BUCKETS unset = auto: each engine alternates both paths in ≥3 s blocks, discards each block's first second and keeps the faster settled path (measured: L4 → on at 0.941/0.935, 4090 → off at 1.069/1.063). 1/0 force, HSPZ_REF_BUCKET_SHIFT pins the window, T4-class L2 forces off.
• Sparse chunk walk (default) — the per-seed-batch ~6.8 MB pageable copy of the cumulative hit array is replaced by a two-level search plus a ≤1 KB fetch per boundary; same chunks. 2× RTX 4090 whole genome W=2: 1,634 → 1,480 s (−9.6%), disjoint reversed pairs; L4 unchanged. HSPZ_CHUNK_WALK=full is the rollback.
• Foundations — semantic --max-hits target vs physical hit capacity (the historical over-cap tail no longer aborts and never changes a chunk); --dump-plan / --dump-manifest / --from-manifest; ledger timers for query pack / swaps / per-worker stage gaps; host-memory budget line. Multi-GPU output is byte-identical to one-GPU output at the same plan and cap.
• Packaging — version 0.0.4, changelog entry, docs page “Runtime tuning knobs”, regenerated CLI reference, assets/tests/multiplicity_gate.sh synced.

Verification (local, ZLUDA)

• cargo oxide build -- --release clean; --locked consistent after the one-line lock bump.
• cargo oxide test -- --release: 116 passed.
• chr20 fixture: digest 70152625c1a75cb3 / 3,638 HSPs identical to the experimental binary at --max-hits 16711680 and 2000000, and with HSPZ_REF_BUCKETS=1 and =0.
• pnpm docs:validate -- --sources and pnpm build pass.

Evidence and reviews live in the experimental repo (results/round85-chunkwalk/, results/round86-autotune/, results/round87-autotune-blocks/, benchmarks/baseline.md round-85 section).

🤖 Generated with Claude Code