harness: O(1) positional freshCases relocation (was a per-case name() scan) - #83
Merged
Conversation
The freshCases path re-found each case in its fresh instance by scanning name() linearly — O(census) boundary calls per case, the dominant cost of per-case-fresh-instance execution at conformance scale: measured under deltic on a synthetic 10k-case suite (PR #82's bench), the scan costs ~3.4us per name() call hot, ~17ms per case on average (33ms worst), ~170s across a full run — an order more than the all() re-enumeration it follows. O(N^2) overall. Enumeration order is contractually deterministic across instances of one artifact (wit/tests.wit enumeration rule; lockfiles already depend on it), so the census index relocates the case in O(1); one name() call verifies. This is stricter than the scan, which silently tolerated cross-instance reordering — a contract violation; order disagreement now throws the same unsound-run drift error a vanished case always did. Verified: just verify-deltic (incl. trap -> fresh-instance -> relocate sequence and striping legs), just verify-viewer; the viewer selftest's synthetic freshCases cases (same-order template, empty re-enumeration) cover both the happy and vanished paths unchanged. Relocation of the last case in the 10k suite: 33.4ms -> 0.0ms. Upstream deltic's runSuite mirrors the old harness semantics and keeps its own scan; tracked separately.
lann
added a commit
that referenced
this pull request
Aug 11, 2026
…ld vanishes The mint benchmark's follow-up (issue #25): with the registry-build half measured as ~3x the lift at 10k cases, try wizening the suite so fresh instances are born with the case table built. It works, but only by driving wasmtime-wizer 47 as a library (Wizer::run_component takes a caller-supplied instantiate closure, so our linker satisfies test-context — a host resource init never calls — plus full WASI). The CLI path is blocked three ways, recorded as finding 22: the invoke grammar rejects versioned interface qualifiers, unknown-import stubbing cannot synthesize resource types, and composed bundles hit 'nested components with modules not currently supported'. The init entry is therefore a bare-named wizer-initialize export: bench-suite's new wizer-init feature adds it as a second inline-WIT world, merged by wasm-component-ld. keep_init_func(false) — the default — emits an invalid component (dangling core-instance export reference); the driver keeps it. Custom sections survive the rewrite: scheduling and drift checks work on the wizened artifact. Measured (findings 23-24), 10k-case suite, medians: - wasmtime: all#1 3.15ms -> 663us (= all#2: born initialized); instantiate unchanged at ~19us (CoW absorbs the 122KB -> 1.29MB snapshot); store drop 80us -> 12us. End-to-end K=1 full-isolation run: 30.8s -> 7.1s sequential, 1.14s at jobs=8 — per-case isolation on a wizened suite now undercuts #22's shared-instance numbers. - deltic: net ~1.5x only — all#1 6.9ms -> 2.9ms but instantiate 0.78ms -> 2.17ms (no CoW; the active data segment is copied per instantiation). K>1 remains the JS-leg lever. Also corrects finding 21's scan-cost constant: hot-loop name() is ~3.4us (a cold single call measures ~26us), so the freshCases scan averaged ~17ms/case at 10k (33ms worst, measured), not 130ms; fixed by positional relocation in PR #83 (33.4ms -> 0.0ms measured). New surfaces, all out of the gates: bench-suite feature wizer-init (default build unchanged — a pure suite world), runner feature wizer with the required-features bin wizer-preinit (optional dep wasmtime-wizer, absent from default builds; clippy clean under the feature).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
freshCasespath re-found each case in its fresh instance by a linearname()scan — O(census) boundary calls per case, which at conformance scale dominates everything else in the per-case-fresh-instance loop. Measured under deltic on #82's synthetic 10k-case suite: ~3.4µs pername()call hot → ~17ms/case average (33ms worst), ~170s of pure scanning across a full 10k run. O(N²) overall.all()order is contractually deterministic across instances of one artifact (the enumeration rule inwit/tests.wit; lockfiles already depend on it), so the census index relocates in O(1); onename()call verifies.Semantics note: this is stricter than the scan, which silently tolerated cross-instance reordering — a contract violation. Order disagreement now throws the same unsound-run drift error a vanished case always did (message says what was found at the index).
Measured effect (deltic, 10k cases, relocating the last case):
Verification
just verify-deltic— includes the real trap → fresh-instance → relocation sequence (fixture/trap/aftergreen afterboom), tag scheduling, striping partition equality.just verify-viewer— viewer selftest's synthetic freshCases cases (same-order template; empty re-enumeration must throw) pass unchanged.Not addressed here: upstream deltic's
runSuitemirrors the old harness semantics and keeps its own scan — worth an upstream issue; and thect-runnerwasmtime path already relocates positionally (session.cases.get(index)), so no change needed there.