diff --git a/contracts/intrinsics.md b/contracts/intrinsics.md index d03e091..bb55927 100644 --- a/contracts/intrinsics.md +++ b/contracts/intrinsics.md @@ -209,7 +209,9 @@ ending at resolution-plus-block rather than activation end" — was a mischaracterization of wasmtime corrected on 2026-08-10 (wasmtime holds the gate for the whole invocation and defers the entry *decision*; see upstream-component-model-repo-findings.md CM-4 and -exams/wasmtime-exclusivity/wasmtime-actual-semantics.md). deltic's +[deltic#43](https://github.com/lann/deltic/issues/43), where the +evidence is distilled — exam kit archived at +`4f3351f:exams/wasmtime-exclusivity/`). deltic's release-at-resolution rule was **removed the same day** ([#43](https://github.com/lann/deltic/issues/43)): the runtime now implements the hold rule (gate lifetime = the core invocation, pristine diff --git a/docs/architecture.md b/docs/architecture.md index ce4fbd9..a7716c8 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -56,22 +56,24 @@ contracts throughout the repo. Related documents: wasmtime exhibits it too. The tie-breaking authority for semantic questions is the spec + `definitions.py`, with wasmtime as corroborating evidence — never the other way around. **One bounded exception** - (operator decision, 2026-08-09; content corrected 2026-08-10): where - `definitions.py` contradicts the spec repo's *own wast corpus* and - wasmtime implements the corpus side, the corpus semantics — **as - wasmtime actually implements it, verified against wasmtime source or - trace, not as inferred from the test alone** (the CM-4 lesson: - sync-streams.wast was satisfiable by two different semantics, and the - first inference picked the wrong one) — is adopted as the working - assumption; the expectation is that the reference will be amended, not - the tests. Each such case must be a named finding in - `upstream-component-model-repo-findings.md` (currently CM-3 and CM-4; - CM-4's corrected model — hold-lifetime gate + deferred entry decision — - landed in the runtime via - [#43](https://github.com/lann/deltic/issues/43); amendment sketch: - `exams/wasmtime-exclusivity/spec-amendment.md`) and flips back if - upstream adjudicates the other way. Bare wasmtime behavior with no - corpus backing never supersedes the reference. + (operator decision, 2026-08-09): where `definitions.py` contradicts the + spec repo's *own wast corpus* and wasmtime implements the corpus side, + the corpus semantics — as wasmtime actually implements it, verified + against wasmtime source or trace, not as inferred from the test alone — + is adopted as the working assumption. Each such case must be a named + finding in `upstream-component-model-repo-findings.md` (currently CM-3 + only) and flips back if upstream adjudicates the other way. Bare + wasmtime behavior with no corpus backing never supersedes the + reference. Guard before invoking the exception (the CM-4 lesson, + adjudicated 2026-08-10 — + [deltic#43](https://github.com/lann/deltic/issues/43)): a corpus + assertion counts as semantic authority only if it is + **schedule-independent**. `sync-streams.wast:145` turned out to pin a + scheduler policy, not semantics — two conforming policies over the same + agreed gate semantics answer it differently — so it is an upstream test + defect and no semantics were ever in conflict; deltic satisfies it via + a non-normative scheduler policy (hold gate + drain-to-quiescence entry + decision, PR #45). - TypeScript throughout the JS side: the runtime, the harness, and all generated bindings. - A performance story that can get fast later without rearchitecting. diff --git a/exams/wasmtime-exclusivity/RESULTS.md b/exams/wasmtime-exclusivity/RESULTS.md deleted file mode 100644 index 7d7efeb..0000000 --- a/exams/wasmtime-exclusivity/RESULTS.md +++ /dev/null @@ -1,123 +0,0 @@ -# wasmtime exclusivity check — is the sync-streams pass deterministic? - -> **CORRECTION (2026-08-10) — see `wasmtime-actual-semantics.md`, which -> supersedes this file's wasmtime claims.** Source + trace verification -> against wasmtime main (and v47.0.3) shows wasmtime does **not** release -> its entry gate at resolution: `do_not_enter` is held for the whole core -> invocation (same lifetime as the reference's `exclusive_thread`), across -> post-`task.return` mid-frame parks included. Its sync-streams pass comes -> from **deferred entry evaluation + FIFO scheduling**: the async caller -> suspends until the first status event, the ready-parked `$C.get` runs to -> *exit* first, and `$C.set` is admitted only after `get` is deleted — no -> same-instance execution ever overlaps the parked span. Consequently: the -> "STARTED while a resolved task sits parked mid-frame" reading of the -> official test below is wrong for wasmtime; §"What the official test -> already demonstrates" item 3 and the "one true scheduling accident" -> section describe the **release-rule** (deltic/fix-patch) execution, not -> wasmtime's. The determinism measurement (50/50) and the CI-provenance -> analysis stand. - -Evidence backing `upstream-component-model-repo-findings.md` CM-4's -2026-08-09 review and the IROH-1 consumer finding. Question posed by the -operator: wasmtime runs the official CM test suite — is *its* pass on the -CM-4 arbiter (`test/async/sync-streams.wast`) itself scheduling accident? - -**Answer: no — deterministic semantics.** A custom counter-experiment wast -was planned and then dropped as redundant: the official test already -contains the full IROH-1 shape, and wasmtime's pass on it is deterministic -by construction and by measurement. - -## What the official test already demonstrates - -`test/async/sync-streams.wast` (spec repo, e8d8005 vintage and current): - -1. `$C.get` calls `task.return` (RESOLVED) and then parks **mid-frame** in a - synchronous `stream.write` (first to the rendezvous). -2. `$D.run` — sync-lowered, so it regains control at `get`'s *resolution*, - while `get`'s thread is still parked — rendezvous-reads, then calls - `$C.set` **without ever yielding in between** (straight-line core wasm). -3. Line 145 asserts `set` reports **STARTED**: the same-instance task was - admitted, and its body *executed* (it runs its own `stream.read` and - memory traffic — clobbering address 16, which `get`'s parked frame also - uses — before blocking). Under `definitions.py`'s `exclusive_thread` - lifetime, `set` would be gated (STARTING) and the guest traps - `unreachable` at line 146. - -That is: entry admitted + interloper body run + shared state mutated, all -while a resolved task sits parked mid-frame — the IROH-1 collision shape, -in the official corpus, asserted as the *expected* behavior. - -## Why the pass is deterministic (structure) - -Between the rendezvous read and the `set` call, `$D.run` executes -straight-line wasm with no suspension point; execution is single-threaded -and cooperative. The machine state at the entry-gate check is therefore a -pure function of the program, and STARTED-vs-gated is decided solely by the -gating rule (wasmtime: `ConcurrentInstanceState.do_not_enter`, scoped to -the sync-call span, ending at resolution). No race exists at the arbiter. - -## Measurement - -wasmtime 49.0.0-dev (`3ebfbe5af`, 2026-08-07 — a current-main dev release; -the 47.0.1 release CLI cannot parse the post-#655 suite syntax): - -```sh -./run.sh # fetches nothing; expects `wasmtime` ≥ the 2026-07 suite syntax on PATH -``` - -Result: **50/50 identical silent passes** (exit 0) of -`third_party/component-model/test/async/sync-streams.wast` with -`-W component-model-async=y -W component-model-more-async-builtins=y`. - -deltic's own lane passes the same file deterministically (green under -`DELTIC_SCHED_SEED=1` and `=4242`; see `harness/src/xfail.ts`'s -sync-streams note and CI). - -## CI provenance (who actually runs this suite) - -- The spec repo's CI runs **only** `design/mvp/canonical-abi/run_tests.py` - — the wast suite is never executed against the reference. -- wasmtime vendors the spec repo as the `tests/component-model` submodule - (bumped to e8d8005 on 2026-07-24, commit b6cb744) and runs the suite in - CI with an explicit exception ledger; `sync-streams.wast` is not on it. - -So the reference↔suite contradiction (CM-4) has no upstream detector: each -CI is green against its own half. - -## The one true "scheduling accident" in the story - -Downstream only: the *semantics* (suite-pinned, wasmtime-implemented, -deltic-implemented) **admit** same-instance task execution during a -resolved task's mid-frame block; whether an admitted interleaving lands on -a *colliding* point (polymorph-iroh's `RefCell` window, IROH-1) is a -scheduler choice — deterministic per host, guaranteed by nothing. wasmtime's -endpoint-matrix green and deltic's ~90% trap rate are the same semantics -under different deterministic schedules. - -## Addendum (2026-08-09): the filing test-diff, and a second contradiction - -`cm4-run-tests.patch` adds `test_resolved_task_gates_entry` to the -reference's own `run_tests.py` — the sync-streams.wast shape in the file's -own idiom (callback-lifted `pump` resolves then parks mid-frame holding a -shared "borrow" flag; same-instance `poke` reads and mutates it; the -suite-pinned expectations are asserted after quiescence). Against pristine -`definitions.py` it fails at -`assert(poke_state == Subtask.State.RETURNED)` — STARTING observed, the -interloper never admitted: CM-4, demonstrated inside the reference's own -harness by `python3 run_tests.py`. - -`cm4-reference-fix.patch` transplants the wasmtime/deltic resolution-scoped -gate into `definitions.py` (3 hunks). With it, the new test passes — and -**exactly one stock test fails**: `test_callback_interleaving`'s second -progress-free poll window, which turns out to *encode the hold-semantics* -(the gated producer is admitted and completes inside the window once the -resolved producer's post-resolution sync read stops holding the slot). -Full trace in `root-cause.md`. So the spec repo's unit tests and its wast -suite pin **contradictory semantics**, invisible upstream because no CI -runs the wast corpus against the reference. - -`verify-cm4.sh` reproduces all four legs from the pristine submodule -copies (stock-pass / test-fails-pristine / test-passes-fixed / -fix-breaks-interleaving). Harness note: the reference hangs after any -failing assertion (non-daemon threads) — legs run under `timeout` and are -judged by traceback text. diff --git a/exams/wasmtime-exclusivity/cm4-reference-fix.patch b/exams/wasmtime-exclusivity/cm4-reference-fix.patch deleted file mode 100644 index c389fd1..0000000 --- a/exams/wasmtime-exclusivity/cm4-reference-fix.patch +++ /dev/null @@ -1,63 +0,0 @@ ---- a/design/mvp/canonical-abi/definitions.py -+++ b/design/mvp/canonical-abi/definitions.py -@@ -376,6 +376,15 @@ - cancelled = Cancelled.FALSE - - def block_internal(self, cancellable): -+ # Entry-gating ends at resolution: a RESOLVED task's thread that blocks -+ # mid-frame stops gating its instance's entry and never retakes the -+ # slot. This matches wasmtime (ConcurrentInstanceState.do_not_enter, -+ # scoped to the sync-call span) and the behavior test/async/ -+ # sync-streams.wast asserts; see test_resolved_task_gates_entry. -+ t = self.task -+ if t.ft.async_ and t.needs_exclusive() and \ -+ t.state == Task.State.RESOLVED and t.inst.exclusive_thread is self: -+ t.inst.exclusive_thread = None - self.cancellable = cancellable - cancelled = block(switch_to = None) - assert(self.running() and (cancellable or not cancelled)) -@@ -503,8 +512,8 @@ - def exit_implicit_thread(self): - assert(current_thread() is self.implicit_thread) - self.unregister_thread(self.implicit_thread) -- if self.ft.async_ and self.needs_exclusive(): -- assert(self.inst.exclusive_thread is self.implicit_thread) -+ if self.ft.async_ and self.needs_exclusive() and \ -+ self.inst.exclusive_thread is self.implicit_thread: - self.inst.exclusive_thread = None - - def unregister_thread(self, thread): -@@ -2184,11 +2193,14 @@ - [packed] = call_and_trap_on_throw(callee, flat_args) - code,si = unpack_callback_result(packed) - while code != CallbackCode.EXIT: -- assert(task.needs_exclusive() and inst.exclusive_thread is task.implicit_thread) -- inst.exclusive_thread = None -+ assert(task.needs_exclusive()) -+ held = inst.exclusive_thread is task.implicit_thread -+ if held: -+ inst.exclusive_thread = None -+ free = (lambda: not inst.exclusive_thread) if held else (lambda: True) - match code: - case CallbackCode.YIELD: -- cancelled = thread.wait_until(lambda: not inst.exclusive_thread, cancellable = True) -+ cancelled = thread.wait_until(free, cancellable = True) - if cancelled: - event = (EventCode.TASK_CANCELLED, 0, 0) - else: -@@ -2196,11 +2208,12 @@ - case CallbackCode.WAIT: - wset = inst.handles.get(si) - trap_if(not isinstance(wset, WaitableSet)) -- event = wset.wait_for_event_and(lambda: not inst.exclusive_thread, cancellable = True) -+ event = wset.wait_for_event_and(free, cancellable = True) - case _: - trap() -- assert(inst.exclusive_thread is None) -- inst.exclusive_thread = task.implicit_thread -+ if held: -+ assert(inst.exclusive_thread is None) -+ inst.exclusive_thread = task.implicit_thread - event_code, p1, p2 = event - [packed] = call_and_trap_on_throw(opts.callback, [event_code, p1, p2]) - code,si = unpack_callback_result(packed) diff --git a/exams/wasmtime-exclusivity/cm4-run-tests.patch b/exams/wasmtime-exclusivity/cm4-run-tests.patch deleted file mode 100644 index 3cf34cf..0000000 --- a/exams/wasmtime-exclusivity/cm4-run-tests.patch +++ /dev/null @@ -1,126 +0,0 @@ ---- a/design/mvp/canonical-abi/run_tests.py -+++ b/design/mvp/canonical-abi/run_tests.py -@@ -1254,6 +1254,115 @@ - assert(got[0] == 84) - - -+def test_resolved_task_gates_entry(): -+ # Mirrors test/async/sync-streams.wast: there, $C.get resolves via -+ # task.return and then parks MID-FRAME in a synchronous stream.write; $D.run -+ # — which never yields in between — then calls $C.set, and the guest asserts -+ # (line 145, driving the assert_return at line 208) that this same-instance -+ # call is admitted, not gated at entry. Wasmtime passes that test -+ # deterministically: its entry gate (ConcurrentInstanceState.do_not_enter) -+ # ends at RESOLUTION. Under definitions.py as written, -+ # inst.exclusive_thread is held for the implicit thread's whole life — -+ # including post-resolution mid-frame blocks — so here `poke` parks -+ # waiting-to-enter (STARTING) and the assertions at the bottom fail. -+ # -+ # `shared` stands in for guest state the parked frame holds across its -+ # block (e.g. a Rust RefCell borrow): if the interloper is admitted (the -+ # sync-streams semantics), it reads and mutates that state under the parked -+ # frame, which the parked frame observes on resume — the interleaving that -+ # definitions.py's exclusive_thread lifetime makes impossible. -+ # -+ # All contested assertions run after quiescence (pump unblocked and done, -+ # poke drained) so the failure is a clean AssertionError, never a hang. -+ store = Store() -+ root_inst = ComponentInstance(store) -+ inst = ComponentInstance(store, root_inst) -+ -+ shared = {'locked': 0, 'poke_saw': None, 'pump_observed': None, 'pump_done': False} -+ fut = RacyBool(False) -+ -+ def never_called_back(args): -+ assert(False) -+ -+ pump_ft = FuncType([], [], async_ = True) -+ pump_opts = mk_opts() -+ pump_opts.async_ = True -+ pump_opts.callback = never_called_back -+ def pump_core(args): -+ assert(len(args) == 0) -+ canon_task_return([], pump_opts, []) # RESOLVED; the task keeps running -+ shared['locked'] = 1 # "borrow" taken... -+ current_thread().wait_until(fut.is_set) # ...held across a mid-frame block -+ shared['pump_observed'] = shared['locked'] -+ shared['locked'] = 0 -+ shared['pump_done'] = True -+ return [CallbackCode.EXIT] -+ pump = store.lift(pump_core, pump_ft, pump_opts, inst) -+ -+ poke_ft = FuncType([], [], async_ = True) -+ poke_opts = mk_opts() -+ poke_opts.async_ = True -+ poke_opts.callback = never_called_back -+ def poke_core(args): -+ assert(len(args) == 0) -+ shared['poke_saw'] = shared['locked'] -+ shared['locked'] = 2 -+ canon_task_return([], poke_opts, []) -+ return [CallbackCode.EXIT] -+ poke = store.lift(poke_core, poke_ft, poke_opts, inst) -+ -+ consumer_heap = Heap(16) -+ consumer_inst = ComponentInstance(store, root_inst) -+ consumer_ft = FuncType([], [U32Type()], async_ = True) -+ consumer_opts = mk_opts(MemInst(consumer_heap.memory, 'i32'), async_ = True) -+ def consumer(args): -+ assert(len(args) == 0) -+ -+ [ret] = store.lower(pump, pump_ft, consumer_opts, consumer_inst)([]) -+ pump_state,pump_subi = unpack_result(ret) -+ assert(pump_state == Subtask.State.RETURNED) # resolved during the call... -+ assert(pump_subi == 0) -+ assert(shared['locked'] == 1) # ...its thread parked mid-frame -+ -+ # sync-streams.wast semantics: admitted and run to completion -+ # (RETURNED, no handle). definitions.py as written: gated at entry -+ # (STARTING, with a handle). Captured here, asserted after quiescence. -+ [ret] = store.lower(poke, poke_ft, consumer_opts, consumer_inst)([]) -+ poke_state,poke_subi = unpack_result(ret) -+ poke_saw_when_lowered = shared['poke_saw'] -+ -+ # Unblock the pump and reach quiescence on both subtasks so a failing -+ # assertion below cannot strand parked threads. -+ fut.set() -+ current_thread().wait_until(lambda: shared['pump_done']) -+ if poke_subi != 0: -+ [seti] = canon_waitable_set_new() -+ [] = canon_waitable_join(poke_subi, seti) -+ [event] = canon_waitable_set_wait(True, MemInst(consumer_heap.memory, 'i32'), seti, 0) -+ assert(event == EventCode.SUBTASK) -+ canon_subtask_drop(poke_subi) -+ [] = canon_waitable_set_drop(seti) -+ -+ # The sync-streams.wast:145/:208 expectations: -+ assert(poke_state == Subtask.State.RETURNED) # admitted, not gated -+ assert(poke_subi == 0) -+ assert(poke_saw_when_lowered == 1) # its body read state under the parked frame -+ assert(shared['pump_observed'] == 2) # the parked frame saw poke's mutation -+ -+ canon_task_return([U32Type()], consumer_opts, [42]) -+ return [] -+ -+ def on_start(): return [] -+ -+ got = None -+ def on_resolve(results): -+ nonlocal got -+ got = results -+ -+ lift_and_run(consumer_opts, consumer_inst, consumer_ft, consumer, on_start, on_resolve) -+ assert(got[0] == 42) -+ -+ - def test_sync_using_wait(): - store = Store() - root_inst = ComponentInstance(store) -@@ -3027,6 +3136,7 @@ - test_async_callback() - test_callback_interleaving() - test_sync_ignores_backpressure() -+test_resolved_task_gates_entry() - test_async_to_sync() - test_async_backpressure() - test_sync_using_wait() diff --git a/exams/wasmtime-exclusivity/root-cause.md b/exams/wasmtime-exclusivity/root-cause.md deleted file mode 100644 index 2dad9e5..0000000 --- a/exams/wasmtime-exclusivity/root-cause.md +++ /dev/null @@ -1,110 +0,0 @@ -# CM-4 root cause: the reference's unit tests pin the semantics its wast suite contradicts - -> **CORRECTION (2026-08-10) — see `wasmtime-actual-semantics.md`.** This -> file's reference-side traces are accurate, but its framing of -> `cm4-reference-fix.patch` as "the wasmtime/deltic rule" is wrong: -> release-at-resolution is **deltic's** rule only. wasmtime holds its gate -> for the whole invocation (like pristine `definitions.py`) and passes the -> wast suite via deferred entry evaluation — under wasmtime's actual -> semantics `test_callback_interleaving` passes **unchanged**, so the -> "irreconcilable" verdict below applies to the release-rule fix, not to -> wasmtime alignment. Likewise `test_resolved_task_gates_entry`'s -> shared-state assertions (`poke_saw == 1`, `pump_observed == 2`) encode -> the release rule: under wasmtime's model poke is deferred until pump's -> invocation exits. The wasmtime-aligned reference fix is an entry-timing -> change in `canon_lower` (report STARTING only after the instance's -> runnable work is exhausted), not a gate-lifetime change. - -Companion to `RESULTS.md`; produced while building the filing artifacts -(`cm4-run-tests.patch`, `cm4-reference-fix.patch`). Line numbers are -pristine `run_tests.py` / `definitions.py` at submodule `73b7ad5`. - -## The new test (cm4-run-tests.patch) - -`test_resolved_task_gates_entry` — the sync-streams.wast shape in -run_tests.py's own idiom: `pump` (async callback lift, `needs_exclusive`) -calls `canon_task_return` then parks mid-frame holding a shared "borrow" -flag; `poke` (same instance) reads and mutates that state; a consumer in -another instance lowers both and asserts the wast suite's expectations -after quiescence. - -- Pristine `definitions.py`: fails at - `assert(poke_state == Subtask.State.RETURNED)` — observed - `STARTING` + a subtask handle, `poke_saw = None` (never admitted): the - CM-4 divergence, exactly. -- With `cm4-reference-fix.patch` applied: passes (3/3 runs), and the - shared-state assertions demonstrate the admitted interleaving - (`poke_saw == 1`, `pump_observed == 2`). - -## The fix experiment (cm4-reference-fix.patch) and what it revealed - -Three hunks transplanting the wasmtime/deltic rule (entry gating ends at -resolution): `Thread.block_internal` releases the slot when a RESOLVED -`needs_exclusive` task's holder-thread parks; `exit_implicit_thread` -releases only-if-holder; the callback loop's release/retake becomes -held-guarded. - -Full-suite result: **everything passes except `test_callback_interleaving`** -(5/5 runs), which fails in its second progress-free poll window -(`assert(ret == EventCode.NONE)`, pristine :989–995 region). - -## Mechanism, fully traced (both windows, both semantics) - -Cast: producer1 and producer2, callback-lifted (`needs_exclusive`) in one -instance; a consumer feeding them futures and polling its waitable set. - -1. producer1 is lowered and parks **pre-resolution** in its initial - frame's sync `future_read(fut11)` (the consumer writes `wfut11` only - later, :957) — slot held. The `STARTING` assertions for `todie`/ - producer2 (:934–:951) therefore hold under **both** semantics: - pre-resolution gating is uncontested. -2. `wfut11` written → producer1's initial frame continues → async - `fut12` read → BLOCKED → returns `CallbackCode.WAIT` — the callback - loop releases the slot (:2187–2188). producer2 is admitted, runs its - initial frame, parks pre-resolution in its sync `future_read(fut21)` - — now producer2 holds the slot. Its STARTING→STARTED event is - consumed at :959–963. -3. First poll window (:966–973): no pending events; producer2 parked - pre-resolution (slot correctly held under both semantics). `NONE` ×10 - passes under both. ✔ -4. `wfut21` written (:975) → producer2 continues → async `fut22` read → - BLOCKED → WAIT → slot released. `wfut12` written earlier delivered - producer1's FUTURE_READ; its callback runs: `canon_task_return(42)` - (**RESOLVED**), then a **post-resolution sync `future_read(fut13)`** - (:888–896 region) — parked mid-frame. The consumer consumes - producer1's RETURNED subtask event (:977–985). -5. `wfut22` written (:987) → producer2's FUTURE_READ event is pending; - producer2 sits in callback-WAIT needing the slot to retake. - - **Pristine**: the slot is held by producer1's post-resolution sync - read → producer2 cannot wake → second poll window (:989–995) sees - `NONE` ×10. ✔ (hold-semantics encoded here) - - **Fix**: `block_internal` released producer1's slot → producer2 - retakes, its callback runs `task.return` → RETURNED → a SUBTASK - event (`state = RETURNED`) lands in the consumer's set → the poll - observes it → `assert(ret == EventCode.NONE)` fails. ✘ - (diagnostic observed: `event=SUBTASK p2=RETURNED` inside the window) - -So: exactly one assertion window in the reference's unit tests depends on -the hold-semantics — and it is load-bearing there, while -`test/async/sync-streams.wast:208` (run by wasmtime CI, deterministically -green) demands the opposite. **The spec repo's two test corpora pin -contradictory semantics**, and since no upstream CI runs the wast corpus -against `definitions.py`, neither side can notice. - -## Verdict for the fix patch - -Honest verdict: irreconcilable without changing existing expectations — -by design, not by defect. A resolution-scoped `definitions.py` must also -rewrite `test_callback_interleaving`'s second poll window (consume the -producer2 RETURNED event there instead of asserting quiescence, and -re-sequence its tail). The patch is shipped as the demonstration of what -the wasmtime-aligned semantics look like in the reference's own model, -not as a ready-to-merge change. - -## Harness note (for the filing's presentation) - -Any failing assertion in a threaded run_tests.py test prints its -traceback and then hangs the interpreter on non-daemon reference threads -— verified by injecting `assert(False)` into stock -`test_async_backpressure` (traceback, then timeout). Run the suite under -`timeout`; judge by the traceback, not the exit. diff --git a/exams/wasmtime-exclusivity/run.sh b/exams/wasmtime-exclusivity/run.sh deleted file mode 100755 index 476708b..0000000 --- a/exams/wasmtime-exclusivity/run.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env sh -# Determinism check for wasmtime on the CM-4 arbiter (see RESULTS.md). -# Requires a `wasmtime` new enough to parse the post-#655 suite syntax -# (a 2026-08+ dev release works; the 47.0.1 release CLI does not). -set -eu -cd "$(dirname "$0")/../.." -WAST=third_party/component-model/test/async/sync-streams.wast -WASMTIME="${WASMTIME:-wasmtime}" -"$WASMTIME" --version -N="${N:-50}" -i=1 -while [ "$i" -le "$N" ]; do - "$WASMTIME" wast -W component-model-async=y -W component-model-more-async-builtins=y "$WAST" - i=$((i + 1)) -done -echo "$N/$N passes: deterministic" diff --git a/exams/wasmtime-exclusivity/spec-amendment.md b/exams/wasmtime-exclusivity/spec-amendment.md deleted file mode 100644 index ed71496..0000000 --- a/exams/wasmtime-exclusivity/spec-amendment.md +++ /dev/null @@ -1,139 +0,0 @@ -# CM-4: how and why the spec would be amended (plain-language sketch) - -Companion to `RESULTS.md` (evidence), `root-cause.md` (reference-side -mechanism), and `wasmtime-actual-semantics.md` (wasmtime-side mechanism, -2026-08-10 — which corrected this document's premises; the previous -revision wrongly attributed a release-at-resolution gate to wasmtime). -Line references: spec submodule `73b7ad5`; wasmtime main as of 2026-08-10. - -## The contradiction, restated correctly - -`test/async/sync-streams.wast:145` requires an async-lowered call to a -same-instance callback-lifted export to report **STARTED** in a state -where the previous task has resolved (`task.return`) but is still parked -mid-frame in a sync builtin — *and is ready to resume*. Pristine -`definitions.py` deterministically reports STARTING there (guest traps at -:146): its `canon_lower` runs the callee **eagerly, inline**, so the -entry check happens at the call instant, while the parked-but-ready -holder still holds `exclusive_thread`. - -wasmtime passes — but not by releasing its gate at resolution. Verified -from source and runtime trace (`wasmtime-actual-semantics.md`): - -- wasmtime's gate (`ConcurrentInstanceState.do_not_enter`) has the **same - lifetime** as the reference's `exclusive_thread`: the whole core - invocation, held across post-return mid-frame parks. -- What differs is **when the entry decision is made**: the call is - queued, the caller suspends until the first status event, and the - executor first drains work queued ahead of the call — including the - ready-to-resume gate holder, which runs to invocation exit and releases - the gate. Only then is the new call's readiness evaluated: STARTED. -- If the holder is *not* ready (parked on an un-rendezvous'd operation), - the new call parks as pending and the caller receives **STARTING** — - hold semantics, observably. - -So, in one sentence, the question the repo must adjudicate: - -> **When is an async-lowered call's initial status decided — eagerly at -> the call instant (`definitions.py` today), or only after the instance's -> runnable work has been exhausted (wasmtime)?** - -Gate lifetime is *not* in dispute between wasmtime and the reference: -both hold to invocation exit. (A third semantics — releasing the gate at -resolution — also satisfies the wast corpus; deltic currently ships it. -See "the release-rule alternative" below.) - -## Why the deferred rule should win upstream - -1. **It is what ships.** wasmtime implements it; the wast corpus is - maintained against wasmtime as its documented runner (test/README.md) - and passes deterministically. Amending the reference changes nothing - in production. -2. **It preserves every existing unit-test expectation.** Under hold + - deferred entry, the reference's `test_callback_interleaving` — both - hold-encoding sites (the :990–995 progress-free window and the - :1009–1011 STARTING tail) — passes **unchanged**. The amendment - touches `canon_lower`'s scheduling, not `exclusive_thread`'s - lifetime, so no existing gate-lifetime semantics move. -3. **It keeps Invariant #3 airtight with no carve-out.** Because the gate - still spans the whole invocation, no same-instance execution is ever - admitted while a task (resolved or not) has live frames parked - mid-block. Single-shadow-stack LIFO discipline (Explainer.md - :3007–3011) is preserved unconditionally — the producer pattern never - interleaves with live frames. -4. **The STARTING-at-the-instant answer is arguably a scheduling artifact - anyway.** The reference reports "not started" about a state in which - the only obstacle is a holder that is already unblocked and merely - unscheduled. The deferred rule reports the status of a settled state. - -Cost, honestly stated: the deferred rule makes the instance-entry status -depend on scheduler drain order. wasmtime's own pass is FIFO-dependent -(had the new call been evaluated before the ready holder resumed, the -caller would see STARTING and the guest would trap). A spec formulation -should therefore be order-robust, e.g.: - -> An async-lowered call reports STARTING only if the callee remains -> unstarted after the instance's runnable work has been exhausted -> (drain to quiescence, not a single scheduling step). - -which is deterministic under any fair scheduler, including seeded-shuffle -testing. - -## What actually changes (all in the spec repo) - -1. `definitions.py` `canon_lower` (async path, :2284+): defer the - initial-status decision — run/park the callee only after ready threads - have been given the chance to run to quiescence (the reference's - cooperative-model equivalent of wasmtime's queued `StartImplicit` + - caller-waits-for-first-status, concurrent.rs:3040–3160, :1497–1522). - `exclusive_thread`, `enter_implicit_thread`, and the callback event - loop are **untouched**. -2. `CanonicalABI.md` — document the deferred entry decision alongside - `canon_lower`; the :3740–3748 lock-lifetime prose stays as-is (it is - correct, and wasmtime agrees with it). -3. `run_tests.py` — no changes required; the existing corpus (including - `test_callback_interleaving`) already encodes hold semantics, which - survive. -4. `Explainer.md` Invariant #3 — no carve-out needed. - -Contrast with the release-rule amendment (previous revision of this -sketch): that path changes `exclusive_thread`'s lifetime (3 hunks), -rewrites `test_callback_interleaving`'s window and tail, and requires a -toolchain-visible Invariant #3 carve-out (post-return code loses the -serialization guarantee at block points). The deferred-entry path -requires none of that. Both reconcile the wast corpus; the deferred path -is the strictly smaller and safer change — and it is the one wasmtime -actually implements. - -## The release-rule alternative (deltic's current shipping semantics) - -Release-at-resolution + eager start also passes the full official wast -suite (measured under deltic, green under scheduler seeds — it is -order-robust by construction, since the gate is simply open). Its costs: - -- Admits same-instance execution *during* a resolved task's mid-frame - park: live-frame interleaving on the single shadow stack → Invariant #3 - carve-out required; and application-state reentrancy the ecosystem's - flagship does not exhibit (the IROH-1 collision is reachable under this - rule and unreachable under wasmtime's — see - `upstream-consumer-findings.md`). -- Contradicts the reference's own `test_callback_interleaving`, so the - upstream diff is larger and touches pinned semantics. - -Per the 2026-08-10 operator decision, deltic migrates to the wasmtime -model (hold + deferred entry); the release rule is no longer proposed -upstream. - -## Why this likely lands (process) - -- Both corpora are the same author's work (Luke Wagner: the exclusivity - model in #553, 2025-08-20; `sync-streams.wast` and fixes from - 2025-09-05). The filing is "your reference's *entry-timing* choice - contradicts your own suite and your flagship's shipped behavior" — a - scheduling fix with zero collateral in the unit-test corpus. -- Precedent: CM-3 (`upstream-component-model-repo-findings.md`) resolved - the same suite+wasmtime-vs-reference triangle in the same direction - (fix the reference). -- Structural follow-up for the same filing: the spec repo's CI runs only - `run_tests.py` (main.yml:16); the contradiction stayed invisible for - ~11 months because no CI runs the wast corpus against the reference. diff --git a/exams/wasmtime-exclusivity/trace-sync-streams-wasmtime-dev.log b/exams/wasmtime-exclusivity/trace-sync-streams-wasmtime-dev.log deleted file mode 100644 index ab3503b..0000000 --- a/exams/wasmtime-exclusivity/trace-sync-streams-wasmtime-dev.log +++ /dev/null @@ -1,151 +0,0 @@ -2026-08-10T15:45:16.303641Z TRACE wasmtime::runtime::component::concurrent: enter sync call RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(1) } -2026-08-10T15:45:16.303663Z TRACE wasmtime::runtime::component::concurrent: exit sync call RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(1) } -2026-08-10T15:45:16.303665Z TRACE wasmtime::runtime::component::concurrent: ready to delete? true (threads_completed: true, has_sync_result: false, pending_completion_event: false, host_future_state: NotApplicable) -2026-08-10T15:45:16.303667Z TRACE wasmtime::runtime::component::concurrent: delete guest task GuestTask(0) -2026-08-10T15:45:16.303671Z TRACE wasmtime::runtime::component::concurrent: enter sync call RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(1) } -2026-08-10T15:45:16.303673Z TRACE wasmtime::runtime::component::concurrent: exit sync call RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(1) } -2026-08-10T15:45:16.303674Z TRACE wasmtime::runtime::component::concurrent: ready to delete? true (threads_completed: true, has_sync_result: false, pending_completion_event: false, host_future_state: NotApplicable) -2026-08-10T15:45:16.303675Z TRACE wasmtime::runtime::component::concurrent: delete guest task GuestTask(0) -2026-08-10T15:45:16.303676Z TRACE wasmtime::runtime::component::concurrent: enter sync call RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(2) } -2026-08-10T15:45:16.303678Z TRACE wasmtime::runtime::component::concurrent: exit sync call RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(2) } -2026-08-10T15:45:16.303679Z TRACE wasmtime::runtime::component::concurrent: ready to delete? true (threads_completed: true, has_sync_result: false, pending_completion_event: false, host_future_state: NotApplicable) -2026-08-10T15:45:16.303680Z TRACE wasmtime::runtime::component::concurrent: delete guest task GuestTask(0) -2026-08-10T15:45:16.303683Z TRACE wasmtime::runtime::component::concurrent: enter sync call RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(2) } -2026-08-10T15:45:16.303684Z TRACE wasmtime::runtime::component::concurrent: exit sync call RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(2) } -2026-08-10T15:45:16.303685Z TRACE wasmtime::runtime::component::concurrent: ready to delete? true (threads_completed: true, has_sync_result: false, pending_completion_event: false, host_future_state: NotApplicable) -2026-08-10T15:45:16.303686Z TRACE wasmtime::runtime::component::concurrent: delete guest task GuestTask(0) -2026-08-10T15:45:16.303692Z TRACE wasmtime::runtime::component::concurrent: queueing call QualifiedThreadId(0, 2) -2026-08-10T15:45:16.303693Z TRACE wasmtime::runtime::component::concurrent: push high priority: GuestCall(RuntimeComponentInstanceIndex(2), GuestCall { thread: QualifiedThreadId(0, 2), kind: StartImplicit }) -2026-08-10T15:45:16.303698Z TRACE wasmtime::runtime::component::concurrent: handle work item GuestCall(RuntimeComponentInstanceIndex(2), GuestCall { thread: QualifiedThreadId(0, 2), kind: StartImplicit }) -2026-08-10T15:45:16.303699Z TRACE wasmtime::runtime::component::concurrent: call GuestCall { thread: QualifiedThreadId(0, 2), kind: StartImplicit } ready? true (do_not_enter: false; backpressure: 0) -2026-08-10T15:45:16.303705Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: save current thread None -2026-08-10T15:45:16.303708Z TRACE wasmtime::runtime::component::concurrent: sync/async-stackful call: replaced None with QualifiedThreadId(0, 2) as current thread -2026-08-10T15:45:16.303709Z TRACE wasmtime::runtime::component::concurrent: enter RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(2) } -2026-08-10T15:45:16.303716Z TRACE wasmtime::runtime::component::concurrent: pushed QualifiedThreadId(3, 5) as current thread; old thread was QualifiedThreadId(0, 2) -2026-08-10T15:45:16.303719Z TRACE wasmtime::runtime::component::concurrent: push high priority: GuestCall(RuntimeComponentInstanceIndex(1), GuestCall { thread: QualifiedThreadId(3, 5), kind: StartImplicit }) -2026-08-10T15:45:16.303720Z TRACE wasmtime::runtime::component::concurrent: waitable GuestTask(3) join set Some(WaitableSet(1)) -2026-08-10T15:45:16.303722Z TRACE wasmtime::runtime::component::concurrent: suspend fiber: Waiting { set: WaitableSet(1), thread: QualifiedThreadId(0, 2), skip_may_block_check: false } -2026-08-10T15:45:16.303724Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: restore current thread None -2026-08-10T15:45:16.303724Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: suspend reason Some(Waiting { set: WaitableSet(1), thread: QualifiedThreadId(0, 2), skip_may_block_check: false }) -2026-08-10T15:45:16.303727Z TRACE wasmtime::runtime::component::concurrent: handle work item GuestCall(RuntimeComponentInstanceIndex(1), GuestCall { thread: QualifiedThreadId(3, 5), kind: StartImplicit }) -2026-08-10T15:45:16.303728Z TRACE wasmtime::runtime::component::concurrent: call GuestCall { thread: QualifiedThreadId(3, 5), kind: StartImplicit } ready? true (do_not_enter: false; backpressure: 0) -2026-08-10T15:45:16.303732Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: save current thread None -2026-08-10T15:45:16.303733Z TRACE wasmtime::runtime::component::concurrent: stackless call: replaced None with QualifiedThreadId(3, 5) as current thread -2026-08-10T15:45:16.303734Z TRACE wasmtime::runtime::component::concurrent: enter RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(1) } -2026-08-10T15:45:16.303735Z TRACE wasmtime::runtime::component::concurrent: set event for GuestTask(3): Some(Subtask { status: Started }) -2026-08-10T15:45:16.303738Z TRACE wasmtime::runtime::component::concurrent: push high priority: ResumeFiber -2026-08-10T15:45:16.303740Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: new transmit: state TransmitState(6); write TransmitHandle(7); read TransmitHandle(8) -2026-08-10T15:45:16.303741Z TRACE wasmtime::runtime::component::concurrent: task.return for QualifiedThreadId(3, 5) -2026-08-10T15:45:16.303744Z TRACE wasmtime::runtime::component::concurrent: set event for GuestTask(3): Some(Subtask { status: Returned }) -2026-08-10T15:45:16.303747Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_write 4 to TransmitHandle(7) (handle 2; state TransmitState(6)); Open -2026-08-10T15:45:16.303749Z TRACE wasmtime::runtime::component::concurrent: waitable TransmitHandle(7) join set Some(WaitableSet(4)) -2026-08-10T15:45:16.303749Z TRACE wasmtime::runtime::component::concurrent: suspend fiber: Waiting { set: WaitableSet(4), thread: QualifiedThreadId(3, 5), skip_may_block_check: false } -2026-08-10T15:45:16.303751Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: restore current thread None -2026-08-10T15:45:16.303751Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: suspend reason Some(Waiting { set: WaitableSet(4), thread: QualifiedThreadId(3, 5), skip_may_block_check: false }) -2026-08-10T15:45:16.303752Z TRACE wasmtime::runtime::component::concurrent: handle work item ResumeFiber -2026-08-10T15:45:16.303753Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: save current thread None -2026-08-10T15:45:16.303754Z TRACE wasmtime::runtime::component::concurrent: taking event for GuestTask(3) -2026-08-10T15:45:16.303754Z TRACE wasmtime::runtime::component::concurrent: status Returned for GuestTask(3) -2026-08-10T15:45:16.303755Z TRACE wasmtime::runtime::component::concurrent: waitable GuestTask(3) join set None -2026-08-10T15:45:16.303756Z TRACE wasmtime::runtime::component::concurrent: popped current thread QualifiedThreadId(3, 5); new thread is QualifiedThreadId(0, 2) -2026-08-10T15:45:16.303758Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_read 4 from TransmitHandle(8) (handle 1; state TransmitState(6)); GuestReady -2026-08-10T15:45:16.303761Z TRACE wasmtime::runtime::component::concurrent: set event for TransmitHandle(7): Some(StreamWrite { code: Completed(4), pending: Some((TypeStreamTableIndex(0), 2)) }) -2026-08-10T15:45:16.303763Z TRACE wasmtime::runtime::component::concurrent: push high priority: ResumeFiber -2026-08-10T15:45:16.303764Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_read result for TransmitHandle(8) (handle 1; state TransmitState(6)): Completed(4) -2026-08-10T15:45:16.303766Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_drop_readable: drop reader TransmitHandle(8) -2026-08-10T15:45:16.303767Z TRACE wasmtime::runtime::component::concurrent: waitable TransmitHandle(8) join set None -2026-08-10T15:45:16.303767Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: host_drop_reader state TransmitState(6); read state Open write state Open -2026-08-10T15:45:16.303768Z TRACE wasmtime::runtime::component::concurrent: set event for TransmitHandle(7): Some(StreamWrite { code: Dropped(4), pending: Some((TypeStreamTableIndex(0), 2)) }) -2026-08-10T15:45:16.303770Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: new transmit: state TransmitState(9); write TransmitHandle(10); read TransmitHandle(11) -2026-08-10T15:45:16.303771Z TRACE wasmtime::runtime::component::concurrent: pushed QualifiedThreadId(12, 14) as current thread; old thread was QualifiedThreadId(0, 2) -2026-08-10T15:45:16.303773Z TRACE wasmtime::runtime::component::concurrent: push high priority: GuestCall(RuntimeComponentInstanceIndex(1), GuestCall { thread: QualifiedThreadId(12, 14), kind: StartImplicit }) -2026-08-10T15:45:16.303774Z TRACE wasmtime::runtime::component::concurrent: waitable GuestTask(12) join set Some(WaitableSet(1)) -2026-08-10T15:45:16.303775Z TRACE wasmtime::runtime::component::concurrent: suspend fiber: Waiting { set: WaitableSet(1), thread: QualifiedThreadId(0, 2), skip_may_block_check: true } -2026-08-10T15:45:16.303776Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: restore current thread None -2026-08-10T15:45:16.303776Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: suspend reason Some(Waiting { set: WaitableSet(1), thread: QualifiedThreadId(0, 2), skip_may_block_check: true }) -2026-08-10T15:45:16.303777Z TRACE wasmtime::runtime::component::concurrent: handle work item ResumeFiber -2026-08-10T15:45:16.303778Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: save current thread None -2026-08-10T15:45:16.303779Z TRACE wasmtime::runtime::component::concurrent: waitable TransmitHandle(7) join set None -2026-08-10T15:45:16.303780Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_write result for TransmitHandle(7) (handle 2; state TransmitState(6)): Dropped(4) -2026-08-10T15:45:16.303781Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_drop_writable: drop writer TransmitHandle(7) -2026-08-10T15:45:16.303781Z TRACE wasmtime::runtime::component::concurrent: waitable TransmitHandle(7) join set None -2026-08-10T15:45:16.303782Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: host_drop_writer state TransmitState(6); read state Dropped writer state Open -2026-08-10T15:45:16.303783Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: host_drop_writer delete TransmitState(6) -2026-08-10T15:45:16.303784Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: delete transmit: state TransmitState(6); write TransmitHandle(7); read TransmitHandle(8) -2026-08-10T15:45:16.303785Z TRACE wasmtime::runtime::component::concurrent: exit RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(1) } -2026-08-10T15:45:16.303786Z TRACE wasmtime::runtime::component::concurrent: stackless call: restored None as current thread -2026-08-10T15:45:16.303787Z TRACE wasmtime::runtime::component::concurrent: received callback code from QualifiedThreadId(3, 5): 0 (set: 0) -2026-08-10T15:45:16.303787Z TRACE wasmtime::runtime::component::concurrent: implicit thread QualifiedThreadId(3, 5) completed -2026-08-10T15:45:16.303789Z TRACE wasmtime::runtime::component::concurrent: ready to delete? true (threads_completed: true, has_sync_result: false, pending_completion_event: false, host_future_state: NotApplicable) -2026-08-10T15:45:16.303790Z TRACE wasmtime::runtime::component::concurrent: delete guest task GuestTask(3) -2026-08-10T15:45:16.303790Z TRACE wasmtime::runtime::component::concurrent: suspend fiber: NeedWork -2026-08-10T15:45:16.303791Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: restore current thread None -2026-08-10T15:45:16.303792Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: suspend reason Some(NeedWork) -2026-08-10T15:45:16.303793Z TRACE wasmtime::runtime::component::concurrent: handle work item GuestCall(RuntimeComponentInstanceIndex(1), GuestCall { thread: QualifiedThreadId(12, 14), kind: StartImplicit }) -2026-08-10T15:45:16.303793Z TRACE wasmtime::runtime::component::concurrent: call GuestCall { thread: QualifiedThreadId(12, 14), kind: StartImplicit } ready? true (do_not_enter: false; backpressure: 0) -2026-08-10T15:45:16.303794Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: save current thread None -2026-08-10T15:45:16.303795Z TRACE wasmtime::runtime::component::concurrent: stackless call: replaced None with QualifiedThreadId(12, 14) as current thread -2026-08-10T15:45:16.303796Z TRACE wasmtime::runtime::component::concurrent: enter RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(1) } -2026-08-10T15:45:16.303797Z TRACE wasmtime::runtime::component::concurrent: set event for GuestTask(12): Some(Subtask { status: Started }) -2026-08-10T15:45:16.303798Z TRACE wasmtime::runtime::component::concurrent: push high priority: ResumeFiber -2026-08-10T15:45:16.303799Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_read 4 from TransmitHandle(11) (handle 2; state TransmitState(9)); Open -2026-08-10T15:45:16.303799Z TRACE wasmtime::runtime::component::concurrent: waitable TransmitHandle(11) join set Some(WaitableSet(13)) -2026-08-10T15:45:16.303800Z TRACE wasmtime::runtime::component::concurrent: suspend fiber: Waiting { set: WaitableSet(13), thread: QualifiedThreadId(12, 14), skip_may_block_check: false } -2026-08-10T15:45:16.303801Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: restore current thread None -2026-08-10T15:45:16.303802Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: suspend reason Some(Waiting { set: WaitableSet(13), thread: QualifiedThreadId(12, 14), skip_may_block_check: false }) -2026-08-10T15:45:16.303803Z TRACE wasmtime::runtime::component::concurrent: handle work item ResumeFiber -2026-08-10T15:45:16.303803Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: save current thread None -2026-08-10T15:45:16.303804Z TRACE wasmtime::runtime::component::concurrent: taking event for GuestTask(12) -2026-08-10T15:45:16.303805Z TRACE wasmtime::runtime::component::concurrent: status Started for GuestTask(12) -2026-08-10T15:45:16.303805Z TRACE wasmtime::runtime::component::concurrent: waitable GuestTask(12) join set None -2026-08-10T15:45:16.303806Z TRACE wasmtime::runtime::component::concurrent: popped current thread QualifiedThreadId(12, 14); new thread is QualifiedThreadId(0, 2) -2026-08-10T15:45:16.303807Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_write 4 to TransmitHandle(10) (handle 2; state TransmitState(9)); GuestReady -2026-08-10T15:45:16.303808Z TRACE wasmtime::runtime::component::concurrent: set event for TransmitHandle(11): Some(StreamRead { code: Completed(4), pending: Some((TypeStreamTableIndex(0), 2)) }) -2026-08-10T15:45:16.303809Z TRACE wasmtime::runtime::component::concurrent: push high priority: ResumeFiber -2026-08-10T15:45:16.303810Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_write result for TransmitHandle(10) (handle 2; state TransmitState(9)): Completed(4) -2026-08-10T15:45:16.303811Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_drop_writable: drop writer TransmitHandle(10) -2026-08-10T15:45:16.303812Z TRACE wasmtime::runtime::component::concurrent: waitable TransmitHandle(10) join set None -2026-08-10T15:45:16.303813Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: host_drop_writer state TransmitState(9); read state Open writer state Open -2026-08-10T15:45:16.303813Z TRACE wasmtime::runtime::component::concurrent: set event for TransmitHandle(11): Some(StreamRead { code: Dropped(4), pending: Some((TypeStreamTableIndex(0), 2)) }) -2026-08-10T15:45:16.303815Z TRACE wasmtime::runtime::component::concurrent: new waitable set WaitableSet(3) (handle 2) -2026-08-10T15:45:16.303816Z TRACE wasmtime::runtime::component::concurrent: waitable GuestTask(12) (handle 1) join set Some(WaitableSet(3)) (handle 2) -2026-08-10T15:45:16.303817Z TRACE wasmtime::runtime::component::concurrent: waitable GuestTask(12) join set Some(WaitableSet(3)) -2026-08-10T15:45:16.303818Z TRACE wasmtime::runtime::component::concurrent: waitable check for QualifiedThreadId(0, 2); set WaitableSet(3) -2026-08-10T15:45:16.303819Z TRACE wasmtime::runtime::component::concurrent: suspend fiber: Waiting { set: WaitableSet(3), thread: QualifiedThreadId(0, 2), skip_may_block_check: false } -2026-08-10T15:45:16.303819Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: restore current thread None -2026-08-10T15:45:16.303820Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: suspend reason Some(Waiting { set: WaitableSet(3), thread: QualifiedThreadId(0, 2), skip_may_block_check: false }) -2026-08-10T15:45:16.303821Z TRACE wasmtime::runtime::component::concurrent: handle work item ResumeFiber -2026-08-10T15:45:16.303822Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: save current thread None -2026-08-10T15:45:16.303822Z TRACE wasmtime::runtime::component::concurrent: waitable TransmitHandle(11) join set None -2026-08-10T15:45:16.303823Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_read result for TransmitHandle(11) (handle 2; state TransmitState(9)): Dropped(4) -2026-08-10T15:45:16.303824Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: guest_drop_readable: drop reader TransmitHandle(11) -2026-08-10T15:45:16.303825Z TRACE wasmtime::runtime::component::concurrent: waitable TransmitHandle(11) join set None -2026-08-10T15:45:16.303826Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: host_drop_reader state TransmitState(9); read state Open write state Dropped -2026-08-10T15:45:16.303826Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: host_drop_reader delete TransmitState(9) -2026-08-10T15:45:16.303827Z TRACE wasmtime::runtime::component::concurrent::futures_and_streams: delete transmit: state TransmitState(9); write TransmitHandle(10); read TransmitHandle(11) -2026-08-10T15:45:16.303828Z TRACE wasmtime::runtime::component::concurrent: task.return for QualifiedThreadId(12, 14) -2026-08-10T15:45:16.303829Z TRACE wasmtime::runtime::component::concurrent: set event for GuestTask(12): Some(Subtask { status: Returned }) -2026-08-10T15:45:16.303830Z TRACE wasmtime::runtime::component::concurrent: push high priority: ResumeFiber -2026-08-10T15:45:16.303830Z TRACE wasmtime::runtime::component::concurrent: exit RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(1) } -2026-08-10T15:45:16.303831Z TRACE wasmtime::runtime::component::concurrent: stackless call: restored None as current thread -2026-08-10T15:45:16.303832Z TRACE wasmtime::runtime::component::concurrent: received callback code from QualifiedThreadId(12, 14): 0 (set: 0) -2026-08-10T15:45:16.303832Z TRACE wasmtime::runtime::component::concurrent: implicit thread QualifiedThreadId(12, 14) completed -2026-08-10T15:45:16.303833Z TRACE wasmtime::runtime::component::concurrent: ready to delete? false (threads_completed: true, has_sync_result: false, pending_completion_event: true, host_future_state: NotApplicable) -2026-08-10T15:45:16.303834Z TRACE wasmtime::runtime::component::concurrent: suspend fiber: NeedWork -2026-08-10T15:45:16.303838Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: restore current thread None -2026-08-10T15:45:16.303839Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: suspend reason Some(NeedWork) -2026-08-10T15:45:16.303840Z TRACE wasmtime::runtime::component::concurrent: handle work item ResumeFiber -2026-08-10T15:45:16.303840Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: save current thread None -2026-08-10T15:45:16.303841Z TRACE wasmtime::runtime::component::concurrent: waitable check for QualifiedThreadId(0, 2); set WaitableSet(3), part two -2026-08-10T15:45:16.303842Z TRACE wasmtime::runtime::component::concurrent: deliver event Subtask { status: Returned } to GuestTask(0) for GuestTask(12) (handle 1); set WaitableSet(3) -2026-08-10T15:45:16.303843Z TRACE wasmtime::runtime::component::concurrent: waitable GuestTask(12) (handle 1) join set None (handle 0) -2026-08-10T15:45:16.303844Z TRACE wasmtime::runtime::component::concurrent: waitable GuestTask(12) join set None -2026-08-10T15:45:16.303844Z TRACE wasmtime::runtime::component::concurrent: ready to delete? true (threads_completed: true, has_sync_result: false, pending_completion_event: false, host_future_state: NotApplicable) -2026-08-10T15:45:16.303845Z TRACE wasmtime::runtime::component::concurrent: delete guest task GuestTask(12) -2026-08-10T15:45:16.303846Z TRACE wasmtime::runtime::component::concurrent: subtask_drop GuestTask(12) (handle 1) -2026-08-10T15:45:16.303847Z TRACE wasmtime::runtime::component::concurrent: exit RuntimeInstance { instance: ComponentInstanceId(0), index: RuntimeComponentInstanceIndex(2) } -2026-08-10T15:45:16.303849Z TRACE wasmtime::runtime::component::concurrent: ready to delete? true (threads_completed: true, has_sync_result: false, pending_completion_event: false, host_future_state: NotApplicable) -2026-08-10T15:45:16.303850Z TRACE wasmtime::runtime::component::concurrent: delete guest task GuestTask(0) -2026-08-10T15:45:16.303850Z TRACE wasmtime::runtime::component::concurrent: suspend fiber: NeedWork -2026-08-10T15:45:16.303851Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: restore current thread None -2026-08-10T15:45:16.303851Z TRACE wasmtime::runtime::component::concurrent: resume_fiber: suspend reason Some(NeedWork) diff --git a/exams/wasmtime-exclusivity/verify-cm4.sh b/exams/wasmtime-exclusivity/verify-cm4.sh deleted file mode 100755 index ba2fb06..0000000 --- a/exams/wasmtime-exclusivity/verify-cm4.sh +++ /dev/null @@ -1,56 +0,0 @@ -#!/usr/bin/env sh -# Reproduces the CM-4 filing evidence from the pristine submodule copies. -# See root-cause.md for the full analysis. The reference harness HANGS after -# any failing assertion (non-daemon threads) — every failing leg runs under -# `timeout` and is judged by its traceback text, not its exit code. -set -eu -cd "$(dirname "$0")/../.." -SRC=third_party/component-model/design/mvp/canonical-abi -ART=exams/wasmtime-exclusivity -WORK="$(mktemp -d)" -trap 'rm -rf "$WORK"' EXIT -mkdir -p "$WORK/design/mvp/canonical-abi" -cp "$SRC/definitions.py" "$SRC/run_tests.py" "$WORK/design/mvp/canonical-abi/" - -echo "== leg 0: stock suite sanity (pristine)" -( cd "$WORK/design/mvp/canonical-abi" && timeout 180 python3 run_tests.py | tail -1 | grep -qx "All tests passed" ) -echo " ok: All tests passed" - -echo "== leg 1: new test vs pristine definitions.py -> must fail at the gating assertion" -( cd "$WORK" && patch -p1 -s < "$OLDPWD/$ART/cm4-run-tests.patch" ) -set +e -( cd "$WORK/design/mvp/canonical-abi" && timeout 120 python3 run_tests.py > "$WORK/leg1.out" 2>&1 ) -set -e -grep -q "assert(poke_state == Subtask.State.RETURNED)" "$WORK/leg1.out" -echo " ok: failed at the sync-streams expectation (STARTING observed; traceback in leg1.out)" - -echo "== leg 2: new test with the resolution-scoped fix -> passes; the one conflicting stock test skipped" -( cd "$WORK" && patch -p1 -s < "$OLDPWD/$ART/cm4-reference-fix.patch" ) -python3 - "$WORK/design/mvp/canonical-abi/run_tests.py" <<'PYEOF' -import sys -p = sys.argv[1] -s = open(p).read() -call = "\ntest_callback_interleaving()\n" -assert s.count(call) == 1 -open(p, "w").write(s.replace(call, - "\n# test_callback_interleaving() # encodes the hold-semantics; see root-cause.md\n", 1)) -PYEOF -( cd "$WORK/design/mvp/canonical-abi" && timeout 180 python3 run_tests.py | tail -1 | grep -qx "All tests passed" ) -echo " ok: All tests passed (incl. test_resolved_task_gates_entry)" - -echo "== leg 3: the fix vs the FULL stock suite -> test_callback_interleaving's NONE window fails" -python3 - "$WORK/design/mvp/canonical-abi/run_tests.py" <<'PYEOF' -import sys -p = sys.argv[1] -s = open(p).read() -open(p, "w").write(s.replace( - "\n# test_callback_interleaving() # encodes the hold-semantics; see root-cause.md\n", - "\ntest_callback_interleaving()\n", 1)) -PYEOF -set +e -( cd "$WORK/design/mvp/canonical-abi" && timeout 120 python3 run_tests.py > "$WORK/leg3.out" 2>&1 ) -set -e -grep -q "assert(ret == EventCode.NONE)" "$WORK/leg3.out" -echo " ok: the progress-free poll window observed the admitted producer (traceback in leg3.out)" - -echo "ALL LEGS OK" diff --git a/exams/wasmtime-exclusivity/wasmtime-actual-semantics.md b/exams/wasmtime-exclusivity/wasmtime-actual-semantics.md deleted file mode 100644 index ae33284..0000000 --- a/exams/wasmtime-exclusivity/wasmtime-actual-semantics.md +++ /dev/null @@ -1,145 +0,0 @@ -# wasmtime's actual entry semantics: hold rule + deferred start (CM-4 correction) - -**2026-08-10, operator-prompted source/trace verification. This document -supersedes the wasmtime characterization in `RESULTS.md`, `root-cause.md`, -the pre-correction `spec-amendment.md`, and CM-4's original evidence -section.** The earlier claim — "wasmtime's gate (`do_not_enter`) is scoped -to the sync-call span, ending at resolution" — is **false**. wasmtime holds -its entry gate for the whole core invocation, exactly like the reference's -`exclusive_thread`; its `sync-streams.wast` pass comes from *when the entry -decision is made*, not from gate lifetime. - -Sources: `crates/wasmtime/src/runtime/component/concurrent.rs` at main -(fetched 2026-08-10; last commit touching it b6cb7446f, 2026-07-24; 6002 -lines) and `v47.0.3` (5949 lines) — the bracket structure is identical in -both; 47.0.3 line numbers in [brackets]. - -## The gate, from source - -- `ConcurrentInstanceState.do_not_enter: bool` — :5227 [:5192]. Exactly - **two** assignment sites in the whole crate (submodules `abort.rs`, - `func.rs`, `futures_and_streams.rs`, etc. grep-clean): - `enter_instance` :2004–2010 [:1998] sets it, `exit_instance` - :2014–2021 [:2008] clears it and re-evaluates `pending` - (`partition_pending`). -- Every caller is a **core-invocation bracket**: - - callback-lift initial invocation :2652/:2662 [:2647/:2657] — enter - before `call(store)`, exit after the invocation returns a callback - code; - - sync-lift invocation :2696–2698/:2714 [:2691/:2709], with the - stackful exemption (`if !async_`); - - each callback invocation (DeliverEvent) :942/:960 [:937/:955]. -- `task_return` :3329–3378 → `task_complete` :3411+: **no gate - interaction**. Resolution does not touch `do_not_enter`. -- The blocking path used by sync builtins (`wait_for_event` :2199–2213, - reached from `futures_and_streams.rs` `wait_for_write`/`wait_for_read` - :3978/:4069) suspends the guest thread's fiber **with the bracket - open**. A resolved task parked mid-frame in a sync builtin therefore - *still gates its instance*. -- Entry readiness (`GuestCall::is_ready` :756–772): - `StartImplicit` requires `!(do_not_enter || backpressure > 0)`; - `DeliverEvent` requires `!do_not_enter`. Both kinds of same-instance - progress — new entries *and* event deliveries to parked-between- - invocations callback tasks — are deferred while any invocation is - mid-frame. - -## The scheduling, from source - -The reference's `canon_lower` runs the callee **eagerly, inline**: the -entry check happens at the call instant and the packed STARTING/STARTED -status reflects that instant. wasmtime instead **defers**: - -- `start_call` :3040–3160: a guest→guest call queues the callee's - `StartImplicit` as a high-priority work item and the **caller suspends - until the first subtask status event** (async-lowered callers take - whatever that first status is, :3138–3153; sync-lowered callers loop - until `Returned`). -- Work-item handling :1497–1522: when a queued call is popped and - `is_ready` is false, a `Status::Starting` event is delivered to the - caller (`starting_sent`) and the call parks in `pending` until an - `exit_instance` → `partition_pending` makes it ready. - -So the status an async caller observes is decided **after the executor has -drained work queued ahead of the call** — in particular, after any -ready-to-resume gate holder has run. - -## Trace proof (`trace-sync-streams-wasmtime-dev.log`) - -`wasmtime 49.0.0 (a276ccbe1 2026-08-10)` dev release, aarch64, flags -`-W component-model-async=y -W component-model-more-async-builtins=y`, -`WASMTIME_LOG='wasmtime::runtime::component::concurrent=trace'`, on the -pristine `test/async/sync-streams.wast` (submodule 73b7ad5). Key lines -(thread `(3,5)` = `$C.get`, `(12,14)` = `$C.set`, `(0,2)` = `$D.run`; -instance index 1 = `$C`): - -1. `enter RuntimeInstance{index:1}` — get's initial invocation; gate set. -2. `task.return for (3,5)` — get resolves. -3. `suspend fiber: Waiting{set:4, thread:(3,5)}` — get parks mid-frame in - the sync `stream.write`. Gate **still set**. -4. D's `guest_read` rendezvouses the write → `push high priority: - ResumeFiber` (get) — queued *before* the `set` call exists. -5. D lowers `set` → `push high priority: GuestCall((12,14), - StartImplicit)`; D suspends awaiting the first status. -6. FIFO: get's fiber resumes, its write returns `Dropped(4)`, the - invocation returns EXIT → **`exit RuntimeInstance{index:1}`** → - `delete guest task GuestTask(3)`. -7. Only now: `call GuestCall{(12,14), StartImplicit} ready? true - (do_not_enter: false)` → `enter` → `Subtask{status: Started}` → D - resumes and sees STARTED. - -At the moment `set` was admitted, `get` was not parked mid-frame — it was -already exited and deleted. No same-instance execution ever overlapped -get's parked span. - -## The corrected semantic model - -wasmtime = **hold rule** (gate lifetime = the core invocation, identical -to `definitions.py`'s `exclusive_thread`) **+ deferred entry decision** -(caller learns STARTING only after runnable work queued ahead of the call -— including ready gate holders — has been exhausted) **+ FIFO work -queue**. - -Consequences: - -- `test/async/sync-streams.wast:145` does **not** pin release-at- - resolution. It is satisfied by (a) wasmtime's hold + deferred start, and - (b) release-at-resolution + eager start (deltic today; the - `cm4-reference-fix.patch` experiment). It rules out only the - reference's exact combination: hold + **eager** start. -- The reference's own `test_callback_interleaving` (both hold-encoding - sites: the :990–995 NONE window and the :1009–1011 STARTING tail) is - **consistent with wasmtime** — under hold + deferred start it passes - unchanged. The "second contradiction" reported in `root-cause.md` is a - property of the release-rule fix, not of wasmtime alignment. -- The exam's `test_resolved_task_gates_entry` (cm4-run-tests.patch) - encodes the **release rule**, not wasmtime: its shared-state assertions - (`poke_saw == 1`, `pump_observed == 2`) require admission *during* the - resolved task's park. Under wasmtime's model poke is deferred until - pump's invocation exits (STARTING at the lower; different interleaving - values). -- The IROH-1 collision window (same-instance execution during a resolved - task's mid-frame block) is **unreachable under wasmtime** for - needs-exclusive shapes — `DeliverEvent` is `do_not_enter`-gated too — - and reachable under deltic's release rule. wasmtime-green vs deltic-trap - is a semantics difference, not host timing. -- wasmtime's own pass is **FIFO-order-dependent**: had the executor popped - `set`'s `StartImplicit` before get's queued `ResumeFiber`, the caller - would have seen STARTING and the guest would trap. Any reformulation for - the reference (or deltic, which must stay green under - `DELTIC_SCHED_SEED` shuffles) should therefore prefer the order-robust - statement: *an async-lowered call reports STARTING only if the callee - is still unstarted after the instance's runnable work is exhausted* - (drain-to-quiescence, not pop-one). - -## What survives from the original exam - -- The reference↔wast-suite contradiction (CM-4 proper): pristine - `definitions.py` deterministically traps on sync-streams.wast:146 - (STARTING where the suite demands STARTED). Still true; `verify-cm4.sh` - legs 0–1 stand. -- The determinism measurement (50/50) and its conclusion — the wasmtime - pass is deterministic. Only the *mechanism attribution* ("the gating - rule alone decides") was wrong: determinism additionally rests on FIFO - order and on get's readiness preceding the `set` lower. -- The CI-provenance analysis (spec repo CI runs only `run_tests.py`; the - wast corpus's runner is wasmtime; no cross-check) — unchanged. diff --git a/harness/src/xfail.ts b/harness/src/xfail.ts index 988def2..2870991 100644 --- a/harness/src/xfail.ts +++ b/harness/src/xfail.ts @@ -320,10 +320,13 @@ export const XFAIL: XfailEntry[] = [ // (a resolved producer blocked mid-sync-write keeps gating), and the // async-lowered call's initial status is decided only after the callee // instance's runnable work has been drained to quiescence — by which time - // the producer has exited and the next task reports STARTED. See - // exams/wasmtime-exclusivity/wasmtime-actual-semantics.md; the former - // release-at-BLOCK divergence is gone. (Before the M2 jspi flip this file - // was xfailed outright.) + // the producer has exited and the next task reports STARTED. Adjudicated + // 2026-08-10 (issue #43): the test's hard STARTED assertion is + // schedule-dependent — an upstream test defect overfitting wasmtime's + // deferred-entry policy (pristine definitions.py answers STARTING) — + // and deltic's drain policy satisfies it as written under any seed. The + // former release-at-BLOCK divergence is gone. (Before the M2 jspi flip + // this file was xfailed outright.) // entry pruned. --- // --- async/trap-if-block-and-sync.json: see entries --- { diff --git a/runtime/src/intrinsics/fact_calls.ts b/runtime/src/intrinsics/fact_calls.ts index ef22534..96da478 100644 --- a/runtime/src/intrinsics/fact_calls.ts +++ b/runtime/src/intrinsics/fact_calls.ts @@ -856,10 +856,11 @@ export function createAsyncStartCall( // it is the reference's atomic run-to-first-block, reconstructed across // the engine's microtask hops. // - // THE DEFERRED ENTRY DECISION (issue #43; wasmtime's model, see - // exams/wasmtime-exclusivity/wasmtime-actual-semantics.md "The - // scheduling, from source"). The determinacy wait above is also where - // the initial *status* is decided, so it is where the deferral lives. + // THE DEFERRED ENTRY DECISION (issue #43; wasmtime's model — source + // walkthrough distilled on the issue, exam kit archived at + // 4f3351f:exams/wasmtime-exclusivity/). The determinacy wait above is + // also where the initial *status* is decided, so it is where the + // deferral lives. // // In wasmtime a guest->guest call queues the callee's `StartImplicit` // and the caller suspends until the first subtask status event @@ -870,7 +871,8 @@ export function createAsyncStartCall( // parked at `enter_implicit_thread`'s gate wait at this point; what // changes here is only WHEN the caller reads `subtask.state`. // - // Order-robust formulation (spec-amendment.md, chosen over wasmtime's + // Order-robust formulation (issue #43; a non-normative scheduler + // policy — entry-status timing is not normative — chosen over wasmtime's // FIFO-dependent one so the seeded-shuffle reruns stay green): while the // callee is still parked at the entry gate, the caller waits until the // callee instance's runnable work is exhausted diff --git a/runtime/src/jspi/bridge.ts b/runtime/src/jspi/bridge.ts index 6605b5d..5870302 100644 --- a/runtime/src/jspi/bridge.ts +++ b/runtime/src/jspi/bridge.ts @@ -543,10 +543,10 @@ export function blockCurrentActivation(input: { // gating its instance — identical to wasmtime, whose // `ConcurrentInstanceState.do_not_enter` is set/cleared only by // `enter_instance`/`exit_instance`, i.e. bracketed on the whole core - // invocation (exams/wasmtime-exclusivity/wasmtime-actual-semantics.md, - // "The gate, from source"). deltic's former release-at-BLOCK divergence - // was removed by issue #43; `test/async/sync-streams.wast` is now green - // via the DEFERRED ENTRY DECISION in intrinsics/fact_calls.ts + // invocation (source refs distilled on issue #43; exam kit archived at + // 4f3351f:exams/wasmtime-exclusivity/). deltic's former release-at-BLOCK + // divergence was removed by issue #43; `test/async/sync-streams.wast` is + // now green via the DEFERRED ENTRY DECISION in intrinsics/fact_calls.ts // (`createAsyncStartCall`), not via gate release. // // WHO is parking — read BEFORE anything below disturbs the ambient. This diff --git a/runtime/src/task/scheduler.ts b/runtime/src/task/scheduler.ts index 90abf23..719b875 100644 --- a/runtime/src/task/scheduler.ts +++ b/runtime/src/task/scheduler.ts @@ -764,12 +764,15 @@ export class Store { * `GuestCall(StartImplicit)` is popped, and if `is_ready` is false * (`do_not_enter || backpressure`) the caller is told STARTING * (concurrent.rs :1497-1522, :3040-3160). That formulation is FIFO-order - * dependent; deltic uses the order-robust one from - * `exams/wasmtime-exclusivity/spec-amendment.md`: *the call reports - * STARTING only if the callee is still unstarted after the instance's - * runnable work has been exhausted* — drain to quiescence, not pop-one. - * That is what keeps `sync-streams.wast` green under `DELTIC_SCHED_SEED` - * shuffles, which wasmtime's own rule would not be. + * dependent; deltic uses the order-robust restatement (issue #43): *the + * call reports STARTING only if the callee is still unstarted after the + * instance's runnable work has been exhausted* — drain to quiescence, not + * pop-one. That is what keeps `sync-streams.wast` green under + * `DELTIC_SCHED_SEED` shuffles, which wasmtime's own rule would not be. + * Adjudicated 2026-08-10 (issue #43): entry-status timing is NOT + * normative — this predicate implements a scheduler *policy*, picked so + * the suite's schedule-overfitted STARTED assertion holds under any + * seed; the hold-rule gate itself is the spec semantics. * * "Runnable work of `inst`" is, exhaustively: * diff --git a/runtime/tests/entry_deferral_test.ts b/runtime/tests/entry_deferral_test.ts index 1a3476f..e9afd88 100644 --- a/runtime/tests/entry_deferral_test.ts +++ b/runtime/tests/entry_deferral_test.ts @@ -1,8 +1,9 @@ // wasmtime-model entry semantics: the HOLD RULE + the DEFERRED ENTRY DECISION // (issue #43). // -// These tests pin the semantics established by -// `exams/wasmtime-exclusivity/wasmtime-actual-semantics.md`, which corrected +// These tests pin the semantics established by the CM-4 investigation +// (issue #43, where the evidence is distilled; the exam kit is archived at +// 4f3351f:exams/wasmtime-exclusivity/), which corrected // the earlier (false) belief that wasmtime releases its instance-entry gate // when a task resolves. It does not: // @@ -20,7 +21,7 @@ // suspends the caller until the first subtask status event, so the // executor first drains the work queued ahead of the call // (concurrent.rs :1497-1522, :3040-3160). deltic uses the order-robust -// restatement from `exams/wasmtime-exclusivity/spec-amendment.md` — *the +// restatement (issue #43) — *the // call reports STARTING only if the callee is still unstarted after the // instance's runnable work has been drained to quiescence* — because // wasmtime's own FIFO-dependent formulation would not survive @@ -28,13 +29,20 @@ // predicate; `createAsyncStartCall` (intrinsics/fact_calls.ts) is its // only consumer. // +// Adjudicated 2026-08-10 (issue #43): entry-status *timing* is not +// normative. The HOLD RULE is spec semantics; the deferred decision is +// deltic's scheduler policy, which makes the suite's schedule-overfitted +// `sync-streams.wast:145` STARTED assertion (an upstream test defect — +// pristine definitions.py answers STARTING) hold under any seed. +// // The shape exercised below is the pump/poke one: a callback-lifted "pump" // task resolves (`task.return`) and then parks mid-frame in a synchronous // read that is NOT ready, while a same-instance "poke" task tries to enter. // Expected, per the wasmtime model: poke is NOT admitted for the whole parked // span (STARTING at the lower), and starts only once the pump's invocation // exits. Note this is deliberately the OPPOSITE of the exam's -// `test_resolved_task_gates_entry` (cm4-run-tests.patch), which encoded +// `test_resolved_task_gates_entry` (cm4-run-tests.patch, archived at +// 4f3351f:exams/wasmtime-exclusivity/), which encoded // deltic's since-removed release-at-resolution rule. // // The no-interleaving assertion uses the shared-state discipline of the diff --git a/upstream-component-model-repo-findings.md b/upstream-component-model-repo-findings.md index 548c806..02cf6bd 100644 --- a/upstream-component-model-repo-findings.md +++ b/upstream-component-model-repo-findings.md @@ -180,164 +180,51 @@ In `cancel_copy`, when the pending event is a stream `COMPLETED`, deliver --- -## CM-4: async-lower entry timing — definitions.py decides STARTING eagerly where wasmtime defers - -**Status:** DRAFT — candidate upstream issue against `definitions.py`. -**Working assumption** (operator decision, 2026-08-10, **superseding the -2026-08-09 adoption of the release rule**): deltic proceeds on -**wasmtime's actual model — hold rule + deferred entry decision** — as -the expected upstream resolution. Gate lifetime: whole core invocation -(wasmtime and the reference already agree). Entry status: an -async-lowered call reports STARTING only if the callee is still unstarted -after the instance's runnable work is drained to quiescence. deltic's -former release-at-resolution rule was removed the same day — the runtime -now implements the corrected model -([#43](https://github.com/lann/deltic/issues/43); `Store.hasRunnableWork` -+ `createAsyncStartCall`, pinned by -`runtime/tests/entry_deferral_test.ts`). Flip-back trigger: -upstream adjudicating otherwise. -**Found:** 2026-08-08, during the JSPI flip (M2 exit; `sync-streams.wast:208` is the arbiter). -**Corrected:** 2026-08-10 — the original evidence section -mischaracterized wasmtime (claimed its gate ends at resolution; it does -not). Full corrected mechanism: -`exams/wasmtime-exclusivity/wasmtime-actual-semantics.md` (source line -refs for main and v47.0.3 + runtime trace). - -### Evidence (corrected 2026-08-10) - -- `definitions.py` gates instance entry on `exclusive_thread`, held for - the whole activation; `canon_lower`'s async path starts the callee - **eagerly, inline**, so the STARTING/STARTED status reflects the call - instant — while a resolved-but-ready holder is still parked, the caller - gets STARTING. -- wasmtime (main, identically v47.0.3) holds - `ConcurrentInstanceState.do_not_enter` for the **same lifetime** (each - core invocation: enter/exit_instance at concurrent.rs:2004–2021 - [v47: :1998/:2008], bracketing callback initial invocations - :2652/:2662, sync-lift invocations :2696/:2714 with the stackful - exemption, and each callback invocation :942/:960; `task_return` never - touches it). What differs is **timing**: `start_call` - (:3040–3160) queues the callee and suspends the caller until the first - status event; a gated callee yields a `Status::Starting` event and - parks pending (:1497–1522). Ready gate-holders queued ahead of the call - run to invocation exit first. Trace proof: - `exams/wasmtime-exclusivity/trace-sync-streams-wasmtime-dev.log` — - `$C.get` is resumed, exited, and *deleted* before `$C.set`'s readiness - is ever evaluated. -- The official suite (`sync-streams.wast:145`) therefore does **not** pin - release-at-resolution; it rules out exactly the reference's - hold+**eager** combination. It is satisfied by wasmtime's - hold+deferred, and also by release+eager (deltic's current rule). -- The reference's own unit tests (`test_callback_interleaving`, both - hold-encoding sites) are **consistent with wasmtime** — no unit-test - collateral in a wasmtime-aligned fix. - -### Why wasmtime looks right - -The reference reports "not started" about a state whose only obstacle is -a holder that is already unblocked and merely unscheduled — a scheduling -artifact surfaced as ABI-visible status. The deferred rule reports the -status of a settled state, preserves every existing unit-test -expectation, and keeps Invariant #3 (single-shadow-stack LIFO) airtight -with no carve-out, since no same-instance execution is ever admitted -while any invocation has live frames parked. (The previously-argued -"producer becomes an accidental instance-wide mutex" liveness cost of the -hold rule is real but bounded: the deferral ends at the producer's -invocation exit, and the flagship + suite accept it.) - -### Suggested change - -Defer `canon_lower`'s async-path entry decision: report STARTING only if -the callee remains unstarted after the instance's runnable work is -exhausted (drain to quiescence — order-robust, unlike wasmtime's own -FIFO-dependent formulation). `exclusive_thread` lifetime, the callback -event loop, and the entire unit-test corpus stay untouched. Sketch: -`exams/wasmtime-exclusivity/spec-amendment.md`. - -### 2026-08-09 review: not a recent-spec-change lag, and structurally invisible upstream - -Reviewed on operator prompt (3-month spec-history window + wasmtime CI -provenance + determinism check; full transcripts in -`exams/wasmtime-exclusivity/RESULTS.md`): - -- **Both sides are ancient.** The exclusivity model (`exclusive`, then - `exclusive_thread`) with the hold-for-the-activation lifetime dates to - ≥ 2025-08-20 (#553); `sync-streams.wast`'s contrary assertion dates to - the file's birth, 2025-09-05 (9b5aa62). The in-window commits — #650 - (2026-05-21, `exclusive: Task` → `exclusive_thread: Thread` + the - entering-set reentrance definition) and #656 (2026-05-29, cooperative - thread built-ins) — refined granularity without touching the release - points. This is a ~11-month-old inconsistency, not wasmtime lagging a - recent change (nor the reverse). -- **No CI cross-checks the two.** The spec repo's CI runs only - `run_tests.py` (definitions.py's own unit tests); the wast suite is - never executed against the reference. wasmtime runs the suite via a - `tests/component-model` submodule (currently e8d8005, bumped 2026-07-24) - with an explicit exception ledger for known misalignments (e.g. - `post-return.wast` pending #680 alignment) — `sync-streams.wast` is not - on it. So: wasmtime CI green, spec CI green, and the reference↔suite - contradiction has no detector by construction. -- **wasmtime's pass is deterministic — but the 2026-08-09 mechanism - attribution was wrong (corrected 2026-08-10).** Empirically: 50/50 - identical passes under wasmtime 49.0.0-dev (3ebfbe5af, 2026-08-07) with - `-W component-model-async=y -W component-model-more-async-builtins=y`; - reconfirmed with trace on a276ccbe1 (2026-08-10). Mechanically the - determinism rests on **deferred entry + FIFO order + the producer's - readiness preceding the `set` lower** (the ready `$C.get` runs to exit - before `$C.set`'s readiness is evaluated), *not* on a gating rule that - ends at resolution — see - `exams/wasmtime-exclusivity/wasmtime-actual-semantics.md`. - (Vintage note: the wasmtime 47.0.1 *release* CLI cannot even parse the - current suite text — its bundled wast crate predates the 2026-07 #655 - syntax adherence pass — so any 47-era corroboration must use the crate - APIs or a dev build.) - -Net: the filing should present this as an internal spec-repo inconsistency -(reference vs its own test corpus) that only external implementations can -currently observe, propose the **entry-timing fix** (defer the async-lower -status decision; gate lifetime untouched), and suggest the structural fix -— run the wast suite against the reference (or at least flag -reference-affecting suite assertions) in the spec repo's own CI. - -### Filing artifacts (2026-08-09 set, re-scoped by the 2026-08-10 correction) - -- **`exams/wasmtime-exclusivity/wasmtime-actual-semantics.md`** + - **`trace-sync-streams-wasmtime-dev.log`** — THE wasmtime-side evidence: - gate lifetime from source (main + v47.0.3 line refs, all - `do_not_enter` sites), deferred-entry mechanism, and the runtime trace - showing `$C.get` exits before `$C.set` is admitted. -- **`exams/wasmtime-exclusivity/cm4-run-tests.patch`** — adds - `test_resolved_task_gates_entry` to the reference's own `run_tests.py`. - Still valid as the reference-side contradiction demo (fails against - pristine `definitions.py` at the STARTING assertion). **Caveat:** its - shared-state assertions (`poke_saw == 1`, `pump_observed == 2`) encode - the *release rule*, not wasmtime's semantics — under hold+deferred the - interloper is admitted only after the holder exits. A wasmtime-aligned - filing should trim it to the STARTING/RETURNED skeleton or re-derive - expectations. -- **`exams/wasmtime-exclusivity/cm4-reference-fix.patch`** — the - release-rule experiment (3 hunks). **No longer the proposed fix**; kept - as the demonstration that release+eager also satisfies the wast corpus - and of its cost (breaks `test_callback_interleaving`, whose second - progress-free window and STARTING tail encode hold semantics — full - trace in `root-cause.md`). The wasmtime-aligned fix touches - `canon_lower` timing instead and has zero unit-test collateral. -- **`exams/wasmtime-exclusivity/verify-cm4.sh`** — legs 0–1 (stock pass; - new test fails pristine) remain the contradiction repro; legs 2–3 - document the release-rule experiment's behavior. Note the reference - harness hangs after any failing assertion (non-daemon threads) — run - under `timeout`, judge by traceback. -- **`exams/wasmtime-exclusivity/spec-amendment.md`** — the amendment - sketch, rewritten 2026-08-10 for the corrected model: deferred entry - decision, no gate-lifetime change, no Invariant #3 carve-out, no - unit-test rewrites; order-robust drain-to-quiescence formulation. - -Net-net for the filing: `definitions.py` and the repo's wast suite -disagree about **when the async-lower entry status is decided** (eager -instant vs after-drain); wasmtime implements (and its CI deterministically -validates) the deferred side with an invocation-lifetime gate identical to -the reference's; nothing upstream validates the reference against the wast -corpus; the artifacts above make all of this reproducible. +## CM-4: `sync-streams.wast:145` overfits wasmtime's scheduler — entry-status timing is not normative + +**Status:** ADJUDICATED (operator, 2026-08-10) — upstream **test defect**, +not a reference-semantics issue. Candidate filing against the test suite; +distilled finding + filing kit: +[deltic#43](https://github.com/lann/deltic/issues/43) (filing itself +tracked by [deltic#15](https://github.com/lann/deltic/issues/15)). +Archived evidence tree (mechanism docs, both experiment patches, trace, +verify script): `4f3351f:exams/wasmtime-exclusivity/`. +**Found:** 2026-08-08 (JSPI flip, M2 exit). **Mechanism corrected:** +2026-08-10 (#44 — the 08-09 analysis wrongly attributed a +release-at-resolution gate to wasmtime). **Runtime migrated:** hold gate + +drain-to-quiescence entry decision, PR #45. + +- Gate semantics — held for the whole core invocation, mid-frame parks + included — are agreed by `definitions.py`, wasmtime, and the + CanonicalABI.md prose alike; no semantics were ever in conflict. What + differs is a **scheduler policy**: *when* an async-lowered call's + STARTING/STARTED status is decided. The reference decides eagerly at + the call instant (STARTING in the wast scenario, under every schedule + it can produce); wasmtime defers until work queued ahead of the call + has drained (STARTED, deterministic under FIFO). Timing is not + normative: two conforming policies over identical gate semantics give + two different ABI-visible answers, so the hard STARTED assertion at + `sync-streams.wast:145` pins wasmtime's policy, not semantics — the + corpus was co-developed on wasmtime as its runner. +- Proposed upstream fix: make the assertion schedule-agnostic (accept + STARTING|STARTED; on STARTING, wait for the SUBTASK event, then assert + as today). Secondary, structural: run the wast corpus against the + reference in spec-repo CI — today `run_tests.py` is the only CI step, + so reference↔corpus contradictions have no detector by construction. +- Same class as NOTE-1 below (tests assuming a particular scheduler), + sharper instance: engine-policy overfit — the reference itself fails + the assertion deterministically. `definitions.py`, CanonicalABI.md, and + `test_callback_interleaving` all need **no change**; both previously + sketched amendments (release-at-resolution; deferred-entry + normativization) are withdrawn. +- deltic disposition: hold-lifetime gate = spec conformance; + drain-to-quiescence entry decision (`Store.hasRunnableWork`, sole + consumer `createAsyncStartCall`, pinned by + `runtime/tests/entry_deferral_test.ts`) = deliberate **non-normative + scheduler policy** — satisfies the suite as written and is order-robust + under `DELTIC_SCHED_SEED` shuffles, unlike wasmtime's FIFO-dependent + formulation. Legal under any upstream adjudication of the test; no + flip-back trigger. --- diff --git a/upstream-consumer-findings.md b/upstream-consumer-findings.md index f505304..a12e69f 100644 --- a/upstream-consumer-findings.md +++ b/upstream-consumer-findings.md @@ -39,8 +39,9 @@ the collision near-certain on any host that interleaves there. **The precise semantics (corrected 2026-08-10; see `upstream-component-model-repo-findings.md` CM-4 and -`exams/wasmtime-exclusivity/wasmtime-actual-semantics.md`):** the -collision window is **deltic-specific**, not spec-pinned. +[deltic#43](https://github.com/lann/deltic/issues/43); exam kit archived +at `4f3351f:exams/wasmtime-exclusivity/`):** the +collision window was **deltic-specific**, not spec-pinned. - *Before* `task.return`, a callback task's instance-entry gate holds across mid-frame blocks on every implementation surveyed (deltic, @@ -52,16 +53,18 @@ collision window is **deltic-specific**, not spec-pinned. the same way (`GuestCall::is_ready`, concurrent.rs:765). Under wasmtime the poller *cannot* be resumed inside the pump's parked signing window — the collision is **unreachable by semantics**, not by timing. - `definitions.py` agrees on the gate lifetime. **deltic today is the - outlier**: its release-at-resolution rule (2026-08-09 CM-4 working - assumption, since corrected) admits same-instance tasks during the - post-resolution parked span — that admitted window is where this trap - lives. The official suite (`sync-streams.wast`) pins neither rule; it - pins deferred entry *timing* (CM-4, corrected). + `definitions.py` agrees on the gate lifetime. **deltic was the + outlier** until #43 landed: its since-removed release-at-resolution + rule (the 2026-08-09 CM-4 working assumption) admitted same-instance + tasks during the post-resolution parked span — that admitted window is + where this trap lived. The official suite (`sync-streams.wast`) pins + neither gate rule; its STARTED assertion is schedule-dependent (CM-4, + adjudicated 2026-08-10: an upstream test defect overfitting wasmtime's + deferred-entry scheduler policy). The endpoint's pump does its `block_on(sign)` **after** `bind` resolved, -inside the window deltic's current rule admits, with the `RefCell` borrow -live. +inside the window deltic's since-removed rule admitted, with the +`RefCell` borrow live. **Why the wasmtime leg is green — a semantics guarantee, not timing luck (corrected 2026-08-10):** the previous revision of this entry @@ -70,8 +73,8 @@ corrected model predicts the opposite — under wasmtime the poller's timer event sits gated in `pending` until the pump's invocation exits, at any signing latency. (Falsifiable both ways: add ~1 ms to the wasmtime host's `sign`; the corrected model says it stays green.) Under -deltic the same window is open by our own rule, and the 5 ms poll -cadence lands in it ~90% of the time with a `crypto.subtle` signer. +pre-#43 deltic the same window was open by our own rule, and the 5 ms +poll cadence landed in it ~90% of the time with a `crypto.subtle` signer. **Disposition (2026-08-10, updated after deltic#43 landed):** deltic now implements wasmtime's hold + deferred-entry model