streams: direct-access byte edges for host stream<u8> ends (A21, #128) - #226
Merged
Conversation
wasmtime DirectSource/DirectDestination-shaped. StreamWriter.writeDirect / Stream.readDirect (and the low-level HostWritableEnd/HostReadableEnd forms) park a direct session: at every rendezvous with a peer operation of nonzero capacity the callback runs exactly once, synchronously, inside the rendezvous, against a scoped view of the peer's bytes — guest linear memory when the peer is a guest, so an external byte mover's own set() IS the single canonical-ABI copy. "more"/"done" is wasmtime's poll cadence spelled event-style; "done" with zero marked retracts (the speculative-park correction), and marks acknowledge on clean return only, so no zero-progress COMPLETED copy is ever emitted. The rendezvous half is a seam in task/streams.ts that collapses to the reference's dst.write(src.read(n)) verbatim whenever neither side is direct; the no-copy outcomes route to branch shapes definitions.py already produces. The session half (scope, mark accounting, verdict cadence, promise) lives in exec/host_streams.ts. Host-to-host stays at the one-copy floor against the chunk forms (scratch-becomes-chunk / view-of-offered-chunk); two direct sessions refuse loudly. Zero-length probes complete without invoking the callback — the armed session is the readiness claim. Contract: embedder-api amendment A21; architecture §7 gains the scoped exception to "views into guest memory are never exposed".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #128.
Contract (embedder-api amendment A21, wasmtime
DirectSource/DirectDestination-shaped per the issue's ask): forstream<u8>only,StreamWriter.writeDirect(produce)andStream.readDirect(consume)— plus the same forms on the low-levelHostWritableEnd/HostReadableEndseam — park a direct session. At every rendezvous with a peer operation of nonzero capacity the callback runs exactly once, synchronously, inside the rendezvous, with a scoped view of the peer's bytes (remaining()+markWritten/markRead). When the peer is a guest the view aliases guest linear memory, so an external byte mover's ownset()is the single canonical-ABI copy — the one-copy floor the issue names, pinned by aview.buffer === memory.buffertest.Design points, all in the amendment text:
"more"/"done"verdicts are wasmtime's poll cadence spelled event-style for JS's push-shaped producers; a session drains across many guest reads/writes with no re-park overhead."done"with zero marked retracts (the JS spelling of wasmtime'sPending: demand arrived while the ring was empty; the peer's op stays parked, no event). Zero-marked-"more"and throwing callbacks reject the session, discard the invocation's marks, and leave the peer parked and the stream alive — a zero-progress COMPLETED copy (unreachable in definitions.py for nonzero ops) is never emitted.memory.growbetween rendezvous of one parked session is safe (pinned by test).writeDirectvs chunkread(max)delivers the scratch as the owned chunk;readDirectvs a parked chunkwriteviews the offered chunk itself (A5 borrow, callback-scoped). Two direct sessions refuse loudly (neither owns memory).PeerTrappedErrorwith the delivered count (with A7's completed-before-trap precision via a low-level-seam-only out-param — the public signatures match the contract exactly), drop/cancel resolve the running total, the A15 transfer guard coversreadDirect, one-in-flight-per-end covers the direct forms.Implementation. The rendezvous half is a seam in task/streams.ts that collapses to the reference's
dst.write(src.read(n))verbatim when neither side is direct, and routes the no-copy outcomes (retraction/failure) onto branch shapes definitions.py already produces (the parked-side-exhausted and zero-capacity-arrival states). The session half — scope, mark accounting, verdict cadence, promise — lives in exec/host_streams.ts; the conventions layer mirrorswrite/read's refusal and peer-trap shapes. 31 new tests (25 raw-seam, 6 e2e throughstream-pass).Full
just gatespass (incl. shells, browsers, smoke-tls, smoke-c0). No breaking labels: additive surface only (new methods + exported types); lockstep manifests untouched.