Skip to content

bench/boundary: stream-shaped lanes (#68) #81

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lann merged 1 commit into from
Aug 11, 2026
Merged

bench/boundary: stream-shaped lanes (#68) #81

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 73 additions & 19 deletions bench/boundary/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,40 +38,94 @@ promise — the wakeup-shaped path a real receive takes). Payload sizes 0
export calls after a warmup call; each export call runs `iters`
boundary crossings.

## Baseline (2026-08-10, linux-arm64 dev box, Node 24.18 / Deno 2.9.5, guest wit-bindgen 0.60)
## Stream shapes ([#68](https://github.com/lann/deltic/issues/68))

| export | what it measures |
| --- | --- |
| `stream-sink: async func(s: stream<u8>) -> u64` (guest drains, returns count) | host→guest payload via rendezvous |
| `stream-source: async func(n: u32) -> stream<u8>` (guest pumps n bytes) | guest→host payload |
| `stream-pass: async func(s: stream<u8>) -> stream<u8>` (guest returns its input unchanged, never reads) | host↔host rendezvous after identity transfer (amendment A5) |

Unlike the calls-per-second shapes above, none of these involve a host
import: the host drives the stream endpoint directly via the embedder's
`Stream` API (`contracts/embedder-api.md` §"Streams and futures"), so
what's measured is the rendezvous/copy cost in isolation. Reported as
MB/s (bytes moved ÷ elapsed), medians of 5 timed runs after a warmup,
same convention as the calls-per-second table. Chunk sizes 1200 B,
16 KiB, 256 KiB; the chunk COUNT per size is chosen (`sweep.mjs`,
`STREAM_CONFIGS`) so one timed run lands in the tens-of-ms range —
smaller chunks need more of them to reach a measurable duration, larger
chunks need fewer. **deltic drivers only**: jco's p3 stream support is
not under test here, so the jco lane is skipped for these rows.

## Baseline (2026-08-11, linux-arm64 dev box, Node 24.18 / Deno 2.9.5, guest wit-bindgen 0.60; post-#63/#67 bulk list copies)

```
shape mode size deltic-node-callback deltic-node-jspi deltic-deno-callback jco-node-jspi
send immediate 0 1,079,807/s 1,016,348/s 1,168,392/s 336/s
send immediate 1200 680,210/s 637,144/s 722,588/s 336/s
send microtask 0 541,613/s 250,349/s 526,868/s 338/s
send microtask 1200 396,974/s 223,360/s 399,989/s 339/s
recv immediate 0 1,182,824/s 1,111,585/s 1,381,300/s 345/s
recv immediate 1200 18,546/s 18,213/s 21,706/s 375/s
recv microtask 0 550,893/s 300,230/s 604,611/s 338/s
recv microtask 1200 18,160/s 17,364/s 20,366/s 490/s
send-sync immediate 0 1,604,379/s 1,057,590/s 1,440,209/s 338,332/s
send-sync immediate 1200 759,075/s 591,627/s 922,990/s 300,576/s
send-sync microtask 0 1,545,879/s 1,234,868/s 1,359,083/s 337,477/s
send-sync microtask 1200 629,432/s 623,742/s 826,725/s 310,692/s
send immediate 0 932,496/s 780,785/s 912,448/s 544/s
send immediate 1200 554,477/s 508,132/s 715,925/s 421/s
send microtask 0 407,627/s 119,069/s 328,809/s 343/s
send microtask 1200 288,646/s 102,759/s 343,579/s 344/s
recv immediate 0 1,063,271/s 905,595/s 1,297,820/s 347/s
recv immediate 1200 1,052,361/s 975,889/s 1,272,385/s 450/s
recv microtask 0 401,310/s 123,669/s 445,292/s 339/s
recv microtask 1200 411,979/s 111,910/s 452,194/s 405/s
send-sync immediate 0 1,536,391/s 1,406,726/s 1,766,432/s 351,160/s
send-sync immediate 1200 795,639/s 589,574/s 894,576/s 311,567/s
send-sync microtask 0 1,653,656/s 788,044/s 1,682,381/s 349,784/s
send-sync microtask 1200 761,535/s 615,453/s 997,213/s 313,497/s

stream lanes (bytes/s; jco lane skipped — see above):
shape size deltic-node-callback deltic-node-jspi deltic-deno-callback
stream-sink 1200 437 MB/s 277 MB/s 417.8 MB/s
stream-sink 16384 3,421.8 MB/s 2,487.2 MB/s 3,604.7 MB/s
stream-sink 262144 7,538.4 MB/s 6,936.1 MB/s 4,711.9 MB/s
stream-source 1200 496.4 MB/s 334.1 MB/s 409.5 MB/s
stream-source 16384 3,318.8 MB/s 3,152.1 MB/s 4,410.3 MB/s
stream-source 262144 14,814.9 MB/s 7,725.9 MB/s 10,997.9 MB/s
stream-pass 1200 756.5 MB/s 625.9 MB/s 838 MB/s
stream-pass 16384 6,665.9 MB/s 5,847 MB/s 7,231.2 MB/s
stream-pass 262144 13,716.5 MB/s 14,229.4 MB/s 22,863.2 MB/s
```

Two methodology footnotes for the stream rows:

- `stream-source` allocates and fills its whole payload inside the guest
within the timed region (one `vec![0x5a; n]` per run), where
`stream-sink`'s host payload is preallocated outside it — the source
lane over-measures by one guest alloc+fill per run. Consistent across
runs, so trend-tracking is unaffected; just don't read sink-vs-source
deltas as pure copy-direction cost.
- "size" means write granularity for `stream-sink` but host *read*
granularity for `stream-source`/`stream-pass` (their producers offer
everything at once; `wit_stream`'s writer does its own internal
chunking). One dimension, two meanings — split it if a finding ever
hinges on the distinction.

What the baseline says:

- **Async import round-trips**: deltic's callback ABI sustains 0.4–1.2 M
- **Async import round-trips**: deltic's callback ABI sustains 0.3–1.1 M
crossings/s; jco's async path costs ~3 ms per call flat
(timer-quantized — its sync path is healthy at ~300 k/s, so the cost
is the async task loop, the same machinery behind lann/jco#11 and
polymorph-iroh's 5 ms polling workaround). For the UDP direct path
(#4) this is the difference between "boundary is free" and "boundary
is the bottleneck".
- **#54**: host→guest `list<u8>` lift runs ~45 ns/byte (~22 MB/s
ceiling, flat across sizes — a per-element copy), while guest→host
lower is bulk (~6.4 GB/s at 16 KiB). The `recv @ 1200` rows are the
regression sentinel for the fix.
- **#54 (fixed in #63; sentinel rows)**: `recv @ 1200` once ran ~18 k/s
(~22 MB/s, a per-element interpreted store); it now tracks the empty
call within ~1 % — the payload copy is bulk in both directions, and
these rows are the regression sentinel. #67 extended the bulk copies
to the remaining flat element types (not separately represented here;
the shapes are `list<u8>`).
- **jspi vs callback** (same runtime, same engine): parity on
immediate-settled paths, ~2× behind on deferred (microtask) paths —
immediate-settled paths, ~2–4× behind on deferred (microtask) paths —
the suspend/resume cost, recorded for #8.
- **#68 stream shapes**: `stream-pass` (host↔host rendezvous, no guest
memory touched — amendment A5) consistently beats `stream-sink` and
`stream-source` (host↔guest, which pay a real memory copy through the
guest's linear memory) at every chunk size, confirming the identity
transfer is doing what it claims. All three scale up sharply with
chunk size — per-rendezvous overhead amortizes over more bytes.

The jco lane pins the family's own toolchain (the lann/jco all-fixes
transpile + preview2-shim release tarballs, the vendored
Expand Down
120 changes: 104 additions & 16 deletions bench/boundary/driver-deltic.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,20 @@
// The bundle is the embedder surface — the LOCAL tree's
// (tools/release-bundle/build.ts output) for tracking this repo, or a
// pinned release asset for cross-version comparison. shape: send | recv
// | send-sync; mode: immediate | microtask (see host.mjs); jspi selects
// jspi-mode suspension (needs --experimental-wasm-jspi under node).
// Emits one JSON line: { lane, shape, mode, size, iters, medianMs,
// callsPerSec }.
// | send-sync | stream-sink | stream-source | stream-pass; mode:
// immediate | microtask (see host.mjs; ignored by the stream-* shapes,
// which have no host import — the host drives the stream endpoint
// directly). jspi selects jspi-mode suspension (needs
// --experimental-wasm-jspi under node).
//
// For the calls-per-second shapes (send/recv/send-sync), `iters` is the
// number of boundary crossings and `size` the payload size; emits
// { lane, shape, mode, size, iters, medianMs, callsPerSec }.
//
// For the stream-* shapes, `iters` is reused as the CHUNK COUNT and
// `size` as the CHUNK SIZE (bytes) — total bytes moved per timed run is
// `iters * size`; emits { lane, shape, mode: "n/a", size, iters,
// totalBytes, medianMs, mbPerSec, kind: "stream" }.
import { makeHost } from "./host.mjs";

const isDeno = typeof Deno !== "undefined";
Expand Down Expand Up @@ -38,17 +48,95 @@ const inst = await deltic.instantiate(
imports,
jspiFlag === "jspi" ? { jspi: true } : { jspi: false },
);
const fn = { send: inst.exports.send, recv: inst.exports.recv, "send-sync": inst.exports.sendSync }[shape];

await fn(Math.min(iters, 1000), size); // warmup
const times = [];
for (let r = 0; r < reps; r++) {
const t0 = performance.now();
await fn(iters, size);
times.push(performance.now() - t0);
}
times.sort((a, b) => a - b);
const median = times[Math.floor(times.length / 2)];

const engine = isDeno ? "deno" : "node";
const lane = `deltic-${engine}-${jspiFlag === "jspi" ? "jspi" : "callback"}`;
console.log(JSON.stringify({ lane, shape, mode, size, iters, medianMs: median, callsPerSec: Math.round(iters / (median / 1000)) }));

if (shape.startsWith("stream-")) {
// Bytes-moved timing for the stream-shaped lanes (issue #68): each
// timed run feeds/drains `iters * size` bytes through one export call.
const chunkCount = iters, chunkSize = size;
const totalBytes = chunkCount * chunkSize;
const payload = new Uint8Array(totalBytes).fill(0xa5);

async function runStreamSink() {
const { stream, writer } = deltic.Stream.create();
const call = inst.exports.streamSink(stream);
const feed = (async () => {
for (let off = 0; off < totalBytes; off += chunkSize) {
await writer.writeAll(payload.subarray(off, Math.min(off + chunkSize, totalBytes)));
}
await writer.close();
})();
const [count] = await Promise.all([call, feed]);
return Number(count);
}

async function runStreamSource() {
const out = await inst.exports.streamSource(totalBytes);
let n = 0;
for (;;) {
const chunk = await out.read(chunkSize);
if (chunk.length === 0) break;
n += chunk.length;
}
out.drop();
return n;
}

async function runStreamPass() {
const { stream, writer } = deltic.Stream.create();
const outP = inst.exports.streamPass(stream);
const feed = (async () => {
for (let off = 0; off < totalBytes; off += chunkSize) {
await writer.writeAll(payload.subarray(off, Math.min(off + chunkSize, totalBytes)));
}
await writer.close();
})();
const out = await outP;
let n = 0;
for (;;) {
const chunk = await out.read(chunkSize);
if (chunk.length === 0) break;
n += chunk.length;
}
await feed;
out.drop();
return n;
}

const run = { "stream-sink": runStreamSink, "stream-source": runStreamSource, "stream-pass": runStreamPass }[shape];
if (!run) throw new Error(`unknown stream shape: ${shape}`);

await run(); // warmup
const times = [];
for (let r = 0; r < reps; r++) {
const t0 = performance.now();
const moved = await run();
times.push(performance.now() - t0);
if (moved !== totalBytes) {
throw new Error(`${shape}: moved ${moved} bytes, expected ${totalBytes}`);
}
}
times.sort((a, b) => a - b);
const median = times[Math.floor(times.length / 2)];
console.log(JSON.stringify({
lane, shape, mode: "n/a", size, iters, totalBytes,
medianMs: median,
mbPerSec: (totalBytes / (1024 * 1024)) / (median / 1000),
kind: "stream",
}));
} else {
const fn = { send: inst.exports.send, recv: inst.exports.recv, "send-sync": inst.exports.sendSync }[shape];

await fn(Math.min(iters, 1000), size); // warmup
const times = [];
for (let r = 0; r < reps; r++) {
const t0 = performance.now();
await fn(iters, size);
times.push(performance.now() - t0);
}
times.sort((a, b) => a - b);
const median = times[Math.floor(times.length / 2)];
console.log(JSON.stringify({ lane, shape, mode, size, iters, medianMs: median, callsPerSec: Math.round(iters / (median / 1000)) }));
}
74 changes: 74 additions & 0 deletions bench/boundary/guest/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion bench/boundary/guest/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ publish = false
crate-type = ["cdylib"]

[dependencies]
wit-bindgen = { version = "=0.60.0", features = ["async"] }
# `async-spawn` backs `stream-source`'s background producer task (issue
# #68); the existing calls-per-second shapes only need plain `async`.
wit-bindgen = { version = "=0.60.0", features = ["async", "async-spawn"] }

[profile.release]
opt-level = "s"
Expand Down
23 changes: 23 additions & 0 deletions bench/boundary/guest/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ mod bindings {
}

use bindings::bench::boundary::host;
use wit_bindgen::rt::async_support::{spawn_local, StreamReader};

struct Component;

Expand Down Expand Up @@ -40,6 +41,28 @@ impl bindings::Guest for Component {
}
acc
}

// Stream-shaped lanes (issue #68): drain / pump / pass-through, no
// host import involved — the host drives the stream endpoint
// directly (see driver-deltic.mjs).

async fn stream_sink(s: StreamReader<u8>) -> u64 {
s.collect().await.len() as u64
}

async fn stream_source(n: u32) -> StreamReader<u8> {
let (mut writer, reader) = bindings::wit_stream::new();
spawn_local(async move {
let payload = vec![0x5au8; n as usize];
writer.write_all(payload).await;
// `writer` drops here, closing the stream.
});
reader
}

async fn stream_pass(s: StreamReader<u8>) -> StreamReader<u8> {
s
}
}

bindings::export!(Component with_types_in bindings);
Loading
Loading