feat(engine): DSpark under continuous batching + load-aware verify length (spec Phases 5/6 remainder)

[pekkah]

Follow-up to PR #426 (DSpark Phases 0-4 + single-user server) and the docs/dspark-plan.md spec (#413).

What's missing

DSpark currently requires the single-user InferenceEngine (MaxBatchSize=1); the loader rejects continuous batching because the tap/draft state is single-sequence:

1. Per-sequence hidden-tap buffers. HiddenTapBuffer is one position-indexed buffer per pass; ContinuousBatchingEngine interleaves sequences with per-sequence KV caches, so capture must be keyed by cache/sequence (the RunBatchedTrunk capture site already guards on the owned cache view — that guard becomes a routing decision instead).
2. Per-sequence draft contexts. IDSparkDraft holds one context (ctx K/V + append cursor). Multi-user needs one draft MODEL (weights shared) with N contexts — likely a CreateContext() handle API, restructuring DSparkDraftModel/CudaDSparkDraftModel state.
3. Batched drafting + packed verify. Per decode step: propose a block per eligible greedy sequence (GPU draft amortizes well; CPU draft per-user is prohibitive), then verify all users' segments in one packed pass — BatchForwardMulti already accepts repeated caches with consecutive positions per user (that's exactly what BatchVerify does single-user), so the trunk machinery may already suffice; per-user accept/rollback via per-sequence TruncateTo.
4. Load-aware verify length (spec §7's server half). Trim each user's verified prefix by confidence AND batch occupancy — the paper's scheduler: under load, shorter verifies keep step latency bounded; when idle, full blocks. The threshold trim already resolves from SHARPI_DSPARK_*; the occupancy signal comes from the batcher.

Why deferred

The payoff only exists where DSpark already wins single-user — which per the Phase-4 measurements means desktop-scale targets (see the companion validation issue). Building the engine's most concurrency-sensitive extension before the single-user win condition is demonstrated would be effort at the wrong altitude.

🤖 Generated with Claude Code

https://claude.ai/code/session_018EWo1niax1g8A5E79RSALJ

[pekkah]

Precondition met + hard data for the verify-length half (post-PR #429)

The "demonstrated win" gate for this issue is cleared: with the draft round at ~11.8 ms (PR #429), DSpark beats plain decode 1.10× on the 4B laptop pairing — but only at the right verify length, which is exactly this issue's scheduling problem:

--dspark-verify-len	t/s	accept
1	55.1	82%
2	64.7	73%
3	67.9	63%
5	65.8	49%
7 (block default)	59.9	37%

plain = 61.5 t/s; same 37%-acceptance-at-k=7 content as the #428 measurements.

Two scheduling implications beyond the load-aware trim already scoped here:

1. The block-size default (k = 7) is the WORST spec config on hard content — it loses to plain while k=2..5 all win. The default should not be "verify the whole block".
2. A fixed short cap beat the confidence trim: --dspark-min-confidence 0.8 reaches only 58.0 t/s vs k=3's 67.9 on this content. The confidence head orders positions well but its absolute calibration doesn't recover the optimal cutoff — an acceptance-EMA-driven dynamic k (target ~55–65% batch acceptance) looks like the better policy, and it composes with the per-request load-aware trim this issue plans for the server.

Note the draft cost is k-independent (the block forward always runs full B), so shortening k costs nothing draft-side — the trade is purely verify tokens (5.05 ms each on 4B) vs expected accepted length. The cost model to tune against: round(k) ≈ 11.8 + 10.6 + 5.05·(k+1) ms vs tokens/round = 1 + E[accepted prefix | k]. #430 (graph-captured verify) would shrink the 10.6 fixed term and shift the optimum k upward — the policy should read the measured verify cost, not hardcode it.

🤖 Generated with Claude Code

https://claude.ai/code/session_018EWo1niax1g8A5E79RSALJ