Skip to content

perf(gpu): run DECODE (num_parts=1) DEEP/FRI on device - #946

Open
ColoCarletti wants to merge 2 commits into
mainfrom
decode-deep-fri-gpu
Open

perf(gpu): run DECODE (num_parts=1) DEEP/FRI on device#946
ColoCarletti wants to merge 2 commits into
mainfrom
decode-deep-fri-gpu

Conversation

@ColoCarletti

Copy link
Copy Markdown
Collaborator

Extends the device-resident composition-parts path to num_parts==1 so the DECODE table's DEEP/FRI runs on GPU instead of host.

For a degree-1 composition, H is already the single part on the LDE coset — no decompose, no re-extension, just an interleaved→slab de-interleave (new comp_h_to_slabs_ext3 kernel, no NTT). The R2 commit, R3 OOD, R4 DEEP, FRI and openings already read the part count from the handle, so no downstream changes are needed; the host fallback is preserved.

Validation (ethrex 100tx, RTX 5090):

  • Wall: 13.13s -> 12.18s (-7.25%, ABBA x3)
  • Cross-verify OK; math-cuda + stark cuda suites green
  • Unit parity: de-interleave bit-exact, evaluate_dev(keep) == host bit-for-bit

@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/bench-gpu

@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/ai-review

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown

GPU Benchmark (ABBA) — a9aead5753 vs main (14 pairs)

RTX 5090 · AMD Ryzen 9 9950X 16-Core Processor (32 threads) · Vast.ai datacenter @ $0.9347222222222221/hr · prover/cuda · ethrex real block, continuations · drift-free A/B/B/A

=== ABBA paired result  (improvement: - = PR faster) ===
  pairs: 14   mean A (PR): 71.796s   mean B (base): 74.506s

  [parametric] paired-t   mean -3.55%   sd 4.69%   se 1.25%
               95% CI: [-6.26%, -0.85%]   (t df=13 = 2.16)
  [robust]     median -4.53%   Wilcoxon W+=14 W-=91  p(exact)=0.0134  (z=-2.39)

  --- server stability (this run; compare across servers) ---
  run-to-run jitter:    A CV 7.20%   B CV 6.93%        (lower = steadier)
  within-session drift: +6.97% over the run, 1st->2nd half +4.27%
    (jitter -> Tier-1 cached gate floor; drift -> whether the cached baseline can be trusted)

  VERDICT: REAL IMPROVEMENT - PR faster by ~3.55% (t-CI and Wilcoxon agree)

  raw pairs: /tmp/abba_run/pairs.csv

- = PR faster. Trust the verdict when paired-t and Wilcoxon agree.

@github-actions

Copy link
Copy Markdown

Codex Code Review

No issues found in the PR changes.

@claude

claude Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Review: perf(gpu): run DECODE (num_parts=1) DEEP/FRI on device

Reviewed the kernel, the math-cuda wrapper, the prover dispatch, and every downstream consumer of GpuLdeExt3 (build_comp_poly_tree_from_slabs_dev, try_barycentric_ext3_on_ext3_handle, deep_composition_ext3, gather_rows_ext3_on_device, download_ext3_columns, materialize_composition_parts_host). No Critical or High safety issues. The design holds up:

  • comp_h_to_slabs_ext3 is a pure grid-strided transpose writing all 3 * num_rows u64, so the unsafe { stream.alloc } (no zero-init) is sound — the contrast with the zero-padded d=2 slab layout is correctly called out in the comment.
  • Slab ordering out[k*num_rows + i] matches the handle contract (c*3 + k) * lde_size at m = 1, and matches decompose_d2_ext3's [h0.abc, h1.abc] layout.
  • Cross-stream handoff mirrors the d=2 path: kernel enqueued on h.stream, PooledEvent recorded, consumers wait_ready_on device-side. GpuCompH::buf's stream-ordered free is ordered behind the kernel on the same stream.
  • Every downstream consumer already reads the part count from handle.m / lde_composition_poly_evaluations.len(), so m = 1 needs no changes — confirmed, and num_parts == 1 already exercised the GPU comp-poly tree before this PR.
  • The host fallbacks (downloaded_hvec![h], and the else if number_of_parts == 1 arm) are value-equivalent to evaluator.evaluate(...).

Findings

Medium — device_only_for gate not extended; the want_host == false branch is dead. device_only_gate is still reached only when composition_poly_degree_bound(n) / n == 2 (prover.rs:1101), and host_trace_empty is set only when that gate held at R1. So for a num_parts == 1 table want_host is always true: the vec![Vec::new()] placeholder in try_deinterleave_comp_h_dev is unreachable, DECODE still does the full H D2H every proof, and the gate's comments ("the device-resident R2 path only exists for the d=2 quotient decomposition", prover.rs:1094 and :1101) are now wrong. Inline detail on the branch.

Low — gpu_comp_h_slabs_calls() is exported but never read. Every sibling counter is asserted in prover/tests/cuda_path_integration.rs; this one isn't, so a silent regression of the d=1 path to the host arm would go unnoticed. Add the assertion or drop the counter.

Low — comp_h_from_host_interleaved is test-only public API on math_cuda::constraint_interp. It's honestly documented as a parity helper and it does need to be pub for the integration test, so this is just a note that it widens the crate's surface for no production caller.

Low — the if number_of_parts == 1 { deinterleave } else { decompose_d2 } dispatch is duplicated verbatim in round_2_compute_composition_polynomial (prover.rs:1702) and xcheck_post_mortem (prover.rs:4299), as is the vec![host_h] vs decompose_and_extend_d2 split. A small helper would keep the two from drifting — the xcheck's whole value is being an apples-to-apples mirror of the producer.

Test coverage

comp_h_to_slabs.rs (bit-exact transpose over log 1..=14, raw non-canonical limbs) and the decode_shaped_program + evaluate_dev(keep=true) == host-drained H parity in gpu_constraint_interp.rs are both well targeted at the two things that could go wrong here. The gap is the integration level — nothing asserts the d=1 device path actually engages end to end (see the counter finding).

Comment thread crypto/stark/src/gpu_lde.rs Outdated
Comment thread crypto/stark/src/gpu_lde.rs
@github-actions

Copy link
Copy Markdown

AI Review

PR #946 · 7 changed files

Findings

No non-rejected structured findings were reported.

Reviewer Lanes

Lane Model Prompt Status Findings
glm openrouter/z-ai/glm-5.2 general success 0
kimi openrouter/moonshotai/kimi-k2.7-code general success 0
minimax minimax/MiniMax-M3 general error: opencode failed (provider/auth/runtime error) and no findings were submitted 0
moonmath zro/minimax-m3 general error: opencode failed (provider/auth/runtime error) and no findings were submitted 0
nemotron openrouter/nvidia/nemotron-3-ultra-550b-a55b general success 0

Native Codex and Claude reviews run separately and post their own comments. They are not included in this structured provenance report.

Raw lane outputs, candidates, final issues, and model metrics are uploaded as workflow artifacts.

@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/bench-gpu

1 similar comment
@ColoCarletti

Copy link
Copy Markdown
Collaborator Author

/bench-gpu

Extend the device-resident composition-parts path to num_parts==1. For d=1, H is already the single part on the LDE coset, so deinterleave it into a 1-part GpuLdeExt3 (comp_h_to_slabs_ext3 kernel, no NTT) instead of running DEEP/FRI on host; the commit, R3 OOD, R4 DEEP, FRI and openings already read the part count from the handle. Proof-identical; host fallback preserved.
…variants

- add prover/tests/cuda_d1_path.rs + `make test-cuda-d1` (gpu_test.sh Group 3):
  lowers LAMBDA_VM_GPU_LDE_THRESHOLD so DECODE engages the d=1 device DEEP/FRI
  path end to end, asserting gpu_comp_h_slabs_calls > 0 and the proof verifies.
  Its own binary because gpu_lde_threshold() caches the env (OnceLock).
- decompose_comp_h_dev: debug_assert want_host and H rows == LDE domain size on
  the d=1 branch (the d=2 arm gets an incidental check via weights.len()).
- try_deinterleave_comp_h_dev: document that the always-drained host part feeds
  the release query-0 canary — the only e2e check on the d=1 layout.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant