jagged: group reshaped q prime commitments by hero78119 · Pull Request #59 · scroll-tech/gkr-backend

hero78119 · 2026-05-13T12:45:47Z

Problem

Jagged reshape can create many q' columns; committing and opening all of them as one large inner batch is too coarse for grouped PCS openings and device-backed witnesses.

Approach

Group reshaped q' columns into fixed-width inner RMMs (JAGGED_RESHAPE_GROUP_WIDTH = 8).
Split jagged inner openings/verifications to match those q' column groups.
Choose the default reshape height from max real physical row height instead of total eval count.
Add witness metadata for device-backed, host-elided matrices and expose height/width helpers.
Clean up post-merge clippy issues in WHIR imports and divisibility checks.

Testing

cargo test -p mpcs jagged::tests --lib — 14 passed
cargo make clippy — passed

Risk / Rollout

Changes are isolated to jagged packing/opening shape and witness metadata. Prover and verifier grouping are updated together; rollback is reverting this PR.

Copilot Reviewer Directive (keep this section)

When Copilot reviews this PR, apply .github/copilot-instructions.md strictly.

Perform review only; do not commit/push/propose code changes.
Prefer inline comments on changed lines for each actionable finding.
If inline comments are unavailable, use [severity] path:line (symbol) format.
Prioritize soundness, performance, and architecture risks over style.
Output order: findings by severity, then open questions, then brief summary.
If PR description is empty or missing key context, report PR metadata: description as a finding.

* claude code impl plan * implement the jagged_sumcheck using time-space tradeoff sumcheck prover algorithm * remove debugging codes * ref to the original paper * add jagged sumcheck bench * #32 parallelize (#34) * par wip * check f(z) * g(z) matches expected evaluation * fix clippy * fix clippy: add #[cfg(test)] to test-only method and fix unused import/variable warnings Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/4a61316e-cf28-47b8-a43e-fb6ab432701e Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com> * refactor test * apply functional programming style * avoid unnecessary BaseField-to-ExtensionField conversion in q_evals access Use E * BaseField multiplication directly instead of converting q_evals elements to extension field with .into() first. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * replace col_row binary search with incremental ColRowIter Add ColRowIter that does one binary search at construction and O(1) per step, replacing per-element binary searches in build_m_table, bind_and_materialize, compute_claimed_sum, and final_evaluations_slow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * extend jagged sumcheck benchmark to cover n=25..31 * switch jagged sumcheck benchmark to BabyBearExt4 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove jagged sumcheck plan doc Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…rove (#38) * claude code impl plan * implement the jagged_sumcheck using time-space tradeoff sumcheck prover algorithm * remove debugging codes * ref to the original paper * add jagged sumcheck bench * #32 parallelize (#34) * par wip * check f(z) * g(z) matches expected evaluation * fix clippy * fix clippy: add #[cfg(test)] to test-only method and fix unused import/variable warnings Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/4a61316e-cf28-47b8-a43e-fb6ab432701e Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com> * refactor test * apply functional programming style * avoid unnecessary BaseField-to-ExtensionField conversion in q_evals access Use E * BaseField multiplication directly instead of converting q_evals elements to extension field with .into() first. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * replace col_row binary search with incremental ColRowIter Add ColRowIter that does one binary search at construction and O(1) per step, replacing per-element binary searches in build_m_table, bind_and_materialize, compute_claimed_sum, and final_evaluations_slow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * extend jagged sumcheck benchmark to cover n=25..31 * switch jagged sumcheck benchmark to BabyBearExt4 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove jagged sumcheck plan doc Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Initial plan * Make EPOCH_SIZES configurable in jagged_sumcheck_prove via optional parameter Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/b18805c7-6e15-44c0-ab03-a1905068d964 Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com> * Fix doc spacing and add debug_assert for epoch_sizes validation Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/b18805c7-6e15-44c0-ab03-a1905068d964 Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com> --------- Co-authored-by: kunxian xia <xiakunxian130@gmail.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: hero78119 <3962077+hero78119@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* implement evaluate_g using width-4 ROBP from Section 3.2/4 Evaluate the MLE of indicator function g(a,b,c,d) = [a+c=b AND b<d] using a width-4 read-once branching program with state (carry, lt). Placed in separate jagged_evaluator.rs module. Includes brute-force test for correctness verification (n=1..4). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * implement forward and backward algorithms for evaluate_g Separate evaluate_g into two functions following the ROBP-based MLE evaluation: forward (source→sinks, MLE definition) and backward (sinks→source, Claim 4.2.1/Lemma 4.2 from jagged PCS paper). Extract shared transition_weights helper. Default to backward for future batch/symbolic evaluation support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Add the opening protocol for the jagged PCS: given K evaluation claims on individual column polynomials, prove they're consistent with the commitment to the giga polynomial q'. Uses the jagged sumcheck to reduce to a single evaluation of q', then opens via the inner PCS. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

… polynomials The zero-padding interpretation (p_i^pad) explains why C_i = eq(z_r[s_i..], 0) is needed when batching polynomials of different heights with a single eq table. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Clarify that each polynomial in q' occupies a full 2^{s_i} block (not the original h_i entries) because bit-reversal scatters the zero-padding throughout the block. Add note comparing with SP1's jagged PCS which uses raw h_i, highlighting the Σ(2^{s_i} - h_i) extra zeros as the cost of suffix-to-prefix bit-reversal. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: implement assist sumcheck to reduce K ROBP evaluations to one Implements the assist sumcheck protocol (Lemma 5.1 of eprint 2025/917) which batches K indicator function evaluations into a single opening, using an interleaved variable ordering and forward-backward ROBP state decomposition. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf: parallelize backward precomputation and fuse round accumulations - Parallelize the O(K × n_robp) backward vector precomputation with rayon - Derive bwd_sum_d from bwd_sum in O(1) instead of a second O(K) pass - Fuse the two per-step weight updates into one - Add assist_sumcheck benchmark (K=100/500/1000) K=1000 drops from 14.1ms to 4.6ms (3.1x speedup with --features parallel). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: annotate key MLE telescoping identity in assist sumcheck Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * bench: update assist sumcheck benchmark to K=1000,2000,4000,8000 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: derive ROBP transition matrices from raw transition table Separate the ROBP state machine logic from the eq-weighting algebra. The raw transition table ROBP_TRANSITION encodes the state machine directly, and symbol_transition_matrices derives M_i^{(c,d)} via the closed formula Σ_{a,b} eq₁(z1,a)·eq₁(z2,b)·[transition(from,(a,b,c,d))=to]. Also document why z₁/z₂ are bound first (reducing alphabet from {0,1}⁴ to {0,1}²) and z₃/z₄ interleaved to match ROBP step order. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: suppress needless_range_loop clippy lint and apply fmt Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * bench: add end-to-end jagged PCS benchmark (commit, open, verify) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: suppress clippy needless_range_loop in evaluator tests, add K=16000 bench Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf: transpose bwd/c_bits/d_bits to step-major layout for cache-friendly access Reorganize data structures from poly-major bwd[y][i] to step-major bwd[i][y], and similarly for c_bits and d_bits. The main loop now scans contiguous memory when accumulating over K polynomials, reducing cache misses at large K (~38% improvement at K=16000). Backward precomputation uses split_at_mut + par_iter for safe parallel writes. Also adds the round polynomial formula (§2.3, Eq. 4) as a code comment to clarify the bwd_sum accumulation logic. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf: parallelize assist sumcheck round-polynomial computation and weight update Thread-local bwd_sum buckets + per-thread p(0),p(1),p(2) evaluation eliminates the sequential O(K) bottleneck in the main step loop. Each thread builds local buckets, computes local round polynomial values, and only the scalars are summed across threads. Benchmark (BabyBearExt4, --features parallel, vs HEAD): K=1000: 7.8ms → 6.8ms (-14%) K=2000: 13.5ms → 10.2ms (-24%) K=4000: 25.6ms → 16.2ms (-38%) K=8000: 50.7ms → 28.0ms (-46%) K=16000: 100.5ms → 53.6ms (-48%) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * perf: eliminate bwd_sum merge by reusing per-thread buckets in round 2i+1 Instead of merging thread-local bwd_sums after round 2i, cache them and absorb alpha directly into each thread's local bwd_sum for round 2i+1. Only scalar p(0),p(1),p(2) values are summed across threads in both rounds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: gate IndexedParallelIterator import behind parallel feature, remove tautological debug_assert Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fmt * fix: use wildcard maybe_rayon import, remove unused direct rayon dep Wildcard `use p3::maybe_rayon::prelude::*` avoids unused-import warnings caused by Cargo feature unification: `p3/parallel` can be activated by other workspace crates even when `mpcs/parallel` is off, making a `#[cfg(feature = "parallel")]` gate on IndexedParallelIterator incorrect. Also remove `dep:rayon` from mpcs — all parallelism goes through `p3::maybe_rayon`, so the direct rayon dependency was unnecessary. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…usion in prover Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/22b3ba65-8e69-45eb-ae09-0a5267f8b518 Co-authored-by: kunxian-xia <1082586+kunxian-xia@users.noreply.github.com>

…trix - Add comparison benchmark (comparison.rs) measuring commit, batch_open, batch_verify, and proof size for jagged PCS vs direct inner PCS - Refactor jagged_commit to accept p3::matrix::dense::RowMajorMatrix instead of witness::RowMajorMatrix, supporting non-power-of-two matrix heights (internally padded to next power of two before bit-reversal) - Update jagged_pcs bench to use BabyBearExt4, parallel make_rmm, jittered non-power-of-two heights, and eq-table-based column evaluation - Cap reshape_log_height to 25 to fit BabyBear two-adicity constraint Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

misc: prefix align without bit reverse

# Conflicts: # crates/sumcheck/src/frontload.rs

…agged_pcs

…agged_pcs_ori

# Conflicts: # crates/mpcs/src/jagged/mod.rs # crates/mpcs/src/lib.rs

hero78119 · 2026-05-15T14:01:56Z

    transcript.append_field_element_exts(evals);
    let num_col_vars = ceil_log2(num_polys).max(1);
-    let z_col: Vec<E> = transcript.sample_and_append_vec(b"jagged_z_col", num_col_vars);
+    let z_col: Vec<E> = transcript.sample_and_append_vec(b"z_col", num_col_vars);


"jagged_z_col" will trigger bytes_to_field util function the panic in debug build, due to Babybear::from_canonical_u32(/* label bytes*/) so here quick fix is shorten "z_col"

kunxian-xia and others added 30 commits April 7, 2026 18:34

wip jagged_commit

80d5537

Merge branch 'main' into feat/jagged_commit_adaptor

83fd31c

update doc comments

a8240d0

polish doc comments

634f725

polish doc comments

ef45229

reorg

20f8dc5

move types to a separate file

a5e0b3e

fix: add missing rayon trait imports for --all-features clippy

a0031e1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Merge branch 'main' into feat/jagged_pcs

80a5818

refactor: rename generic parameter Pcs to InnerPcs for readability

ed522af

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Merge branch 'main' into feat/jagged_pcs

3de7ad4

docs: label batched opening claim and document correction factor incl…

560f5a9

…usion in prover Agent-Logs-Url: https://github.com/scroll-tech/gkr-backend/sessions/22b3ba65-8e69-45eb-ae09-0a5267f8b518 Co-authored-by: kunxian-xia <1082586+kunxian-xia@users.noreply.github.com>

feat: add reshape step to jagged PCS (#48)

41d40f7

feat: make JaggedSumcheckInput::compute_claimed_sum public

c530e7f

misc: prefix align without bit reverse

eef1ee2

jagged: direct concat for prefix-aligned points (#56)

85dbfef

misc: prefix align without bit reverse

Merge branch 'main' into feat/jagged_pcs

fced6a7

WIP integrate jagged pcs cpu

c15f312

Merge remote-tracking branch 'upstream/main' into feat/jagged_pcs

5443c88

# Conflicts: # crates/sumcheck/src/frontload.rs

Fix jagged PCS padded opening normalization

a075bdc

misc: format

320130c

misc: clippy fix

1886aef

hero78119 added 8 commits May 12, 2026 20:33

avoid reconstruct q_mles

0bca53c

Merge branch 'main' of github.com:scroll-tech/gkr-backend into feat/j…

83ad331

…agged_pcs

more docs

9c32d21

Merge branch 'main' of github.com:scroll-tech/gkr-backend into feat/j…

6aec32e

…agged_pcs_ori

Merge branch 'feat/jagged_pcs_ori' into feat/jagged_pcs

0f720f4

misc: fmt and clippy

ab73c00

feat: group jagged q prime commitments

cfa769f

Merge remote-tracking branch 'upstream/main' into feat/jagged_pcs_impl

b633b57

# Conflicts: # crates/mpcs/src/jagged/mod.rs # crates/mpcs/src/lib.rs

hero78119 changed the title ~~Feat/jagged pcs~~ jagged: group reshaped q prime commitments May 13, 2026

hero78119 added 2 commits May 14, 2026 08:06

retain fast path for one column

5f887bf

fix label

a1ec1fa

hero78119 commented May 15, 2026

View reviewed changes

misc: function scope

11f984e

kunxian-xia approved these changes May 18, 2026

View reviewed changes

kunxian-xia merged commit 2222cf2 into main May 18, 2026
2 checks passed

kunxian-xia deleted the feat/jagged_pcs branch May 18, 2026 02:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

jagged: group reshaped q prime commitments#59

jagged: group reshaped q prime commitments#59
kunxian-xia merged 41 commits into
mainfrom
feat/jagged_pcs

hero78119 commented May 13, 2026 •

edited

Loading

Uh oh!

hero78119 May 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

hero78119 commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Approach

Testing

Risk / Rollout

Copilot Reviewer Directive (keep this section)

Uh oh!

hero78119 May 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hero78119 commented May 13, 2026 •

edited

Loading