Skip to content

ci: add performance regression gate with criterion benchmarks#157

Merged
membphis merged 9 commits into
mainfrom
feat/criterion-bench-ci
Jun 2, 2026
Merged

ci: add performance regression gate with criterion benchmarks#157
membphis merged 9 commits into
mainfrom
feat/criterion-bench-ci

Conversation

@membphis
Copy link
Copy Markdown
Collaborator

@membphis membphis commented Jun 2, 2026

Summary

Closes #151

Add automated performance regression detection to CI using criterion benchmarks and benchmark-action.

Changes

  • Add __bench_api module exposing internal Document, Options, QJSON_MODE_LAZY for benchmark use
  • Add criterion 0.5 dev-dependency with html_reports feature
  • Create benches/rust_bench.rs with three benchmark groups:
    • parse_eager - eager mode parsing across 3 fixtures
    • parse_lazy - lazy mode parsing across 3 fixtures
    • field_access - FFI field access operations (get_str, get_f64, nested paths)
  • Add .github/workflows/bench.yml CI workflow that:
    • Runs on push to main and all PRs
    • Stores baseline in gh-pages branch (auto-push on main only)
    • Posts PR comment when >5% regression detected
    • Does not block merge (reviewer decides)
  • Add make bench-rust target

Testing

  • All benchmarks pass locally (cargo bench --bench rust_bench -- --test)
  • Clippy passes with no warnings
  • Existing tests pass

Notes

  • First merge to main will auto-create baseline in gh-pages
  • PR comments show comparison table with regression highlighting
  • 5% regression marked but does not block merge

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 2, 2026

Review Change Stack

Warning

Review limit reached

@membphis, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 54 minutes and 23 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 050267fb-e2f6-43d1-aa2f-7125caefcdca

📥 Commits

Reviewing files that changed from the base of the PR and between 4d4ea4f and c2fd9c1.

📒 Files selected for processing (6)
  • .coderabbit.yaml
  • .github/workflows/bench.yml
  • Cargo.toml
  • Makefile
  • benches/rust_bench.rs
  • src/lib.rs
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/criterion-bench-ci

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

membphis added 2 commits June 2, 2026 11:13
- Remove unused submodules checkout option
- Add comments explaining workflow permissions
- Add SAFETY comments for unsafe FFI calls in benchmarks
- Add doc comment explaining field_access benchmarks measure FFI overhead
@membphis
Copy link
Copy Markdown
Collaborator Author

membphis commented Jun 2, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 2, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@membphis membphis marked this pull request as ready for review June 2, 2026 03:21
@membphis membphis merged commit 715b83e into main Jun 2, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: add performance regression gate with criterion benchmarks

1 participant