Skip to content

Add CodSpeed continuous performance benchmarks#727

Closed
codspeed-hq[bot] wants to merge 3 commits intomasterfrom
codspeed-wizard-1776277848593
Closed

Add CodSpeed continuous performance benchmarks#727
codspeed-hq[bot] wants to merge 3 commits intomasterfrom
codspeed-wizard-1776277848593

Conversation

@codspeed-hq
Copy link
Copy Markdown

@codspeed-hq codspeed-hq bot commented Apr 15, 2026

Summary

This PR integrates CodSpeed for continuous performance monitoring of ext-php-rs core operations.

What's included

  • Criterion benchmarks using the CodSpeed compatibility layer (codspeed-criterion-compat), added as a new [[bench]] target in the main crate with the embed feature.
  • CodSpeed GitHub Actions workflow (.github/workflows/codspeed.yml) that runs on pushes to master, pull requests, and manual dispatch. It uses OIDC authentication and simulation mode for reproducible measurements.
  • CodSpeed badge added to the README.

Benchmark coverage

The benchmarks test performance-sensitive paths across the FFI boundary:

  • PHP eval: simple expressions, string concatenation, array creation
  • ZendHashTable: sequential push, string-key insertion, lookup by key/index, iteration over 100 entries
  • ZendStr: short and long string creation
  • Zval: integer set/read, string round-trip

All benchmarks use Embed::run() to start an embedded PHP SAPI, testing the actual runtime behavior of the library rather than synthetic stubs.

CI workflow details

The workflow mirrors the existing benchmark CI setup for PHP and LLVM/Clang dependencies:

  • PHP 8.4 with libphp-embed for the embed feature
  • LLVM/Clang 17 for bindgen
  • cargo-codspeed for building and running benchmarks in simulation mode

Next steps

  • Once merged and the first run completes on master, CodSpeed will start reporting performance diffs on subsequent pull requests.
  • The benchmark suite can be extended over time to cover additional operations (e.g., closures, object property access, class instantiation).

@codspeed-hq codspeed-hq bot force-pushed the codspeed-wizard-1776277848593 branch from ac32d5d to 0d22210 Compare April 15, 2026 18:38
@coveralls
Copy link
Copy Markdown

coveralls commented Apr 15, 2026

Coverage Report for CI Build 24471981668

Coverage decreased (-0.02%) to 65.871%

Details

  • Coverage decreased (-0.02%) from the base build.
  • Patch coverage: No coverable lines changed in this PR.
  • 3 coverage regressions across 1 file.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

3 previously-covered lines in 1 file lost coverage.

File Lines Losing Coverage Coverage
src/internal/mod.rs 3 0.0%

Coverage Stats

Coverage Status
Relevant Lines: 12995
Covered Lines: 8560
Line Coverage: 65.87%
Coverage Strength: 32.96 hits per line

💛 - Coveralls

@codspeed-hq codspeed-hq bot force-pushed the codspeed-wizard-1776277848593 branch from 340237c to fd4327c Compare April 15, 2026 18:43
@codspeed-hq
Copy link
Copy Markdown
Author

codspeed-hq bot commented Apr 15, 2026

CodSpeed setup failed

The CodSpeed benchmark setup for ext-php-rs exhausted all 3 retry attempts due to Rust compilation errors in the benchmark file. The benchmarks use Embed::run() which requires FnOnce() -> R + UnwindSafe, but capturing &mut Criterion in the closure violates this bound. The correct fix (wrapping the entire closure with AssertUnwindSafe instead of just the reference) was implemented locally but could not be pushed because the retry limit was reached. The fix commit (04d289a82) changes Embed::run(|| { c.bench_function(...) }) to Embed::run(AssertUnwindSafe(|| { c.bench_function(...) })) for all 12 benchmark functions. This is a straightforward code fix that would resolve the build on the next attempt.

@github-actions
Copy link
Copy Markdown

🐰 Bencher Report

Branchcodspeed-wizard-1776277848593
TestbedPHP 8.4.20 (cli) (built: Apr 11 2026 00:52:25) (NTS)

⚠️ WARNING: Truncated view!

The full continuous benchmarking report exceeds the maximum length allowed on this platform.

⚠️ WARNING: No Threshold found!

Without a Threshold, no Alerts will ever be generated.

🐰 View full continuous benchmarking report in Bencher

@ptondereau ptondereau closed this Apr 15, 2026
@ptondereau ptondereau deleted the codspeed-wizard-1776277848593 branch April 15, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants