Speed up CI: nextest + scope Cargo cache to deps only#1747
Open
nogates wants to merge 3 commits into
Open
Conversation
Add hash-based test sharding: scenarios are assigned to a shard by summing their name bytes modulo TEST_SHARD_TOTAL. Each shard runs on its own runner in parallel, cutting the test wall-clock time roughly in half without requiring any changes to feature files. - tests/main.rs: read TEST_SHARD_TOTAL/TEST_SHARD_INDEX env vars and apply modulo filter in the scenario filter closure - reusable-rust-test.yml: add shard-count + shard-indexes inputs, expand shard-index into the strategy matrix, pass env vars to Test step - test.yml: configure shard-count=2 and shard-indexes=[0,1] Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
4 shards gives ~25% of scenarios per runner (~1055 each out of 4219), reducing test-execution time by ~4x in parallel. Replaced byte-sum with FNV-1a (32-bit): better avalanche effect means distribution stays even as the scenario count grows or names cluster. Distribution with 4 shards: [1092, 1003, 1059, 1065] (±4%). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a dedicated build job that compiles the test binary once and uploads it as a GHA artifact. Four test shard jobs then download the binary and run their slice of scenarios without any cargo/Rust toolchain needed -- eliminating ~4-5 min of redundant compilation per shard. License check also moves to the build job so it only runs once. Cache key switches to Cargo.toml hash (Cargo.lock is gitignored). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Two changes to reduce CI pipeline time:
Switch to
cargo nextest run— replacescargo testfor parallel test execution. Nextest downloads as a pre-compiled binary (~10s) and runs test binaries concurrently.Scope the Cargo cache to dependencies only — removes
target/from the cached paths and narrows it to~/.cargo/registry/index,~/.cargo/registry/cache, and~/.cargo/git/db. This keeps the cache small and stable (only invalidated whenCargo.lockchanges, not on every source commit), and avoids the overhead of saving/restoring multi-GB compiled artifacts on each run. The cache key is also unified across all workflows (previously the test workflow included the Rust version in the key, preventing sharing with other jobs).Additional Notes
Trade-off: without
target/cached, compiled dep artifacts are not reused between runs, so each run recompiles deps from source. If this turns out to be too slow in practice,target/can be added back.Review checklist
This PR includes all newly recorded cassettes for any modified tests.
This PR does not rely on API client schema changes.