Skip to content

ci: key the e2e build cache on its entries, cache Gradle, drop the pnpm store cache - #909

Merged
janicduplessis merged 3 commits into
mainfrom
@janic/ci-cache-audit
Sep 20, 2026
Merged

janicduplessis merged 3 commits into
mainfrom
@janic/ci-cache-audit

Conversation

@janicduplessis

@janicduplessis janicduplessis commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

Description

The repository's Actions cache is at 10.35 GB across 70 entries, past the 10 GB limit, so GitHub is already evicting least-recently-used entries. Audit of the last 20 executed Native E2E runs and 20 CI runs (9/12 to 9/19), from the job logs and the caches API:

Cache Lane Key Size Observed Verdict
pnpm store (pnpm/setup cache: true) CI test, docs, e2e ios/android pnpm-cache-Linux-x64-<lockfile hash> 14 entries, 3.30 GB (206 MB each) restored on 100% of jobs; pnpm install 4 to 6 s with the store, 4.4 to 5.1 s cold (reused 0, downloaded 1314) drop: buys nothing
pnpm store CI test-windows, windows e2e pnpm-cache-Windows-x64-<hash> 12 entries, 1.79 GB restore 12 to 16 s + install 13 to 15 s; cold: install 15 to 16 s + save 11 s drop: net zero
pnpm store e2e ios (macOS) pnpm-cache-macOS-arm64-<hash> 4 entries, 0.79 GB install 7 to 15 s warm; no cold sample, job is 40 min drop
pnpm store (setup-node cache: pnpm) CI runtime-floor node-cache-Linux-x64-pnpm-<hash> 14 entries, 2.18 GB install+build 6 s with or without it drop
Stim build cache (STIM_BUILD_CACHE) e2e loop, 4 lanes stim-cli-buildcache-<platform>-<framework>-<lockfile hash> + prefix restore-keys 25 entries, 2.27 GB (56 to 127 MB) an archive was restored in 60/60 loop jobs, but the first build hit its fingerprint in 5/40 (12%): ios-bare 0/8, ios-expo 1/7, android-bare 3/13, android-expo 1/12. actions/cache saves only on an exact-key miss, so each entry froze at the first save per lockfile hash while the generated fixture's fingerprint moved (daily on ios-bare). 10 entries (1 GB) were saved from qa/* dispatch branches, unreadable by main rekey (below)
Gradle (~/.gradle) e2e android linux, windows none 0 Windows build log: 16 s distribution download, then 3 min from daemon start to Configure project :app; Linux cold build 4m08 to 6m12 versus 3m37 in the same job with dependencies present add modules-2 + wrapper
CocoaPods, RN prebuilt tarballs e2e ios none 0 pod install 1m35 total; the Hermes/ReactNativeCore/Dependencies downloads take about 6 s leave: not worth an entry
Android system image (Windows) windows e2e none 0 45 to 57 s per run; about 1.2 GB on disk, and Windows restored the 135 MB pnpm store at 12 s leave: a restore would cost as much
Homebrew SimSlim, create-expo-app, cli init e2e ios none 0 5 to 7 s; about 50 s including the template's npm install, floating on @latest leave
cancel-in-progress all both cache actions save in a post step gated on job success; the 6 cancelled runs in the sample saved nothing, and the API lists only committed entries no half-written entries

Solution

Drop the pnpm store cache on every lane (cache: false in setup-stim, no cache: pnpm in runtime-floor, docs.yml). It was 8 GB of the budget for no measurable install time.

Rekey the build cache on its contents. Nothing in the repository predicts the generated fixture's fingerprint, so the restore uses the lane prefix alone and restore-keys picks the newest entry. After the suite, scripts/e2e-build-cache.mjs trims the directory to the two newest fingerprints per platform and names them in the key (stim-cli-buildcache-v2-ios-expo-<fp1>-<fp2>); actions/cache/save runs only when that key differs from the restored one, so an unchanged fixture saves nothing and a moved fingerprint saves one entry, with the previous one expiring 7 days later. Only main saves. Pull requests restore main's entry and never save, because GitHub scopes an entry to the ref that saved it. The detect step prints build cache: HIT <key> or build cache: MISS and derives STIM_E2E_WARM_CACHE from cache-matched-key.

Cache Gradle on the Android lanes. ~/.gradle/caches/modules-2 and ~/.gradle/wrapper (the two directories the caches suite seeds its throwaway home from) on Linux and Windows under gradle-v1-<os>-<framework>-<yyyy-mm> with a prefix restore-key: the first main run of each month refreshes the entry from the previous one. Like the build cache it saves from main only, and not from the caches suite, which never writes to the real Gradle home; modules-2.lock and gc.properties are excluded because the daemon holds the lock open and that fails the archive on Windows (actions/setup-java#633). Entry size is an estimate (0.7 to 1 GB) until the first run reports it.

Expected outcome:

  • Budget: build cache 4 lanes x one archive of at most two fingerprints (about 0.6 GB); a fingerprint move adds one archive that lingers 7 days, so daily moves on every lane would hold up to ~7 GB of archives, of which only the newest per lane is ever read again. Gradle 3 lanes x one entry (2 to 3 GB), 4 to 6 GB for the week after a month rollover; an exact monthly hit skips the save, so a dependency added mid-month is downloaded until the rollover. Only main saves either, so pull requests add nothing. Under 4 GB typical; the worst overlap can touch the 10 GB limit, and what LRU evicts then is a stale archive nothing restores.
  • Build cache hit rate on the first build: 12% today, to every run whose fixture fingerprint has not moved since the last main run (2 to 6 min of cold build saved on iOS and Linux Android).
  • Minutes per lane: CI Linux lanes unchanged; Windows unit lane about 15 s shorter; Linux Android about a minute less Gradle download on a cold build; Windows Android about 3 min less per run (every push to main and every e2e-smoke/e2e-loop labeled pull request).

Test plan

  • pnpm test scripts/e2e-build-cache.test.mjs: keeps the two newest entries per platform, drops older ones and *.staging-* leftovers, keys an unchanged set identically and a changed set differently, no key for an empty root.

  • node scripts/e2e-build-cache.mjs /tmp/bc stim-cli-buildcache-v2-ios-expo on a hand-made root printed key=stim-cli-buildcache-v2-ios-expo-aaaa1111aaaa and removed the staging directory.

  • The full check set (format:check, lint, build, typecheck, knip, test) passes.

  • The rekeying is only fully verifiable after merge: on this branch (e2e-smoke label) the Android jobs should log Cache not found for input keys: gradle-v1-... and the build cache build cache: MISS, with no save of either; the first main run should save gradle-v1-* and stim-cli-buildcache-v2-* entries, the latter naming at most two fingerprints, and the nightly after it should log build cache: HIT and wt1 warm-started from the cross-run cache.

  • Total size, to read after the first nightly:

    gh api repos/appandflow/stim/actions/caches --paginate --jq '.actions_caches[] | "\(.size_in_bytes) \(.key)"' | awk '{s+=$1} END {print s/1e9 " GB"}'

    The 70 existing entries stop being requested and expire after 7 days; gh api -X DELETE "repos/appandflow/stim/actions/caches?key=<key>" clears one sooner.

Fixes #904.

@janicduplessis
janicduplessis added this pull request to stack #906 September 19, 2026 12:56
Base automatically changed from @janic/e2e-labels-plan to main September 20, 2026 00:07
@janicduplessis janicduplessis added e2e-smoke one worktree builds, launches and stops on iOS, Linux Android and Windows Android and removed e2e-smoke one worktree builds, launches and stops on iOS, Linux Android and Windows Android labels Sep 20, 2026
@janicduplessis
janicduplessis marked this pull request as ready for review September 20, 2026 00:46
@janicduplessis
janicduplessis merged commit 8bb4892 into main Sep 20, 2026
8 checks passed
@janicduplessis
janicduplessis deleted the @janic/ci-cache-audit branch September 20, 2026 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Audit and fix CI caching: pnpm store, Stim build cache, Gradle, CocoaPods, fixture downloads

1 participant