Skip to content

JS-2017 Fix JS coverage handoff for Sonar analysis - #7481

Merged
erwan-leforestier-sonarsource merged 2 commits into
masterfrom
fix/workflow-js-coverage-handoff
Jul 7, 2026
Merged

JS-2017 Fix JS coverage handoff for Sonar analysis#7481
erwan-leforestier-sonarsource merged 2 commits into
masterfrom
fix/workflow-js-coverage-handoff

Conversation

@erwan-leforestier-sonarsource

Copy link
Copy Markdown
Contributor

Summary

This PR fixes the JS coverage handoff in CI.

test_js can still reuse an existing coverage/js cache to skip rerunning bridge coverage, but the coverage files consumed by Analyze in SonarQube NEXT are now passed explicitly as a workflow artifact instead of relying on implicit cache behavior across jobs.

Root cause

The latent workflow bug was introduced by 16cc127bc27f7968e3bb9e6b3ae5e5000a87db2c (JS-943 try bigger machines + windows dependencies (#5909), 2025-11-11).

That change split the flow in two jobs:

  • test_js checked coverage/js with SonarSource/gh-action_cache@v1 and lookup-only: true
  • Analyze in SonarQube NEXT later tried to restore coverage/js from the cache using the same js-files-hash

This was fragile because lookup-only: true means the test_js job is not supposed to restore or save anything. The workflow only kept working because the action implementation still ended up saving branch caches as a side effect.

Why it started failing now

This did not start because of a new SonarJS workflow change. It started because SonarSource/gh-action_cache@v1 is a floating ref and its behavior changed on 2026-06-30 in 8d5f45db5f8e71b8b09d15d3a8f751576cd33afd (BUILD-11220 Skip duplicate S3 cache save when content matches the default-branch fallback).

Before that update, earlier successful runs on PR #7445 still saved a branch cache from test_js after a cold miss, and Analyze in SonarQube NEXT restored that branch cache later in the same workflow.

After the update, the action started enforcing lookup-only mode explicitly:

  • mode=lookup
  • lookup-only input set — no restore, no save

Earlier successful runs used action SHA 339d9d71cc5210bfa866a831dd8eba8e32c40491 and showed:

  • test_js: Cache saved with key: fix/js-1995-s2486-single-statement-try/js-coverage-...
  • Analyze in SonarQube NEXT: Cache restored from key: fix/js-1995-s2486-single-statement-try/js-coverage-...

The failing run used action SHA 4e40632e780e11a8bbe9b721985ab22b42847cc4 and logged the explicit lookup-only behavior instead.

Once @v1 advanced, the latent bug became visible. PR #7445 exposed it because changing files under rules/ changed js-files-hash, which forced a cold miss on a new cache key. test_js no longer saved coverage for that key, Analyze in SonarQube NEXT could not restore anything from the branch or master, and the job failed when coverage/js/test-report.xml was missing.

Fix

  • keep the cache as an optimization inside test_js, but use a normal restore instead of a lookup-only probe
  • always upload coverage/js/lcov.info and coverage/js/test-report.xml from test_js
  • make the Sonar analysis jobs download those reports explicitly

This removes the hidden dependency on cache side effects while preserving the fast path when the JS coverage cache already exists.

@hashicorp-vault-sonar-prod hashicorp-vault-sonar-prod Bot changed the title Fix JS coverage handoff for Sonar analysis JS-2017 Fix JS coverage handoff for Sonar analysis Jul 7, 2026
@hashicorp-vault-sonar-prod

hashicorp-vault-sonar-prod Bot commented Jul 7, 2026

Copy link
Copy Markdown

JS-2017

Comment thread .github/workflows/build.yml
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Ruling Report

No changes to ruling expected issues in this PR

@gitar-bot

gitar-bot Bot commented Jul 7, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 1 resolved / 1 findings

Replaces fragile implicit cache side effects with explicit workflow artifacts for JS coverage reports, resolving the issue of incomplete reports and adding a fast-fail check for missing files.

✅ 1 resolved
Edge Case: Partial coverage cache can produce incomplete artifact silently

📄 .github/workflows/build.yml:458-466
In test_js, the Upload JS coverage reports step always runs, including on a cache hit where the build steps are skipped and coverage/js is populated purely from the restored cache. if-no-files-found: error only fails when zero matching files are found. If a restored cache contains only one of the two expected files (e.g. lcov.info present but test-report.xml missing), the upload succeeds with a partial artifact and the failure surfaces later in analyze_primary/analyze_shadows as a confusing missing-report error — which is exactly the class of failure this PR set out to fix.

In practice this is unlikely because the cache key is derived from js-files-hash and the directory is saved atomically, so the risk is low. If you want to be defensive, consider validating both files exist before upload (or on cache hit), e.g. a small run step that checks coverage/js/lcov.info and coverage/js/test-report.xml and fails fast with a clear message in test_js rather than deferring the failure to the Sonar analysis jobs.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqube-next

sonarqube-next Bot commented Jul 7, 2026

Copy link
Copy Markdown

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0 Dependency risks
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@zglicz zglicz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest doing one or the other:

  1. remove the cache completely and potentially create a task to improve the CI, to reintroduce it
  2. fix the caching behavior and remove the artifacts.

I'll just comment, as I'll leave this up to engineers still owning this repo.

Comment thread .github/workflows/build.yml
Comment thread .github/workflows/build.yml

@zglicz zglicz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, I take it back

@erwan-leforestier-sonarsource
erwan-leforestier-sonarsource merged commit 5094ea3 into master Jul 7, 2026
44 checks passed
@erwan-leforestier-sonarsource
erwan-leforestier-sonarsource deleted the fix/workflow-js-coverage-handoff branch July 7, 2026 14:57
@gitar-bot

gitar-bot Bot commented Jul 7, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 1 resolved / 1 findings

Replaces fragile implicit cache side effects with explicit workflow artifacts for JS coverage reports, resolving the issue of incomplete reports and adding a fast-fail check for missing files.

✅ 1 resolved
Edge Case: Partial coverage cache can produce incomplete artifact silently

📄 .github/workflows/build.yml:458-466
In test_js, the Upload JS coverage reports step always runs, including on a cache hit where the build steps are skipped and coverage/js is populated purely from the restored cache. if-no-files-found: error only fails when zero matching files are found. If a restored cache contains only one of the two expected files (e.g. lcov.info present but test-report.xml missing), the upload succeeds with a partial artifact and the failure surfaces later in analyze_primary/analyze_shadows as a confusing missing-report error — which is exactly the class of failure this PR set out to fix.

In practice this is unlikely because the cache key is derived from js-files-hash and the directory is saved atomically, so the risk is low. If you want to be defensive, consider validating both files exist before upload (or on cache hit), e.g. a small run step that checks coverage/js/lcov.info and coverage/js/test-report.xml and fails fast with a clear message in test_js rather than deferring the failure to the Sonar analysis jobs.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

nathsou added a commit that referenced this pull request Jul 8, 2026
lookup-only: true on populate_npm_cache's cache step means the action
never actually saves node_modules under the branch key (mode=lookup:
"no restore, no save"), same latent bug already fixed for the JS
coverage cache in #7481. It only appeared to work when a branch's
lockfile hash matched master's, via the fallback-exact-key path.

Drop lookup-only so the step does a normal restore-or-install, letting
the action's post-step genuinely save under the branch's own key.
Downstream jobs (build, test_js, test_js_win, prepare_rspec_rule_data)
already do a plain restore against that same key and get a real hit
once it's actually populated - no need to pin gh-action_cache back.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants