Skip to content

[Bug]: Coverage report fails silently when PR branch is behind develop #15635

Description

@AryanSharma48

Description

In .github/workflows/run_tests_coverage_pr.yml, the workflow fails to detect changed packages whenever a pull request branch is behind develop. As a result, the PR coverage report comments:

Coverage Report

No coverage information available.

even though the tests run, succeed, and all other CI checks are green.

What Happens:

  1. Shallow Fetch of Base Branch: In .github/workflows/run_tests_coverage_pr.yml (lines 124–127), the step Get list of changed directories executes a shallow fetch (--depth=1).
  2. Divergent History with No Merge Base: Because --depth=1 only fetches the single latest commit on develop, if develop has moved forward since the feature branch was created, git cannot find a common ancestor with HEAD.
  3. Silent Failure via continue-on-error: Because there is no merge base in the shallow history, git diff origin/${{ github.base_ref }}...HEAD fails. Because the step specifies continue-on-error: true, the error does not fail the job. Instead, files and directories evaluate to an empty string "".
  4. Coverage Script Exits Early: The downstream script .github/workflows/scripts/run_tests_coverage/run receives an empty directory list and exits with code 0.
  5. Bot Posts Default Fallback: Because the $TABLE is empty, the stdlib-bot publishes the fallback "No coverage information available." comment instead of the coverage table.

Comparison with Other Workflows:
Other workflows in the repository (such as .github/workflows/lint_changed_files.yml and .github/workflows/run_affected_tests.yml) avoid this issue by computing the common ancestor commit using git merge-base. Because actions/checkout already fetches history (fetch-depth: 1000), the git merge-base calculation works reliably even if the branch is behind develop.

Proposed Fix:
Update the Get list of changed directories step in .github/workflows/run_tests_coverage_pr.yml to use git merge-base, matching the standard used in the linting workflow:

- git fetch origin ${{ github.base_ref }} --depth=1
- files=$(git diff --diff-filter=AM --name-only origin/${{ github.base_ref }}...HEAD)
+ ancestor_commit=$(git merge-base ${{ github.event.pull_request.base.sha }} ${{ github.event.pull_request.head.sha }})
+ files=$(git diff --diff-filter=AM --name-only $ancestor_commit ${{ github.event.pull_request.head.sha }})
  directories=$(for file in $files; do dirname $file; done | sort -u | tr '\n' ' ' | sed 's/ $//')
  echo "directories=${directories}" >> $GITHUB_OUTPUT

Related Issues

None.

Questions

If the proposed fix looks good, I am happy to open a PR to update the workflow file!

Demo

This bug can be observed in PR #15633.

  • On the initial commit, the feature branch was behind develop, and the coverage action silently failed with the fallback message.
  • Once develop was merged into the PR branch (syncing the history), the coverage table generated successfully, confirming that the missing merge-base was the root cause.

Reproduction

  1. Open a Pull Request where the feature branch is behind develop.
  2. Wait for the run_tests_coverage_pr.yml GitHub Actions workflow to execute.
  3. Observe the Get list of changed directories step silently failing to find changed files.
  4. Observe the stdlib-bot commenting the fallback text: "No coverage information available."

Expected Results

The workflow should correctly identify the changed directories using the merge base (identical to how .github/workflows/lint_changed_files.yml operates) and generate a valid coverage table for the PR.

Actual Results

The workflow silently fails to calculate the git diff due to the shallow fetch, resulting in the bot commenting a useless fallback message on the PR.

Version

develop (Latest)

Environments

N/A

Browser Version

No response

Node.js / npm Version

No response

Platform

Ubuntu (GitHub Actions Runners)

Checklist

  • Read and understood the Code of Conduct.
  • Searched for existing issues and pull requests.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugSomething isn't working.CIIssue or pull request specific to continuous integration environments.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions