diff --git a/SUMMARY.md b/SUMMARY.md index 7e5229cd..c738af3b 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -11,6 +11,7 @@ * [Sharded runs](getting-started/ci-setup/github-actions/re-run-failed-only-tests-sharded.md) * [Orchestrated runs](getting-started/ci-setup/github-actions/re-run-failed-only-tests-orchestrated-v2.md) * [Custom CI Build ID for Reruns](getting-started/ci-setup/github-actions/custom-ci-build-id-for-reruns.md) + * [Cancel Runs on Workflow Cancellation](getting-started/ci-setup/github-actions/cancel-runs.md) * [Commit data for GitHub Actions](getting-started/ci-setup/github-actions/commit-data-for-github-actions.md) * [Custom Docker runners](getting-started/ci-setup/github-actions/custom-docker-runners.md) * [Named Runners](getting-started/ci-setup/github-actions/named-runners.md) @@ -170,6 +171,7 @@ * [currents api](resources/reporters/currents-cmd/currents-api.md) * [currents upload](resources/reporters/currents-cmd/currents-upload.md) * [currents cache](resources/reporters/currents-cmd/currents-cache.md) + * [currents cancel](resources/reporters/currents-cmd/currents-cancel.md) * [currents convert](resources/reporters/currents-cmd/currents-convert.md) * [Changelog](https://github.com/currents-dev/currents-reporter/blob/main/packages/cmd/CHANGELOG.md) * [@currents/jest](resources/reporters/currents-jest/README.md) diff --git a/dashboard/runs/cancel-run.md b/dashboard/runs/cancel-run.md index d103b547..815c3bea 100644 --- a/dashboard/runs/cancel-run.md +++ b/dashboard/runs/cancel-run.md @@ -26,6 +26,20 @@ The canceled run will be tagged accordingly, and the dashboard will display the

Example of a run cancelled by a dashboard user

+## Cancelling Runs from CI + +When a CI job is cancelled — manually, or because a newer commit superseded it — it stops reporting mid-run, and the run stays in progress until it hits the project's [run-timeouts.md](run-timeouts.md "mention"). Add a step that cancels the run when the job is cancelled: + +```yaml +- name: Cancel the run if the workflow is cancelled + if: ${{ cancelled() }} + run: npx currents cancel +``` + +[`currents cancel`](../../resources/reporters/currents-cmd/currents-cancel.md) authenticates with the [record-key.md](../../guides/record-key.md "mention") the job already uses to report results, so it needs no additional secret, and it identifies the run by its [ci-build-id.md](../../guides/parallelization-guide/ci-build-id.md "mention") or its run ID — which makes it work on any CI provider. See [currents-cancel.md](../../resources/reporters/currents-cmd/currents-cancel.md "mention") for the available options. + +On GitHub Actions the same thing is available as an action — see [cancel-runs.md](../../getting-started/ci-setup/github-actions/cancel-runs.md "mention"). + ## Cancelling Runs via API You can programmatically cancel a run via the `PUT runs/:runId/cancel` HTTP API call. For example, here is an example of `curl` command that cancels a particular run @@ -51,27 +65,28 @@ If you have [fail-fast-strategy.md](../../guides/ci-optimization/fail-fast-strat ## GitHub Actions Workflow Cancellation -You can automatically cancel Currents runs (cypress and playwright) when cancelling GitHub Actions workflow using [cancel-run-gh-action](https://github.com/currents-dev/cancel-run-gh-action). - -Check out the [example workflow configuration](https://github.com/currents-dev/currents-examples/blob/main/cypress/github-actions/.github/workflows/currents.yml): +On GitHub Actions the same cancellation is also available as an action, [cancel-run-gh-action](https://github.com/currents-dev/cancel-run-gh-action). See [cancel-runs.md](../../getting-started/ci-setup/github-actions/cancel-runs.md "mention") for a complete workflow. ```yaml - # Run all Currents tests - - name: Run Cypress on Currents.dev - uses: cypress-io/github-action@v4 - with: - command: npx cypress-cloud run --record --parallel --browser chrome --key ${{ secrets.CURRENTS_RECORD_KEY }} --ci-build-id ${{ github.repository }}-${{ github.run_id }}-${{ github.run_attempt}} + - name: Run tests + env: + CURRENTS_RECORD_KEY: ${{ secrets.CURRENTS_RECORD_KEY }} + CURRENTS_PROJECT_ID: my-project-id + CURRENTS_CI_BUILD_ID: ${{ github.repository }}-${{ github.run_id }}-${{ github.run_attempt }} + run: npx playwright test - name: Cancel the run if the workflow is cancelled if: ${{ cancelled() }} uses: currents-dev/cancel-run-gh-action@v1 with: - api-token: ${{ secrets.CURRENTS_API_KEY }} - github-run-id: ${{ github.run_id }} - github-run-attempt: ${{ github.run_attempt }} + record-key: ${{ secrets.CURRENTS_RECORD_KEY }} + project-id: my-project-id + ci-build-id: ${{ github.repository }}-${{ github.run_id }}-${{ github.run_attempt }} ``` -The example above uses a [GitHub Actions repository secret](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) named `CURRENTS_API_KEY.` For creating a new API secret please refer to [Authentication](https://app.gitbook.com/s/lcxad7NaXT7D2V6owvHN/get-started/authentication "mention"). +All three inputs default to `CURRENTS_RECORD_KEY`, `CURRENTS_PROJECT_ID` and `CURRENTS_CI_BUILD_ID`, so a job that already exports them for the reporting step can use the action with no inputs at all. + +Pass `run-id` instead of `ci-build-id` to cancel a run you already have the id of. It defaults to `CURRENTS_RUN_ID` and takes precedence when both are set. After the step is enabled, cancelling a GitHub Actions workflow will trigger cancellation: @@ -81,39 +96,36 @@ The associated Currents run will be cancelled with the corresponding notes:
-#### Cancelling with CI Build ID +### Cancelling with an API key + +The action also accepts an [api-keys.md](../administration/api-keys.md "mention") instead of a record key. It then identifies the run by the GitHub run id and attempt recorded on it, so no other input is needed: -The current implementation allows to cancel a run with CI information that is usually available in the Github environment variables. +```yaml + - name: Cancel the run if the workflow is cancelled + if: ${{ cancelled() }} + uses: currents-dev/cancel-run-gh-action@v1 + with: + api-token: ${{ secrets.CURRENTS_API_KEY }} +``` -But sometimes is required to explicitly define what run needs to be cancelled, so this functionality allows cancelling a run with a known CI Build ID (see [ci-build-id.md](../../guides/parallelization-guide/ci-build-id.md "mention")) and a project ID (see [projects](../projects/ "mention")) which are usually known beforehand in a CI environment as these are required parameters for executing a run. +Pass `project-id` and `ci-build-id` as well to identify the run by its [ci-build-id.md](../../guides/parallelization-guide/ci-build-id.md "mention") instead — which is what you need when the workflow records under a CI build ID of its own: ```yaml - # Run all Currents tests - - name: Run Cypress on Currents.dev - uses: cypress-io/github-action@v4 + - name: Run tests env: CURRENTS_CI_BUILD_ID: "a-custom-ci-build-id" CURRENTS_PROJECT_ID: "my-project-id" - with: - command: npx pwc --key ${{ secrets.CURRENTS_RECORD_KEY }} --project-id ${{ CURRENTS_PROJECT_ID }} --ci-build-id ${{ CURRENTS_CI_BUILD_ID }} + run: npx pwc --key ${{ secrets.CURRENTS_RECORD_KEY }} - name: Cancel the run if the workflow is cancelled if: ${{ cancelled() }} uses: currents-dev/cancel-run-gh-action@v1 with: api-token: ${{ secrets.CURRENTS_API_KEY }} - github-run-id: ${{ github.run_id }} - github-run-attempt: ${{ github.run_attempt }} - ci-build-id: ${{ env.CURRENTS_CI_BUILD_ID }} project-id: ${{ env.CURRENTS_PROJECT_ID }} + ci-build-id: ${{ env.CURRENTS_CI_BUILD_ID }} ``` -The example above uses `CURRENTS_CI_BUILD_ID` and `CURRENTS_PROJECT_ID` as beforehand known variables to pass it down to the run cancellation workflow. - -{% hint style="info" %} -`api-token`, `github-run-id` and `github-run-attempt` are still required parameters that must be passed to the cancellation workflow. -{% endhint %} - ## FAQ ### What happens when a run is cancelled? diff --git a/getting-started/ci-setup/github-actions/README.md b/getting-started/ci-setup/github-actions/README.md index 7a5835b5..11f3d19b 100644 --- a/getting-started/ci-setup/github-actions/README.md +++ b/getting-started/ci-setup/github-actions/README.md @@ -1,2 +1,27 @@ +--- +description: Running tests with Currents in GitHub Actions +--- + # GitHub Actions +Start with [playwright-github-actions.md](playwright-github-actions.md "mention") for a workflow that records to Currents, then add what your setup needs from the articles below. + +## Setup + +* [playwright-github-actions.md](playwright-github-actions.md "mention") — a workflow that records tests to Currents, and how to parallelize it. +* [commit-data-for-github-actions.md](commit-data-for-github-actions.md "mention") — get the correct commit, branch and pull request on a run. +* [custom-docker-runners.md](custom-docker-runners.md "mention") — the environment variables to pass when the job runs in your own container. +* [named-runners.md](named-runners.md "mention") — show which runner executed each spec file. + +## Reruns + +* [re-run-failed-only-tests.md](re-run-failed-only-tests.md "mention") — re-run only the tests that failed, for [sharded](re-run-failed-only-tests-sharded.md) and [orchestrated](re-run-failed-only-tests-orchestrated-v2.md) runs. +* [custom-ci-build-id-for-reruns.md](custom-ci-build-id-for-reruns.md "mention") — when your workflow sets its own CI build ID. + +## Cancellation + +* [cancel-runs.md](cancel-runs.md "mention") — cancel the Currents run when the workflow is cancelled, so it does not sit in progress until the run timeout. + +## Examples + +The [currents-examples](https://github.com/currents-dev/currents-examples/tree/main/playwright/ci/github-actions) repository has complete workflows for sharding, orchestration, reruns and visual testing. diff --git a/getting-started/ci-setup/github-actions/cancel-runs.md b/getting-started/ci-setup/github-actions/cancel-runs.md new file mode 100644 index 00000000..28d13abe --- /dev/null +++ b/getting-started/ci-setup/github-actions/cancel-runs.md @@ -0,0 +1,80 @@ +--- +description: Cancel the Currents run when a GitHub Actions workflow is cancelled +--- + +# Cancel Runs on Workflow Cancellation + +A cancelled workflow stops reporting mid-run. Currents has no way to tell that apart from a job that is still working, so the run stays in progress until it hits the project's [run-timeouts.md](../../../dashboard/runs/run-timeouts.md "mention") — up to an hour of a run sitting in the feed as if it were live. + +Add a step that cancels the run when the job is cancelled: + +```yaml +- name: Cancel the Currents run + if: ${{ cancelled() }} + run: npx currents cancel +``` + +`if: cancelled()` runs the step only when the workflow was cancelled, so it costs nothing on a normal run. + +## Which credential to use + +[`currents cancel`](../../../resources/reporters/currents-cmd/currents-cancel.md) authenticates with the [record-key.md](../../../guides/record-key.md "mention") the job already uses to report results, so no additional secret is needed. It identifies the run by its [ci-build-id.md](../../../guides/parallelization-guide/ci-build-id.md "mention") or its run ID, which means the same step works on any CI provider. + +{% hint style="warning" %} +Set `CURRENTS_CI_BUILD_ID` on the job, as the example below does. Without it Currents generates a CI build id that includes the test framework, and the cancelling step cannot reconstruct that value from the environment — it would report that there is no run to cancel. +{% endhint %} + +The [cancel-run-gh-action](https://github.com/currents-dev/cancel-run-gh-action) does the same as a GitHub action and accepts either a record key or an [api-keys.md](../../../dashboard/administration/api-keys.md "mention"): + +```yaml +- name: Cancel the Currents run + if: ${{ cancelled() }} + uses: currents-dev/cancel-run-gh-action@v1 +``` + +With no inputs it reads `CURRENTS_RECORD_KEY`, `CURRENTS_PROJECT_ID` and `CURRENTS_CI_BUILD_ID` from the environment the reporting step already sets. Like the command, it can also identify the run by its run id — the `run-id` input, or `CURRENTS_RUN_ID`. See the [action's README](https://github.com/currents-dev/cancel-run-gh-action#inputs) for every input. + +## Full example + +```yaml +name: Run Playwright Tests +on: + pull_request: + branches: [main] + +# Cancel the previous run when a new commit is pushed to the same branch. +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + run-tests: + runs-on: ubuntu-latest + env: + CURRENTS_PROJECT_ID: ${{ vars.CURRENTS_PROJECT_ID }} + CURRENTS_RECORD_KEY: ${{ secrets.CURRENTS_RECORD_KEY }} + CURRENTS_CI_BUILD_ID: ${{ github.repository }}-${{ github.run_id }}-${{ github.run_attempt }} + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: "24.x" + - run: npm ci + + - name: Playwright Tests + run: npx playwright test + + - name: Cancel the Currents run + if: ${{ cancelled() }} + run: npx currents cancel +``` + +`concurrency` with `cancel-in-progress: true` is what makes this worth setting up: every push to a branch cancels the workflow still running for the previous commit, and each of those leaves a run behind. + +## Notes + +* **Parallel jobs.** Every job of a parallelized run records into the same run, and every one of them can run the cancellation step. Cancelling a run that is already cancelled succeeds. +* **Nothing recorded yet.** A workflow cancelled before the first results reached Currents has no run to cancel. The step reports that and succeeds, so it does not add a failed step to an already cancelled workflow. +* **Hard cancellations.** A job killed without running its remaining steps — a cancelled job that does not honour `if: cancelled()`, or a runner that disappears — never reaches the step. Those runs still end at the inactivity timeout. + +See [cancel-run.md](../../../dashboard/runs/cancel-run.md "mention") for what cancelling a run affects: test statuses, plan usage, analytics and integrations. diff --git a/resources/reporters/currents-cmd/README.md b/resources/reporters/currents-cmd/README.md index 6702ad3e..fcdb9424 100644 --- a/resources/reporters/currents-cmd/README.md +++ b/resources/reporters/currents-cmd/README.md @@ -28,6 +28,7 @@ npm install @currents/cmd --save-dev * [`currents api`](currents-api.md) - retrieve data from Currents [Resources](https://app.gitbook.com/s/lcxad7NaXT7D2V6owvHN/resources "mention") entities * [`currents upload`](currents-upload.md) - upload the test results into the Currents Dashboard * [`currents cache`](currents-cache.md) - manage test artifacts and configuration cache +* [`currents cancel`](currents-cancel.md) - cancel a run in progress, e.g. when its CI job is cancelled * [`currents convert`](currents-convert.md) - convert test reports to Currents internal format ### Troubleshooting diff --git a/resources/reporters/currents-cmd/currents-cancel.md b/resources/reporters/currents-cmd/currents-cancel.md new file mode 100644 index 00000000..c62d1a09 --- /dev/null +++ b/resources/reporters/currents-cmd/currents-cancel.md @@ -0,0 +1,83 @@ +--- +description: Learn how to cancel a run from CI using the currents cancel CLI command +--- + +# currents cancel + +`currents cancel` cancels a run that is still in progress, for example when the CI job that recorded it is cancelled. + +A cancelled CI job stops reporting mid-run, so without an explicit cancellation the run stays in progress until it hits the project's [run-timeouts.md](../../../dashboard/runs/run-timeouts.md "mention"). + +The command is available from `@currents/cmd` 1.10.0. `npx currents` resolves the latest version, so no change is needed unless the version is pinned. + +### Usage + +{% hint style="info" %} +The command authenticates with the [record-key.md](../../../guides/record-key.md "mention") the job already uses to report results — no API key is needed. It accepts `--key`, `--project-id`, `--ci-build-id` and `--run-id`, or the `CURRENTS_RECORD_KEY`, `CURRENTS_PROJECT_ID`, `CURRENTS_CI_BUILD_ID` and `CURRENTS_RUN_ID` environment variables. +{% endhint %} + +```bash +npx currents cancel --key --project-id --ci-build-id +``` + +### Identifying the run + +Pass either the [ci-build-id.md](../../../guides/parallelization-guide/ci-build-id.md "mention") the run was recorded with, or the run id: + +```bash +npx currents cancel --key --project-id --run-id +``` + +`--run-id` takes precedence when both are set. + +Use `--ci-build-id` for cancelling from CI. Set `CURRENTS_CI_BUILD_ID` on the job — for example `${{ github.repository }}-${{ github.run_id }}-${{ github.run_attempt }}` — and both the reporting step and the cancelling step read the same variable, so no value has to be passed between them. + +Use `--run-id` when you already have the run id: it is the last segment of the run URL, `https://app.currents.dev/run/`. This is the option for cancelling a specific run from a script or by hand. + +{% hint style="warning" %} +If the job does not set `CURRENTS_CI_BUILD_ID`, Currents generates a CI build id from the CI environment, and the generated value includes the test framework — for example `pw:owner/repo-16873-1`. A cancelling step that rebuilds the CI build id from environment variables will not produce that string and will report that there is no run to cancel. Set `CURRENTS_CI_BUILD_ID` explicitly on any job you want to cancel from CI. +{% endhint %} + +### Cancelling from GitHub Actions + +A job that already exports the record key, project and CI build id needs no arguments: + +```yaml +- name: Run tests + env: + CURRENTS_RECORD_KEY: ${{ secrets.CURRENTS_RECORD_KEY }} + CURRENTS_PROJECT_ID: my-project-id + CURRENTS_CI_BUILD_ID: ${{ github.repository }}-${{ github.run_id }}-${{ github.run_attempt }} + run: npx playwright test + +- name: Cancel the run if the workflow is cancelled + if: ${{ cancelled() }} + env: + CURRENTS_RECORD_KEY: ${{ secrets.CURRENTS_RECORD_KEY }} + CURRENTS_PROJECT_ID: my-project-id + CURRENTS_CI_BUILD_ID: ${{ github.repository }}-${{ github.run_id }}-${{ github.run_attempt }} + run: npx currents cancel +``` + +### Cancelling from other CI providers + +The command only needs the record key, the project and the CI build id, so the same step works anywhere. Set the CI build id for the whole pipeline, so the job that reports and the job that cancels use the same value. GitLab CI, for example: + +```yaml +variables: + CURRENTS_CI_BUILD_ID: $CI_PIPELINE_ID + +cancel_currents_run: + stage: .post + when: on_failure + script: + - npx currents cancel +``` + +`CURRENTS_RECORD_KEY` and `CURRENTS_PROJECT_ID` come from the pipeline's variables, the same ones the reporting job uses. + +### Notes + +* Every job of a parallelized run records into the same run. Cancelling a run that is already cancelled succeeds, so it is safe for each job to run the command. +* A run only exists once results have been recorded. Cancelling before the first results were uploaded reports that there is no run to cancel and exits successfully, so the step does not fail on an already cancelled job. +* Cancelled runs are marked in the dashboard and trigger the usual integrations. See [cancel-run.md](../../../dashboard/runs/cancel-run.md "mention") for what cancelling a run affects.