Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,18 @@ This folder ships ready-to-copy GitHub Actions workflows that exercise the
runs `evidence capture-evidence` on every pull request and posts a PR
comment with the captured screenshot. Drop into
`.github/workflows/capture-evidence.yml` in your iOS app repo.
- [`workflows/capture-pr-on-pr.yml`](workflows/capture-pr-on-pr.yml) —
runs `evidence capture-pr` on every pull request, posts a concise report
comment with the before/after SHAs and status, and uploads the generated
evidence bundle. Drop into `.github/workflows/capture-pr-on-pr.yml` in your
iOS app repo.
- [`workflows/capture-screenshots-on-tag.yml`](workflows/capture-screenshots-on-tag.yml) —
runs `evidence capture-screenshots` against the configured device matrix
whenever a release tag is pushed, then uploads the screenshots as a build
artifact ready for App Store Connect.
- [`workflows/capture-web-on-pr.yml`](workflows/capture-web-on-pr.yml) —
starts a local web server, runs `evidence capture-web` on every pull request,
and posts the captured viewport screenshots as a PR comment.

## Fixture project

Expand Down
40 changes: 40 additions & 0 deletions Examples/workflows/capture-pr-on-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Capture PR evidence

# Drop this file into your iOS app repo at .github/workflows/capture-pr-on-pr.yml.
# It runs the evidence capture-pr flow on every pull request, posts a concise
# report comment, and uploads the full evidence bundle for reviewer download.

on:
pull_request:
branches:
- main

jobs:
capture-pr:
runs-on: macos-14
permissions:
pull-requests: write
contents: read
steps:
- name: Check out app code
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Capture before/after PR evidence
id: capture-pr
uses: RiddimSoftware/evidence@v0
with:
subcommand: capture-pr
plan: .evidence/pr-home.json
output-dir: docs/build-evidence/pr-${{ github.event.pull_request.number }}
comment-on-pr: 'true'
github-token: ${{ secrets.GITHUB_TOKEN }}

- name: Upload PR evidence artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: evidence-pr-${{ github.event.pull_request.number }}
path: ${{ steps.capture-pr.outputs['output-dir'] }}
if-no-files-found: ignore
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,15 +272,16 @@ jobs:
github-token: ${{ secrets.GITHUB_TOKEN }}
```

The Action accepts a `subcommand` input matching the CLI verb (`capture-screenshots`, `capture-evidence`, `capture-pr`, `capture-web`, `resize`, `render-marketing`, `record-preview`, `upload-screenshots`) along with passthrough inputs for `config`, `ticket`, `output-dir`, and `extra-args`. Set `comment-on-pr: 'true'` and pass `github-token` to have the Action post a PR comment listing every artifact produced by the run; the comment step is automatically skipped when no token is supplied or when the workflow does not run on a `pull_request` event.
The Action accepts a `subcommand` input matching the CLI verb (`capture-screenshots`, `capture-evidence`, `capture-pr`, `capture-web`, `resize`, `render-marketing`, `record-preview`, `upload-screenshots`) along with passthrough inputs for `config`, `ticket`, `output-dir`, and `extra-args`. For `capture-pr`, use `plan` plus optional `pr`, `before-ref`, `after-ref`, `keep-worktrees`, and `summary-only`; pull request workflows default `pr`, `before-ref`, and `after-ref` from the GitHub event. Set `comment-on-pr: 'true'` and pass `github-token` to have the Action post a PR comment: standard captures list artifacts, while `capture-pr` summarizes `report.md` with status, before/after SHAs, artifact count, and the report path. The comment step is automatically skipped when no token is supplied or when the workflow does not run on a `pull_request` event.

The `platform` input selects the capture mode: `ios` (default) for iOS simulator captures on macOS runners, or `web` for Playwright Chromium screenshots on any runner (including `ubuntu-latest`). When `platform: web`, Node.js 20 and the Playwright Chromium browser are installed and cached automatically.

ImageMagick and ffmpeg are installed and cached the first time the Action runs on an iOS runner, so warm runs reuse the formula tarballs. The `evidence` CLI itself is built once per release ref and cached under `~/runner.temp/evidence-build/.build`.

Three ready-to-copy workflows live under [`Examples/workflows/`](Examples/workflows/):
Four ready-to-copy workflows live under [`Examples/workflows/`](Examples/workflows/):

- `capture-evidence-on-pr.yml` — captures a screenshot per pull request, posts it as a PR comment, and uploads it as an artifact.
- `capture-pr-on-pr.yml` — captures before/after PR evidence, posts a concise report comment, and uploads the evidence bundle as an artifact.
- `capture-screenshots-on-tag.yml` — captures the full App Store screenshot matrix when you push a release tag.
- `capture-web-on-pr.yml` — starts a local HTTP server and captures Playwright web screenshots on every PR, posting a comment with the results.

Expand Down
6 changes: 6 additions & 0 deletions Sources/EvidenceCLIKit/ActionDefinition.swift
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ public struct ActionDefinitionValidator {
"platform",
"config",
"ticket",
"pr",
"plan",
"before-ref",
"after-ref",
"keep-worktrees",
"summary-only",
"output-dir",
"extra-args",
"comment-on-pr",
Expand Down
58 changes: 58 additions & 0 deletions Tests/EvidenceCLIKitTests/ActionDefinitionTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,64 @@ final class ActionDefinitionTests: XCTestCase {
)
}

func testDefaultRequirementsIncludeCapturePRInputs() {
let requiredInputs = Set(ActionDefinitionValidator.defaultRequirements.requiredInputNames)

for input in ["pr", "plan", "before-ref", "after-ref", "keep-worktrees", "summary-only"] {
XCTAssertTrue(
requiredInputs.contains(input),
"ActionDefinitionValidator.defaultRequirements must include capture-pr input '\(input)'."
)
}
}

func testRepositoryActionYmlWiresCapturePRInputsAndGitHubContextDefaults() throws {
let actionURL = repositoryRoot().appendingPathComponent("action.yml")
let source = try String(contentsOf: actionURL, encoding: .utf8)

for input in ["pr", "plan", "before-ref", "after-ref", "keep-worktrees", "summary-only"] {
XCTAssertTrue(
source.contains("\n \(input):"),
"action.yml must declare the capture-pr input '\(input)'."
)
}

XCTAssertTrue(source.contains("EVIDENCE_PR_INPUT: ${{ inputs.pr }}"))
XCTAssertTrue(source.contains("EVIDENCE_PLAN: ${{ inputs.plan }}"))
XCTAssertTrue(source.contains("EVIDENCE_GITHUB_REPOSITORY: ${{ github.repository }}"))
XCTAssertTrue(source.contains("EVIDENCE_GITHUB_PR_NUMBER: ${{ github.event.pull_request.number }}"))
XCTAssertTrue(source.contains("EVIDENCE_GITHUB_BASE_SHA: ${{ github.event.pull_request.base.sha }}"))
XCTAssertTrue(source.contains("EVIDENCE_GITHUB_HEAD_SHA: ${{ github.event.pull_request.head.sha }}"))
XCTAssertTrue(
source.contains("GH_TOKEN: ${{ inputs.github-token }}"),
"Run evidence must expose github-token to gh for capture-pr metadata reads."
)
XCTAssertTrue(source.contains("args+=(\"--repo\" \"${repo}\")"))
XCTAssertTrue(source.contains("args+=(\"--pr\" \"${pr_number}\")"))
XCTAssertTrue(source.contains("args+=(\"--plan\" \"${EVIDENCE_PLAN}\")"))
XCTAssertTrue(source.contains("args+=(\"--before-ref\" \"${before_ref}\")"))
XCTAssertTrue(source.contains("args+=(\"--after-ref\" \"${after_ref}\")"))
}

func testRepositoryActionYmlCapturePRCommentSummarizesReport() throws {
let actionURL = repositoryRoot().appendingPathComponent("action.yml")
let source = try String(contentsOf: actionURL, encoding: .utf8)

XCTAssertTrue(source.contains("report.md"))
XCTAssertTrue(source.contains("Report status: **${report_status}**"))
XCTAssertTrue(source.contains("Before SHA:"))
XCTAssertTrue(source.contains("before_sha"))
XCTAssertTrue(source.contains("After SHA:"))
XCTAssertTrue(source.contains("after_sha"))
XCTAssertTrue(source.contains("Artifact count: ${EVIDENCE_ARTIFACT_COUNT}"))
XCTAssertTrue(source.contains("Report:"))
XCTAssertTrue(source.contains("report_rel"))
XCTAssertTrue(
source.contains("Overall status:"),
"capture-pr PR comments must be based on the generated report status, not just raw artifact paths."
)
}

func testValidatorRejectsActionMissingRequiredInput() {
let source = """
name: 'evidence'
Expand Down
19 changes: 19 additions & 0 deletions Tests/EvidenceCLIKitTests/ExampleWorkflowsTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,25 @@ final class ExampleWorkflowsTests: XCTestCase {
}
}

func testCapturePROnPRWorkflowIsCopyPasteable() throws {
let root = repositoryRoot()
let workflowURL = root.appendingPathComponent("Examples/workflows/capture-pr-on-pr.yml")
let workflow = try String(contentsOf: workflowURL, encoding: .utf8)

XCTAssertTrue(workflow.contains("on:\n pull_request:"))
XCTAssertTrue(workflow.contains("runs-on: macos-14") || workflow.contains("runs-on: macos-15"))
XCTAssertTrue(workflow.contains("uses: actions/checkout@v4"))
XCTAssertTrue(workflow.contains("fetch-depth: 0"))
XCTAssertTrue(workflow.contains("uses: RiddimSoftware/evidence@v0"))
XCTAssertTrue(workflow.contains("subcommand: capture-pr"))
XCTAssertTrue(workflow.contains("plan: .evidence/pr-home.json"))
XCTAssertTrue(workflow.contains("output-dir: docs/build-evidence/pr-${{ github.event.pull_request.number }}"))
XCTAssertTrue(workflow.contains("comment-on-pr: 'true'"))
XCTAssertTrue(workflow.contains("github-token: ${{ secrets.GITHUB_TOKEN }}"))
XCTAssertTrue(workflow.contains("uses: actions/upload-artifact@v4"))
XCTAssertTrue(workflow.contains("path: ${{ steps.capture-pr.outputs['output-dir'] }}"))
}

private func parseInputNames(from actionSource: String) -> Set<String> {
var names: Set<String> = []
var inInputs = false
Expand Down
Loading
Loading