Skip to content

[EVI-8]: add capture-pr examples and fixtures#40

Merged
riddim-developer-bot[bot] merged 1 commit into
mainfrom
symphony/evi-8-add-examples-docs-and-test-fixtures-for-capture
May 18, 2026
Merged

[EVI-8]: add capture-pr examples and fixtures#40
riddim-developer-bot[bot] merged 1 commit into
mainfrom
symphony/evi-8-add-examples-docs-and-test-fixtures-for-capture

Conversation

@riddim-developer-bot
Copy link
Copy Markdown
Contributor

Reviewer-Boundary: review-only

Summary

  • Add a generic Examples/pr-change-evidence-plan.json fixture for capture-pr using the launch-only simctl runner.
  • Expand README and Examples docs with the capture-pr use case, local CLI and GitHub Action examples, output artifacts, and XCTest harness limitation.
  • Extend troubleshooting for stale PR SHAs, simulator availability, missing XCTest harnesses, missing GitHub tokens, dependency failures, and empty artifact outputs.
  • Add tests that parse the sample PR plan and actionlint example workflows when actionlint is available.

Verification

  • swift build — passed.
  • swift test — passed: 120 tests, 0 failures, 2 skipped because node was not found for optional web integration tests.
  • swift test --filter ExampleWorkflowsTests — passed: 5 tests, including plan parsing and actionlint-backed workflow validation.
  • actionlint Examples/workflows/*.yml .github/workflows/*.yml — passed.
  • git diff --check — passed.

Notes

  • No generated screenshots, videos, app-specific assets, or EPAC-only plans are included.

@riddim-developer-bot riddim-developer-bot Bot added the autonomous Autonomous agent PR label May 18, 2026
@linear-code
Copy link
Copy Markdown

linear-code Bot commented May 18, 2026

EVI-8 Add examples docs and test fixtures for capture-pr

Context / background

Parent Project: Evidence PR Change Proof MVP (0c4adda0-5d2d-4ee1-b4bd-717499212ec6). The MVP only helps autonomous teams if they can discover and integrate it without asking the original author how it works. Evidence already has README examples and workflow examples; capture-pr needs the same level of self-contained documentation and fixture coverage.

Acceptance criteria

  • README.md explains the capture-pr use case in plain language: demonstrate what a PR changed with before/after screenshots, videos, manifest, and report.
  • README.md includes a local CLI example and a GitHub Action example.
  • README.md documents the MVP limitation that arbitrary UI actions require an app-side XCTest Evidence harness; launch-only simctl plans support only launch/openURL/wait/screenshot/video.
  • Examples/ contains a sample PR evidence plan JSON file with at least two steps.
  • Examples/workflows/ contains the final capture-pr-on-pr.yml workflow introduced by the action issue or updates it if it already exists.
  • docs/troubleshooting.md includes common failure cases: old SHA no longer builds, simulator unavailable, app lacks XCTest harness, missing ImageMagick, missing ffmpeg, missing GitHub token, and no artifacts produced.
  • Tests assert that sample plans and workflows stay parseable and actionlint-compatible.
  • No generated app-specific screenshots, videos, or EPAC-only plans are committed.
  • swift test passes.

External validation gates

None. Human acceptance for the whole MVP is consolidated in the Project-level human handoff issue.

Out of scope

  • Implementing missing CLI behavior.
  • Adding consuming app repository integration.
  • Publishing documentation externally beyond files in this repository.
  • Creating Linear or GitHub comments automatically.

Inputs / dependencies

Blocked by: EVI-7.

Target repository: RiddimSoftware/evidence.

Existing files to inspect:

  • README.md
  • docs/troubleshooting.md
  • Examples/README.md
  • Examples/workflows/
  • Tests/EvidenceCLIKitTests/ExampleWorkflowsTests.swift

Documentation should use RiddimSoftware/epac#479 as a narrative-style motivating example only if no EPAC-specific files or assumptions are committed. Keep actual reusable examples generic.

Risks / notes for implementer

  • Avoid overstating what MVP can do. The docs must make the XCTest harness requirement explicit.
  • Do not create a separate tutorial that drifts from README command names. Prefer one canonical command example and link back to it.
  • Keep examples copy-pasteable but clearly marked as placeholders where app-specific scheme/bundle identifiers are required.

Definition of Done

Done when a new consuming repo maintainer can read the README, copy an example workflow and plan, understand the app-side harness requirement, and know how to debug the most common failures without asking the Evidence team.

Architecture Impact

Documentation/examples only. They should reflect the architecture created by earlier issues rather than introducing new runtime behavior.

Clean Architecture Shape

Use case:

  • N/A - documentation and examples only.

Entities / value objects:

  • N/A.

Ports:

  • N/A.

Adapters:

  • N/A.

Boundary rule:

  • N/A - no runtime code should be introduced beyond fixtures/tests needed to keep documentation honest.

Catalog update:

  • N/A.

Mergeability / change ownership

Single reason to change: document and fixture the completed capture-pr MVP. Primary code owner: README/docs/examples. Expected hot files/modules: README.md, docs/troubleshooting.md, Examples/, Tests/EvidenceCLIKitTests/ExampleWorkflowsTests.swift. Sibling collisions: EVI-7 may edit workflows first; this issue should land after it. Conflict risk: low after EVI-7. Sequencing lane: integration.

Review in Linear

@riddim-reviewer-bot riddim-reviewer-bot Bot added the agent:working Symphony developer-bot lease (not a CI required check) label May 18, 2026
@riddim-developer-bot riddim-developer-bot Bot enabled auto-merge (squash) May 18, 2026 18:45
Copy link
Copy Markdown

@riddim-reviewer-bot riddim-reviewer-bot Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReviewAutonomousPR

  • Verdict: approve
  • Reviewer boundary: review_only
  • Acceptance criteria coverage: covered=9, missing=0, unclear=0

Summary

PR satisfies the EVI-8 documentation and fixture acceptance criteria. The sample plan is generic and parse-tested, the capture-pr workflow exists and is actionlint-covered, README/troubleshooting cover the MVP limits and common failures, and GitHub checks report swift-test and actionlint success.

Actionable findings

  • None.

Acceptance criteria coverage

  • covered — README explains capture-pr use case with before/after screenshots, videos, manifest, and report.
    • README now describes reviewer-oriented before/after PR evidence and lists the expected output artifacts.
    • Actionability: none
    • Evidence: README.md Pull Request Change Evidence section
  • covered — README includes a local CLI example and a GitHub Action example.
    • README includes both a multiline local evidence capture-pr invocation and an iOS GitHub Actions example using the composite action.
    • Actionability: none
    • Evidence: README.md capture-pr CLI and CI examples
  • covered — README documents MVP limitation requiring an app-side XCTest Evidence harness for arbitrary UI actions.
    • README explicitly distinguishes launch-only simctl support from arbitrary UI actions requiring runner = "xctest" plus an app-side harness.
    • Actionability: none
    • Evidence: README.md Pull Request Change Evidence section
  • covered — Examples/ contains a sample PR evidence plan JSON file with at least two steps.
    • Examples/pr-change-evidence-plan.json contains a generic simctl plan with six steps.
    • Actionability: none
    • Evidence: Examples/pr-change-evidence-plan.json
  • covered — Examples/workflows/ contains final capture-pr-on-pr.yml workflow or updates existing one.
    • The head revision contains Examples/workflows/capture-pr-on-pr.yml; this PR also keeps tests asserting its expected copy-pasteable shape.
    • Actionability: none
    • Evidence: Examples/workflows/capture-pr-on-pr.yml and ExampleWorkflowsTests
  • covered — docs/troubleshooting.md includes common capture-pr failure cases.
    • Troubleshooting now covers old SHAs, simulator availability, missing XCTest harness, missing ImageMagick, missing ffmpeg, missing GitHub token, and empty artifact outputs.
    • Actionability: none
    • Evidence: docs/troubleshooting.md
  • covered — Tests assert sample plans and workflows stay parseable and actionlint-compatible.
    • Tests load the sample PR plan through PRChangeEvidencePlan.load and run actionlint over example workflows when actionlint is installed; CI also reports actionlint success.
    • Actionability: none
    • Evidence: Tests/EvidenceCLIKitTests/ExampleWorkflowsTests.swift; PR status checks
  • covered — No generated app-specific screenshots, videos, or EPAC-only plans are committed.
    • Changed files are docs, tests, a generic JSON fixture, and examples only; no binary/generated evidence artifacts or EPAC-specific plans were added.
    • Actionability: none
    • Evidence: PR file list
  • covered — swift test passes.
    • GitHub status check build-and-test passed for the PR head SHA.
    • Actionability: none
    • Evidence: PR statusCheckRollup for f9d86ba

@riddim-developer-bot riddim-developer-bot Bot merged commit fc65484 into main May 18, 2026
5 checks passed
@riddim-developer-bot riddim-developer-bot Bot deleted the symphony/evi-8-add-examples-docs-and-test-fixtures-for-capture branch May 18, 2026 18:48
@riddim-reviewer-bot riddim-reviewer-bot Bot removed the agent:working Symphony developer-bot lease (not a CI required check) label May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autonomous Autonomous agent PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants