docs: add visual regression testing guide#266
Conversation
Recommends @wdio/visual-service as the VRT solution for both @wdio/electron-service and @wdio/tauri-service. The doc covers the small amount of glue needed (recommended config for both services, per-OS baseline convention, the Playwright-style stabilisation recipe) and the two non-obvious caveats: - Tauri providers produce structurally different captures (embedded is webview-only; CrabNebula's macOS driver captures the OS window including native chrome via Screen Recording). Per-provider baseline directories are mandatory. - Two known platform incompatibilities — `official` driver + Linux + visual-service hangs in the service's `before()` hook; CrabNebula + macOS-CI can't satisfy TCC's Screen Recording permission programmatically — both with documented workarounds. The doc deliberately defers the visual-service API surface to the upstream docs and links the wdio-desktop-mobile-example repo for a working CI matrix. Cross-linked from the root README docs index, the per-package READMEs (electron-service Operations, tauri-service Guides), and a new "Cross-cutting Capabilities" section in ROADMAP.md so users searching for VRT support find a clear answer. Companion to the spike validated in goosewobbler/wdio-desktop-mobile-example#123. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Release Preview — no release
Updated automatically by ReleaseKit |
Greptile SummaryAdds
Confidence Score: 5/5Documentation-only PR; no runtime code is changed. Safe to merge. All five changed files are Markdown documentation or README updates. No source code, tests, or build configuration are touched. All internal cross-links resolve to existing files, and the new guide's config snippets are consistent with how both services register themselves in WDIO. No files require special attention.
|
| Filename | Overview |
|---|---|
| docs/visual-testing.md | New guide covering VRT setup for Electron and Tauri. The browser.execute(async () => { await document.fonts.ready; }) pattern is still technically incorrect because WebDriverIO's Execute Script does not await Promises — flagged in a previous thread. The idempotency guard on the style injection has been addressed in this version. |
| README.md | Adds docs/visual-testing.md entry to the docs index table. Path and description are correct. |
| ROADMAP.md | Adds a "Cross-cutting Capabilities" section with VRT and video recording rows. Links and content are accurate. |
| packages/electron-service/README.md | Adds a Visual Testing pointer under Operations. Relative path ../../docs/visual-testing.md resolves correctly from this location. |
| packages/tauri-service/README.md | Adds a Visual Testing pointer under Guides. Relative path ../../docs/visual-testing.md resolves correctly from this location. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Test Run] --> B{Local or CI?}
B -- Local autoSaveBaseline=true --> C[First run: write baseline to __visual__/platform/arch/provider/]
B -- CI autoSaveBaseline=false --> D{Baseline exists?}
C --> E[Second run: compare screenshot to baseline]
D -- No --> F[Fail loudly missing baseline]
D -- Yes --> E
E --> G{Mismatch %}
G -- lte MAX_MISMATCH_PCT 1% --> H[Pass]
G -- gt MAX_MISMATCH_PCT --> I[Fail visual diff artifact]
Reviews (6): Last reviewed commit: "docs(visual): note ESM equivalent for __..." | Re-trigger Greptile
Greptile review on PR #266: the animation-suppression snippet appended a fresh <style> on every call. The doc tells readers to run it before every checkScreen / checkElement call, which naturally leads to a beforeEach hook — without an idempotency guard, that accumulates redundant <style> nodes across tests. Add an id check so calling this repeatedly is a no-op after the first invocation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two issues from the latest Greptile pass on PR #266: 1. Unused `$` import in the spec example. Extended the spec with a second test case that does element-level checking via `browser.checkElement($('header.toolbar'), ...)`. This uses the imported `$` and demonstrates more of the visual-service API (full-screen + element scoping) for readers copying the snippet. 2. `browser.execute(() => document.fonts.ready)` returned an unawaited Promise from the lambda. WDIO 9 does await Promise return values implicitly, but the snippet is cleaner if the await is explicit so readers don't have to know that. Switched both call sites to an `async () => { await document.fonts.ready }` form. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Greptile flagged that the Electron and Tauri config snippets use `__dirname`, which isn't defined in ES module configs (`"type": "module"` in package.json, or `wdio.conf.mts`). Add a small callout between the snippets and the spec example showing the `import.meta.url` derivation so ESM users aren't tripped up. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs: add video recording guide Recommends wdio-video-reporter for capturing video of test runs against @wdio/electron-service and @wdio/tauri-service. Companion to the visual-testing doc — video is for debugging, visual is for regression detection. The guide covers the small amount of glue needed (recommended config for both services, per-OS output convention, retention modes) and is deliberately honest about what `wdio-video-reporter` actually does: screenshot stitching at command granularity, not a real video pipeline. So the output is a 3–10 fps slideshow suitable for "what happened during this failed test?" debugging, not smooth screencast. Surprising-positive finding from the spike (PR #124 in wdio-desktop-mobile-example): tauri-official + Linux works here, even though the same cell is broken for @wdio/visual-service. The reporter has no before() initialisation script, so it sidesteps the executeAsync hang. Documented in the Tauri provider notes. Cross-linked from electron-service README Operations and tauri-service README Guides, mirroring the visual-testing cross-links established in PR #266. Companion to the spike validated in goosewobbler/wdio-desktop-mobile-example#124. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(video): clarify output path for Tauri's per-provider layout The setup-finished description named only the Electron output path. Tauri users following the Tauri config snippet (which adds a `<provider>` segment) would look for the file at the documented path and not find it. Split into two bullets so each service's actual layout is on the page. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: trim and service-agnosticise visual-testing and video-recording guides Replace parallel Electron/Tauri config blocks with a single service-agnostic snippet, foregrounding what's unique to this monorepo — the Tauri provider × OS matrix and the known issues (CrabNebula × macOS-CI, official × Linux executeAsync hang) — and trimming the install/wire-in/spec boilerplate that paraphrased upstream docs. Both guides now share the same shape so they're scannable side-by-side. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: reframe visual-testing and video-recording around interop-as-default Drop the "Both @wdio/electron-service and @wdio/tauri-service compose with X" opener in favour of "WDIO's service architecture lets services compose freely — no special integration needed." Framework names move from canonical enumerations to illustrative examples (e.g. mock-API snippets, output-path exceptions). New framework services landing from the roadmap won't need doc updates to be acknowledged as supported — only framework-specific sections (e.g. Tauri provider notes) gain content. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: address review feedback on visual-testing and video-recording - Default config snippets to ESM (drop CJS + workaround callouts) - Inline subdirs in snippets so they run as-is; note Tauri's driverProvider segment in a comment - Rewrite Dioxus VRT section: drop empty provider table, lift Xvfb to a shared Linux subsection (Tauri + Dioxus, both wry → WebKitGTK, 16-bit colour depth) - Drop visual-service hang reference from video doc's Tauri table (out of context) - Generalise the video "what you get" capture bullet across all webview-scoped providers - Swap placeholder Dioxus mock example for clipboard_read (native-surface parallel to Electron Menu / Tauri dialog) - Add Video Recording link to Dioxus README for parity with Electron/Tauri - Reword the mock-APIs closing line to be matter-of-fact Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: refine review fixes — autoXvfb credit, drop redundant gitignore step - Credit WDIO's autoXvfb feature for Electron's Linux display handling (was wrongly attributed to Electron itself) - Drop "Ignore the output directory" as a numbered Quick start step; fold the __video__ gitignore note into Output paths Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Summary
Adds a single new doc —
docs/visual-testing.md— that recommends@wdio/visual-serviceas the VRT solution for both@wdio/electron-serviceand@wdio/tauri-service. The doc is deliberately brief: it covers the small amount of glue needed (recommended config snippets for both services, per-OS baseline convention, the Playwright-style stabilisation recipe) and the two non-obvious caveats. Everything else defers to the upstream visual-service docs and the working example repo.What's covered
__visual__/<platform>/<arch>/[<provider>/]...withautoSaveBaseline: !process.env.CI.embedded(webview-only),official(webview-only on Windows; known hang on Linux),crabnebula(OS-window with native chrome on macOS, requires Screen Recording permission). Per-provider baselines explained.browser.electron.mock,browser.tauri.mock) for menu/tray/dialog assertions.Cross-linking
README.md→ docs index gets the new entry.packages/electron-service/README.md→ "Operations" section gets a Visual Testing pointer.packages/tauri-service/README.md→ "Guides" section gets a Visual Testing pointer.ROADMAP.md→ new "Cross-cutting Capabilities" section so users searching the roadmap for VRT find a clear answer (resolved → use@wdio/visual-service; video recording flagged as a separate future track).Companion / context
goosewobbler/wdio-desktop-mobile-example#123— full CI matrix run across electron-builder/forge/script + tauri (embedded/official/crabnebula) on macOS / Linux / Windows.feat: native OS window screenshotapproach — see that PR's closing comment for why we pivoted away from rolling our own native screenshot capture.Test plan
docs/visual-testing.mdon GitHub for table layout, code block syntax highlighting, and link targets.packages/electron-service/docs/electron-apis.md,api-reference.md,packages/tauri-service/docs/usage-examples.md,api-reference.md).🤖 Generated with Claude Code