Skip to content

feat: run Playwright accessibility scan on pull requests - #1580

Open
coseeian wants to merge 1 commit into
processing:mainfrom
coseeian:a11y-ci-scan
Open

feat: run Playwright accessibility scan on pull requests#1580
coseeian wants to merge 1 commit into
processing:mainfrom
coseeian:a11y-ci-scan

Conversation

@coseeian

@coseeian coseeian commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Goal

This PR wires the existing Playwright + axe-core a11y suite into CI so regressions get caught on PRs.

Notes for reviewers

  • The scan currently covers the EN homepage (both desktop/mobile). We'll expand coverage to other pages once we've monitored the CI stability on PRs from other contributors for a short period.
  • By default, axe-core runs automatic accessibility check rules tagged for WCAG 2.0 and WCAG 2.1 at Level A and Level AA, along with selected best practices.
  • A failed scan generates a GitHub Action summary (sample) containing the scan results and downloadable artifacts. The default message format should be understandable, but I'm happy to refine the report output for other contributors down the line if we see a need.
  • The check is currently blocking - any new axe violations will fail the build. Once we confirm the CI is stable, I’d suggest we consider making this newly added action a required check in the branch protection rules, so PRs can’t be merged with violations.

@coseeian coseeian added the Accessibility: Best Practice No obvious current impact, but which may lead to problems in the future label Aug 20, 2026
@ksen0

ksen0 commented Aug 25, 2026

Copy link
Copy Markdown
Member

cc @Vaivaswat2244 if you have any thoughts on this one too, please share!

@Vaivaswat2244

Copy link
Copy Markdown

Hi folks! Thank you @coseeian for putting this up. Its really helpful. The CI vs local split in the config reads clean.

I looked at the report in your last run's artifact. The two tests took about 3s each, but the whole Playwright run was 425s. So roughly 422s of it is npm run build && npm run preview starting up before any test runs.

test.yml already has a build job running npm run build with the same env vars, so I think we end up building the site twice on every PR. I think one option to avoid this is to upload dist/ from the existing build job and have this job download it and just run npm run preview. That would need the webServer command to be overridable. Other than that the report looks great.

@Vaivaswat2244

Copy link
Copy Markdown

One more thing I had in mind, since the build is almost all of the runtime, scanning a few more pages only costs a few seconds each.A reference page, a tutorial page and a listing page would cover markup the homepage won't. It's just extra page.goto calls, the server will already serve the whole site. Might be better to add them now, before this becomes a required check, since any existing violations there are easier to deal with early.

On that, I'd suggest continue-on-error: true for the first couple of weeks. We still get the signal, without a red check on people who haven't seen axe output before, and turning it back on is one line. Thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Accessibility: Best Practice No obvious current impact, but which may lead to problems in the future

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants