Move CI integration tests to wp-env, harden and cache the workflows#30
Merged
Move CI integration tests to wp-env, harden and cache the workflows#30
Conversation
Recent dependabot merges exposed two problems with the integration test job: install-wp-tests.sh fails with `svn: command not found` because GitHub's ubuntu-latest runners no longer pre-install Subversion, and the script's host-side MySQL/svn approach has drifted from the plugin standards in CI_WORKFLOWS.md and DEVELOPMENT_ENVIRONMENT.md, which now prescribe @wordpress/env (wp-env). The active a8c plugins (edit-flow, co-authors-plus, liveblog) all run integration tests under wp-env, so the boilerplate should match. Replace the bespoke install-wp-tests.sh with wp-env. Add a `.wp-env.json` describing the plugin source, host PHP version, and a tests env that pins WordPress to the new 6.9 floor. Update `composer.json` so the `integration` script proxies through `wp-env run tests-cli`, and drop the now-unused `prepare` script. Reshape the CI workflow to install wp-env, check the source out into a `plugin-slug/` directory so wp-env mounts at `wp-content/plugins/plugin-slug` (matching the placeholder slug used throughout the boilerplate), and remove the MySQL/prepare steps in favour of `wp-env start`. The integration bootstrap fallback now points at `/wordpress-phpunit`, where wp-env exposes the WP test suite inside the tests-cli container. While restructuring the matrix, raise the supported floors to PHP 8.4 and WordPress 6.9, and replace the dynamic WyriHaximus/composer-php-versions-in-range job with a static min/max matrix following the plugin standards examples: PHP 8.4 against WP 6.9, and PHP 8.5 against WP master. The plugin headers in plugin-slug.php and the `php` constraint in composer.json move in lockstep so the declared minimums match what CI actually exercises.
Cut wasted CI time and tighten the workflow attack surface ahead of the
SHA-pinning audit pass.
Switch composer install steps to ramsey/composer-install, which restores
the vendor cache from previous runs keyed by composer.json (the lockfile
is gitignored, so we cache on the manifest hash). Add an actions/cache
step for ~/.npm so wp-env's tarball download is reused across runs, and
introduce actions/setup-node to give npm a managed Node toolchain.
Apply the workflow security defaults from plugin-standards/CI_WORKFLOWS:
declare top-level `permissions: {}`, grant per-job `contents: read`, set
`persist-credentials: false` on every checkout, and add a concurrency
group so superseded runs on the same branch are cancelled. Replace the
unsafe `actions/checkout@master` reference in the asset/readme workflow
with `@v4`. Keep the 10up actions on `@stable` for now; the follow-up
pinact pass will resolve everything to immutable SHAs.
Floating tags like @v4 or @stable are mutable references that an attacker who compromises the publishing repo (or the publisher's account) can silently repoint. Pinning every action to a full commit SHA, with the human-readable version preserved as a trailing comment, removes that supply-chain risk while keeping Dependabot able to bump versions. Pinact handled the standard actions (actions/checkout, actions/setup-node, actions/cache, ramsey/composer-install, shivammathur/setup-php, ChristophWurst/xmllint-action). The two 10up actions were tracking @stable, which pinact cannot resolve, so they are pinned by hand to the SHAs that @stable currently points at (which match their latest release tags, 2.2.0 and 2.3.0 respectively).
Three categories of finding addressed.
In ci.yml the Setup Problem Matchers step expanded ${{ runner.tool_cache }}
directly into a shell command, which zizmor's template-injection audit flags
because GitHub-context expansion runs before the shell sees the line. Routing
the value through a step-level env var keeps the path identical but moves the
expansion out of the shell command, so a hypothetical poisoned value cannot
turn into executable shell.
The two 10up SVN workflows had no concurrency guard. Two overlapping runs
could each try to commit to the same WordPress.org SVN repository, which is
a recipe for tag clashes and partial deploys. A workflow-scoped concurrency
group with cancel-in-progress: false serialises runs without aborting one
that may already be partway through pushing.
Four secrets-outside-env warnings on SVN_USERNAME / SVN_PASSWORD are left in
place. Resolving them requires creating a GitHub Environment in repo
settings and binding the secrets to it; adding environment: to the YAML
without that setup would break the workflows. Flagged here so it can be
followed up as a repo configuration task.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The integration test job in
ci.ymlstarted failing on every dependabotmerge with
svn: command not found, because GitHub'subuntu-latestrunners no longer pre-install Subversion and the existing
bin/install-wp-tests.shscript relied onsvn coagainstdevelop.svn.wordpress.org. Rather than patch around that, this branchbrings the boilerplate's CI in line with the wp-env-based pattern that
plugin-standards now prescribes and that edit-flow, co-authors-plus, and
liveblog already use.
bin/install-wp-tests.shis gone; in its place is a.wp-env.jsondescribing the plugin source, host PHP version, and a tests env pinned
to WordPress 6.9. The
composer integrationscript now proxies throughwp-env run tests-cli, so contributors get the same environment as CIwith a single
wp-env start. The CI workflow itself drops the hostMySQL service and svn dependency, checks the source out into a
plugin-slug/directory so wp-env mounts the plugin atwp-content/plugins/plugin-slug(matching the placeholder slug usedthroughout the boilerplate), and starts wp-env with
WP_ENV_COREandWP_ENV_PHP_VERSIONset per matrix entry.While restructuring the test job, the supported floors move up to PHP
8.4 and WordPress 6.9 in both
plugin-slug.phpandcomposer.json, andthe previous dynamic
WyriHaximus/composer-php-versions-in-rangematrix is replaced with astatic min/max strategy: PHP 8.4 against WP 6.9 and PHP 8.5 against WP
master.
The remaining commits make the workflows quicker and safer. Composer
installs are now handled by
ramsey/composer-install, which restoresvendor/keyed oncomposer.json(the lockfile is gitignored).actions/setup-nodeplus anactions/cachestep on~/.npmreuseswp-env's tarball download between runs. A workflow-level
concurrencygroup cancels superseded CI runs on the same branch, andthe two SVN-pushing workflows get their own concurrency groups with
cancel-in-progress: falseso a deploy mid-flight is never interrupted.Top-level
permissions: {}plus per-jobcontents: read,persist-credentials: falseon every checkout, and the unsafeactions/checkout@masterreference in the asset/readme workflow swappedout, bring the workflows into line with the security defaults in
plugin-standards/CI_WORKFLOWS.md.Finally, every
uses:reference is pinned to an immutable SHA viapinact, with a version comment for human readability, and zizmor's
pedantic findings are addressed: a
template-injectionwarning onrunner.tool_cacheis routed through a step-level env var. Foursecrets-outside-envwarnings onSVN_USERNAME/SVN_PASSWORDareintentionally left until a GitHub Environment is created in repo
settings to bind them to.
Test plan
test job).
wp-env start && composer integrationworks locally on a freshclone after
composer installandnpm install -g @wordpress/env.the previous
svn: command not foundfailure.