Run the Go test suites verbosely so CI shows what executed - #167
Open
stevebeattie wants to merge 1 commit into
Open
Run the Go test suites verbosely so CI shows what executed#167stevebeattie wants to merge 1 commit into
stevebeattie wants to merge 1 commit into
Conversation
stevebeattie
enabled auto-merge
August 26, 2026 06:17
A bare `go test` prints only `ok <package>`, which cannot distinguish a suite
that ran from one that silently covered less than expected. A newly added
table row or fixture that never runs looks exactly like one that passes, and
that is not a hypothetical failure mode here: the offline matrix pairs
fixtures with package-level blobs that Go does not report as unused, so a row
dropped in a refactor leaves a compiling, green, quieter suite.
Add -v to test-offline and validate_mirrors. The logs now name every test and
subtest, so coverage is checkable after the fact rather than assumed:
- the offline matrix lists its 47 rows by name
- the mirror check reports "compared 7 standalone OVAL file(s)" and which
definitions exist only in the datastream, which were previously only
visible when running it locally
That last point is what prompted this: chainguard-dev#164 passes the mirror check in CI, but
its log showed only `ok`, so the thing the PR is about — the file count going
from 7 to 8 — could not be confirmed from the run.
Costs 846 log lines for the full offline suite, which is a fair trade for
being able to tell a green run from a quiet one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
stevebeattie
force-pushed
the
verbose-go-tests
branch
from
August 26, 2026 16:24
934ddf6 to
9ff0d38
Compare
stevebeattie
disabled auto-merge
August 26, 2026 16:27
stevebeattie
marked this pull request as draft
August 26, 2026 16:27
stevebeattie
marked this pull request as ready for review
August 26, 2026 18:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A bare
go testprints onlyok <package>. That cannot distinguish a suitethat ran from one that silently covered less than expected — a newly added
table row or fixture that never runs looks exactly like one that passes.
Not hypothetical here: the offline matrix pairs fixtures with package-level
blobs, which Go does not report as unused, so a row dropped in a refactor leaves
a compiling, green, quieter suite. Nothing in the log would say so.
The immediate prompt was #164. It passes the mirror check in CI, but the log
showed only
ok— so the thing that PR is about, the compared-file countgoing from 7 to 8, could not be confirmed from the run at all.
Change
-vontest-offlineandvalidate_mirrors. One-line each, plus a commentexplaining why.
What the logs now show
The matrix row count and the mirror check's file count are both now readable
from a CI run, so a drop in either is visible rather than inferred.
Verification
make validate_mirrorsandmake test-offlineboth pass: 169 named subtests,47 matrix rows, 0 failures. The full offline log grows to 846 lines, which seems
a fair trade for being able to tell a green run from a quiet one.
Note
Touches the
Makefile, as does #166. The edits are in different regions(
lint-goversus these two recipes) so they should merge cleanly in eitherorder, but worth knowing if both are in flight.
🤖 Generated with Claude Code