ci: run the core test suite on pull requests - #3910
Draft
arimxyer wants to merge 1 commit into
Draft
Conversation
validate.yml runs bun validate and the SDK suite, but the 111 core tests — including the repository-wide data invariants (base_model hygiene, metadata leakage, weights-link coverage) — never execute in CI, so data regressions those tests guard against land silently. Add the missing step and correct the AGENTS.md claim that no test framework exists. Depends on the open-weights metadata fixes landing first so the weights-link invariant passes on dev. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
validate.ymlrunsbun validateand the SDK suite on every PR, but the 111 core tests — including the repository-wide data invariants (base_modelhygiene, metadata leakage, weights-link coverage, provider-namespace rules) — never execute in CI. Data regressions those tests were written to catch land silently: the weights-link invariant had accumulated 32 violations ondevbefore #3909.Core testsstep (bun testinpackages/core) tovalidate.yml, between validate and the SDK stepAGENTS.mdclaim that no test framework exists (which steers both agents and humans away from running the suite)Dependency
Needs #3909 (open-weights metadata fixes) to land first — the weights-link invariant is red on current
dev, so enabling enforcement before the data fix would break every open PR. With #3909 merged, the core suite is fully green (111 pass / 0 fail, verified locally on that branch).🤖 Generated with Claude Code