test: realign antigravity + pi mapping assertions with pruned references#1987
Conversation
Commit e7ddc25 ('Prune per-harness tool-mapping boilerplate') deliberately removed the skill-loading explainers and generic action->tool tables from antigravity-tools.md and pi-tools.md, keeping only the harness-specific notes (subagent dispatch, task tracking). It did not touch tests/, so two content-assertion tests kept asserting the removed tokens and now fail on both dev and main: - tests/antigravity/test-antigravity-tools.sh: asserted view_file, IsSkillFile, run_command, grep_search (all pruned) - tests/pi/test-pi-extension.mjs: asserted read/write/edit/bash (pruned) Update both to assert only the surviving harness-specific mappings. No reference or skill content is changed; only the stale test assertions.
|
The antigravity half holds up. I stripped the mapping table out of The pi half doesn't hold. All four of the new tokens also occur in the prose, so the assertion survives the thing it exists to detect. On your branch, with the pi mapping table deleted:
So the test passes on a Scoping the match to the table rows closes it: const rows = text.split('\n').filter((line) => line.startsWith('|'));
assert.ok(rows.some((row) => /subagent/i.test(row)), 'mapping table documents subagent dispatch');
assert.ok(rows.some((row) => /todo|task/i.test(row)), 'mapping table documents task tracking');That fails on the stripped file and passes on Disclosure: agent-assisted — Claude Opus 4.8 ( |
The pi tokens (subagent, pi-subagents, Task, TODO.md) also appear in the surrounding prose, so matching the whole file passed even with the mapping table deleted — the exact regression this test exists to catch. Filter to table rows (lines starting with '|') so the assertion fails when the table is gone and passes on dev. Reported by @muunkky on obra#1987 (approach from obra#1983); verified failing-first by stripping the table rows from pi-tools.md.
|
Thanks @muunkky — this is exactly right, and I reproduced it before changing anything. Stripping the table rows from Pushed the table-scoped version ( I checked the antigravity half against the same test: it already fails when the table is stripped ( Disclosure: agent-assisted — Claude Opus 4.8 ( |
Asked whether another team could pick this up. Four things say no, and three of them are live bugs for us too. 1. The PR template was a fabricated-attestation generator. The disclosure table shipped with its answers pre-filled, including 'Human partner who reviewed this diff: Cameron Rout (@muunkky)'. Hand this to another team and their first PR attests a human who never read their diff — the obra#1906 kill, except the account is real. It is also why our own four PRs under-disclosed: the template had answers, so nobody read the machine. Every cell now says how to read it, and the reviewer cell is empty by construction. 2. fork-setup.sh did not exist. SKILL.md told you to keep one and showed its contents in a code block, and nobody had ever created it — so on a fresh clone there is no .git/info/exclude and no fetch-only upstream, and 'git add -A' stages .gitban/ and docs/prds/ into the upstream PR. The guardrail the entire code/artifact split rests on was prose. Documented is not shipped. It now exists, is idempotent, and proves the ignore works with a probe file instead of asserting it. 3. Both audit scripts hardcoded ME=muunkky. Run by anyone else they enumerate our threads, find none of theirs, and print 'clean'. SKILL.md makes exactly this argument about check-upstream's manifest and then hardcodes the account two lines later. Identity now comes from the origin/upstream remotes; missing remotes exit non-zero. 4. The gitban-pr overlay had drifted from its appended copy in SKILL.md. Resynced. Verified: fork-setup rescues a clone that was staging .gitban/card.md and docs/prds/x.md — after it, both are invisible. All three scripts pass shellcheck; the sweep still finds our four PRs and now sees the obra#1987 thread.
Asked whether we were still confident in the open PRs. Checked instead of answering, and obra#1982 — the strongest of the three — was resting on '0 of 3 on dev, 3 of 3 with the change' with no transcript on disk. Re-ran it. n=8 per arm, trees verified to differ by exactly our 9 lines: dev releases the finished subagents 3 of 8 (two agents call TaskStop as an explicit numbered step), ours 8 of 8. Both published figures were wrong in our favour. obra re-runs eval claims across independent sessions; he would have found this. The change still works, but only under the question it actually teaches: 'names the slot as the reason' is 0 of 8 on dev vs 7 of 8 with it. dev's agents do sometimes stop subagents — for context hygiene, never for capacity. That is a better argument than the one we were making, and it is true. Also logged: I miscounted this eval three times with greps, in both directions, including calling the adversarial arm a total failure when it was a total success. Every miscount was caught only by opening the transcripts. That is the same defect we reported to gaurav0107 on obra#1987 the same morning. Grep proposes; reading decides. Body corrected on-thread with the retraction stated plainly.
e7ddc25 (obra#1847) intentionally trimmed the skill-loading explainer and the generic action→tool table from antigravity-tools.md, pruning to "the harness-specific notes that still carry weight." But tests/antigravity still asserted the removed tokens (view_file, IsSkillFile, SKILL.md, run_command, grep_search), so the antigravity suite has been red on dev ever since. Realign the test with the slimmed reference: assert only what the doc still documents — subagent dispatch (invoke_subagent, self/research types), the write_to_file/replace_file_content tools, and the task artifact mechanism — plus the SKILL.md Platform Adaptation link. Drop the assertions for the intentionally-removed tokens. Agents don't need the removed details: verified with a clean agy run where the plugin installs, skills load, and brainstorming auto-triggers without them. Related: obra#1987 independently proposed realigning these assertions (antigravity + pi); this is the narrower antigravity-only change.
|
Backing this one — and I've closed my overlapping PR (#1992) in favor of it. I hit the same red antigravity suite on Reproduced locally:
Honest caveat: I couldn't run pi extension cases 2–5 headlessly (they load the Thanks for taking the pi half down to the table rows too. |
Who is submitting this PR? (required)
claude-opus-4-8[1m]superhuman@superhuman,superpowers@claude-plugins-officialWhat problem are you trying to solve?
The plugin-infrastructure test suite fails on
dev(and onmain, 6.1.1). Two per-harness mapping tests assert content that was intentionally deleted:Root cause: commit
e7ddc25("Prune per-harness tool-mapping boilerplate", 2026-06-24) deliberately removed the skill-loading explainers and the generic action→tool tables fromantigravity-tools.mdandpi-tools.md, trimming each file "to the harness-specific notes that still carry weight (subagent dispatch, task tracking…)". That commit did not touchtests/, so the two content-assertion tests still grep for tokens the prune removed —view_file,IsSkillFile,run_command,grep_search(antigravity) andread/write/edit/bash(pi). With no CI in the repo, the breakage went unnoticed.What does this PR change?
Updates the two orphaned tests to assert only the harness-specific mappings
e7ddc25kept (subagent dispatch and task tracking) instead of the boilerplate it removed. No reference, skill, or product content is touched — only the stale test assertions (and their now-inaccurate header comment / PASS string).Is this change appropriate for the core library?
Yes. These are the repo's own plugin-infrastructure tests under
tests/. No new dependency, no skill-behavior change, no third-party integration — it makes the existing suite green again ondev.What alternatives did you consider?
e7ddc25's stated intent, and the restore-direction attempt PR Document Antigravity tool mappings for skill loading #1902 ("Document Antigravity tool mappings for skill loading") was already closed. Reintroducing the boilerplate would re-break what the maintainer deliberately trimmed.Does this PR contain multiple unrelated changes?
No. One root cause —
e7ddc25orphaned the per-harness mapping-content tests. Both files fail for the identical reason and are fixed the identical way. One problem.Existing PRs
e7ddc25fallout (dead markdown links to the fully-deletedclaude-code-tools.md/copilot-tools.md), not these test assertions. No open PR targets these two test files as a focused fix. (Several large, CONFLICTING, stale sweep-PRs incidentally include these files but do not repair the assertions.)Environment tested
claude-opus-4-8[1m]Tests run locally on macOS (
bash+node --test).New harness support (required if this PR adds a new harness)
N/A — this PR does not add a harness.
Evaluation
Not a behavior/skill change, so there is no eval-session delta to report. Verification instead:
dev):tests/antigravity/test-antigravity-tools.sh→ exit 1;tests/pi/test-pi-extension.mjs→ 5 pass / 1 fail.PASS; pi → 6 pass / 0 fail.tests/antigravity/run-tests.sh(all passed) and the repo shell-lint tests (pass). Confirmed each surviving asserted token is actually present in the pruned reference file.Rigor
devandmain, and confirmed the new assertions match the pruned references.Human review