Add the Rook coding-agent skill - #18
Open
samyakLambda wants to merge 21 commits into
Open
Conversation
Three bash harnesses under scripts/, in the style of the install tests: mirrors stay byte-identical to skill-installer/skills, the frontmatter is named rook and fits the 1024-character description limit, and every rook flag the skill names exists in `rook help` of the pinned 0.1.1. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
…them Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
`GROUPS` was bash's read-only special array (real group ids), so the assignment silently did nothing, `sub` was never set, and every flag was checked as a substring of the whole `rook help <group>` output — letting `rook profile test --what x` pass even though `--what` belongs to `profile fix`. Rename to `GROUP_CMDS`, glob-protect the line tokenizer with `set -f`/`set +f`, and switch both flag checks to a word-boundary match so `--jso` or `--all` can no longer pass as a prefix of a real flag. Refactor the per-line check into `check_line()` so a self-test (`SKILL_FLAGS_SELFTEST=1`) can assert the exact regression against two built-in lines before scanning the skill, and wire that env var into the CI harness-loop step so it runs on every push. Also drop the stale comment claiming `rook <group> <sub> --help` prints the root help — at 0.1.1 it prints that subcommand's own help. Bump the workflow's node-version to 22 to match rook's engines.node (>=22). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
- mcp.md: a target's `.mcp.json` servers discovered by `explore` carry origin `discovered`, not `project` (`project` names the scope file `.testmuai/rook/mcp.json`, which rook itself writes). Only `project` and `discovered` origins wait for `approve`; `user` and `local` are used as declared. - SKILL.md Safety: rook 0.1.1's headless run does not pause for a per-target write-tool confirmation. Point agents at `agent.yaml` under the agent's folder, whose `calls[]` entries carry `write: true`, and get the user's consent before the first run instead. - SKILL.md: note that headless rook refuses any ungranted bash/fetch/MCP call and prints the `--allow` rule to fix it; read tools need no grant. Add a CI note that a fresh runner needs `project use`/`agent use` before `explore`. - `profile test` calls the user's agent once and spends no rook credits — drop it from the costed list in SKILL.md and headless-contract.md's Commands table, and add its missing `[--json] [--verbose]` flags. - headless-contract.md: `explore`, `generate`, `sync` and `profile` keep their text on stdout even under `--json`, matching the JSON-documents section above the Streams bullets. Add `profiles/` and `agents/` as the two sibling directories under an agent's folder that were missing from the on-disk layout. - verdicts.md: name the results-table fields precisely as `totals.executed` and `metrics.credits` from the run's `report.yaml`, and say where the previous run for a diff comes from. - scenarios.md: `--only` takes one comma-separated argument; a second, space-separated id is read as instruction text, not accumulated. - README.md: add a fourth rule for AI coding agents (costed commands), and reformat the exit-code table to the file's compact style. Regenerated `.claude/skills/rook` and `.agents/skills/rook` from the canonical skill via scripts/sync-skill-mirrors.sh. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
The on-disk tree listed manifest.yaml, discovery.yaml, features.yaml and context.md under an agent's folder. rook 0.1.1 does not write them. What it writes there is agent.yaml, features/<feature-id>.yaml and findings.yaml, plus the scenarios/, runs/, profiles/ and agents/ directories; each run folder also gets an agent.yaml, features.yaml and profile.yaml snapshot. The tree now lists those. The README's agent section gained a fourth rule about credits in the previous commit; its lead-in now says so. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
…fety order Four findings from a Codex review of the branch, each verified against rook 0.1.1 before the fix: - consent before the first command that reaches the agent (profile add, profile fix, profile test), not only before run; profile test gets a reply-only goal - scenarios exclude takes IDs as separate arguments; a comma-joined string is one unknown ID and the document still says ok: true - report --rca --json emits no document; follow it with rook report <run-id> --json for the explained report - a gate refusal on report, status or ask exits 1 with empty stdout; the error is the last stderr line, translated via troubleshooting.md Mirrors re-synced; mirror and flag harnesses pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0125aHyCFGXLcDNe5vrMsi6C
Distinguish refused and incomplete runs from completed evidence, gate CI on the current run's report, and add synthetic coverage for the documented recipe. Clarify target consent, command-specific output, credits, and saved results. Document project installation through the skills CLI, tighten triggering, and keep the Claude Code and Codex mirrors identical to the canonical skill.
Use behavioral evaluation findings to preserve unknown metadata and halted counts, choose profiles from actual target configuration, and inspect read-only verification before predicting unverifiable results.
Explain zero-agent discovery recovery and preserve both reported credits and account balance changes without inventing a billing explanation. Make the absence of a CLI aggregate credit-cap flag explicit.
Preserve recorded verdicts when an empty execute call array or JSON-text raw response hides evidence from the checker.
This was referenced Sep 9, 2026
samyakLambda
changed the title
samyakLambda
marked this pull request as ready for review
September 10, 2026 13:59
samyakLambda
requested review from
gauravBansal06 and
vipulkv
as code owners
September 10, 2026 13:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a general Rook coding-agent skill for setup, scoped testing and evidence-based reporting. One canonical bundle supplies Claude Code and Codex mirrors, with focused references for profiles, scenarios, MCP, CI and troubleshooting.
Use installed CLI help and validate actual output fields. Compatible minor/patch releases need no skill update; refresh guidance when commands, workflows or output contracts change materially. Target authorization, credit tracking and separate Pass/Fail/Unable to Verify results remain required.
Validation: 48 command examples pass on Rook 0.1.1 and 0.1.2; all 16 offline CI contract cases and mirror checks pass. The CI recipe accepts compatible newer versions while rejecting incomplete or malformed results. Repository CI checks the current published CLI.
Earlier guided trials on #17 completed four scenarios with one Fail from a generated regex and three Unable to Verify results from trace/JSON limits. Full live behavior on 0.1.2 was not rerun.
Companions: #19 adds the user guide; #20 adds npm installation/publication.