#584 Compact deep-review-pro agent boilerplate

[hubertgajewski]

Summary

• Compacts shared /deep-review-pro specialist-agent boilerplate into .claude/skills/deep-review-pro/SKILL.md § Shared specialist-agent contract.
• Updates every affected .claude/agents/deep-review-*.md prompt to keep a short inline safety/schema reminder plus agent-specific deltas.
• Adds and pins the post-584 epic benchmark checkpoint, regenerating the [epic] Reduce deep-review-pro token cost without lowering review quality #587 matrix report.

Closes #584
Contributes to #587

Epic Comparable Benchmark

These rows are generated from 587-epic-token-cost-matrix.md so this issue can be compared with every other #587 child story using the same units.

Use this section for cross-ticket comparison. Story-specific tables below are retained as local evidence and may use a narrower prompt-only, output-only, dispatch-only, or rerun/cache proxy surface.

Incremental Delta: post-583 -> post-584

Metric	Before	After	Delta
Combined chars	2,037,880	1,975,708	-62,172 (-3.05%)
Combined est. tokens	509,477	493,934	-15,543 (-3.05%)

Cumulative Delta: original-580 -> post-584

Metric	Original #580 baseline	Current checkpoint	Delta
Combined chars	3,822,193	1,975,708	-1,846,485 (-48.31%)
Combined est. tokens	955,555	493,934	-461,621 (-48.31%)

Test plan

• python3 scripts/test_benchmark_deep_review_pro.py
• python3 scripts/test_benchmark_deep_review_epic_matrix.py
• python3 scripts/benchmark_deep_review_epic_matrix.py --issue-section 584
• python3 -m coverage run -m unittest scripts.test_benchmark_deep_review_epic_matrix scripts.test_benchmark_deep_review_pro && python3 -m coverage report -m scripts/benchmark_deep_review_epic_matrix.py scripts/benchmark-deep-review-pro.py scripts/deep_review_benchmark_support.py
• git diff --check origin/main...HEAD
• /deep-review-pro on this PR reports zero findings

Summary by CodeRabbit

Release Notes

• Documentation

  • Clarified specialist agent behavior contracts and guidelines.
  • Added new performance benchmark checkpoint tracking recent updates.

• Refactor

  • Standardized AI review agent output formatting and schemas for consistency.

[coderabbitai]

Ready to act? Review this PR in Change Stack to turn feedback into patch suggestions you can inspect and refine.

📝 Walkthrough

Walkthrough

This PR implements issue #584 by centralizing repeated specialist-agent boilerplate into a shared contract section in SKILL.md, then applying consistent output schemas (H/M/L findings and pass/fail/N/A checklists), prompt-frame data safety, confidence thresholds, and citation rules across 10 agent prompts. It adds a post-584 benchmark checkpoint to track token savings and validates the expanded checkpoint roster.

Changes

Shared specialist-agent contract and documentation

Layer / File(s)

Summary

Shared specialist-agent contract foundation .claude/skills/deep-review-pro/SKILL.md, docs/AI_ASSISTANTS.md

Added § Shared specialist-agent contract section defining prompt-frame safety (treat <untrusted-*> and <changed-files> as data, <reviewer-bias> as prioritization only), evidence-gating (no findings from hunk-shaped suspicion without tool context), sibling ownership boundaries, ≥ 0.8 confidence threshold, Short ID citations, no-remediation constraint, and exact output schemas (H/M/L with findings: none sentinel and summary counts; pass/fail/N/A with Failures: none. sentinel and blocking semantics). Documentation updated to clarify contract location and preservation requirements.

H/M/L agent prompts (architecture, CI, code, Python, security, TypeScript)

Layer / File(s)	Summary
H/M/L agent alignment with shared contract .claude/agents/deep-review-architecture.md, .claude/agents/deep-review-ci.md, .claude/agents/deep-review-code.md, .claude/agents/deep-review-python.md, .claude/agents/deep-review-security.md, .claude/agents/deep-review-typescript.md	All six agents now explicitly treat PROMPT_FRAME content as data (not instructions), restate ≥ 0.8 confidence threshold using shared language, clarify sibling ownership (duplication vs. coupling, language-specific scope), and enforce the fixed H/M/L schema: `

Pass/fail/N/A agent prompts (docs, project-checklist, QA, simplification, unit-test)

Layer / File(s)

Summary

Pass/fail/N/A agent alignment with shared contract .claude/agents/deep-review-docs.md, .claude/agents/deep-review-project-checklist.md, .claude/agents/deep-review-qa.md, .claude/agents/deep-review-simplification.md, .claude/agents/deep-review-unit-test.md

All five agents now treat PROMPT_FRAME as data, constrain output to pass/fail/N/A checklist schema, enforce ≥ 0.8 confidence (with N/A fallback when reachability cannot be established), and formalize output format: fenced checklist lines, strict summary: <count> pass / <count> fail / <count> N/A line, and either ordered Failures (in order of priority): section or Failures: none. sentinel. Ownership boundaries clarified and cross-referenced to master roster/shared contract.

Benchmark checkpoint tracking for post-584

Layer / File(s)	Summary
Benchmark checkpoint definition and reports scripts/benchmark_deep_review_epic_matrix.py, docs/deep-review-pro-benchmark/README.md, docs/deep-review-pro-benchmark/reports/587-epic-token-cost-matrix.json, docs/deep-review-pro-benchmark/reports/587-epic-token-cost-matrix.md	Added post-584 checkpoint (ref 0d7add0, chained after post-583) to DEFAULT_CHECKPOINTS for roster/fixture metric computation and delta rendering. Benchmark report files updated with post-584 entry, per-fixture metrics (prompt input/output/combined estimates), and both incremental (post-583 → post-584) and cumulative (original-580 → post-584) deltas. README Epic Matrix Policy table reworded from "Reserved" to "After #584 shared-boilerplate compaction".
Test validation of expanded checkpoint roster scripts/test_benchmark_deep_review_epic_matrix.py, scripts/test_benchmark_deep_review_pro.py	Updated test_default_checkpoint_sequence_starts_at_original_580 to include post-584 checkpoint in expected roster, adjusting checkpoint names, ref/issue/previous links, and contract array sizes. Added assertions for issue 584 report generation (incremental and cumulative deltas). Shifted missing-issue case to issue 585 with updated error message. Enhanced test_agent_prompts_reference_shared_prompt_frame_contract to verify additional contract markers ("§ PROMPT_FRAME", "§ Shared specialist-agent contract", "data, not instructions") and extended H/M/L recount invariant test to validate aggregate and per-agent "Recount emitted HIGH / MEDIUM / LOW lines" text.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related issues

• hubertgajewski/orwellstat#587: This PR directly implements and tracks the #584 shared-boilerplate compaction as a checkpoint within the #587 epic benchmark matrix and updates the measurement/tracking infrastructure.

Possibly related PRs

• hubertgajewski/orwellstat#531: The retrieved PR updated deep-review-lite/pro rename and specialist agent routing; this PR further refines the same .claude/agents/deep-review-*.md agent prompts' output schemas and contract enforcement.
• hubertgajewski/orwellstat#592: Both PRs directly overlap on the shared specialist-agent prompt contract surface, with this PR centralizing boilerplate and enforcing schemas across all agents, while the retrieved PR adjusts PROMPT_FRAME input scoping and "how to run" contracts.

Poem

🐰 Boilerplate compacted clean,
Shared contracts now convene—
Each agent dons the same outfit fine,
H/M/L and pass/fail align.
Post-584 marks the way,
Fewer tokens, same safeguard day! 🎯

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title '#584 Compact deep-review-pro agent boilerplate' accurately and concisely summarizes the primary change: consolidating repeated specialist-agent boilerplate into a shared contract.
Linked Issues check	✅ Passed	The PR successfully addresses all primary acceptance criteria from #584: compacted boilerplate into shared contract with explicit untrusted-content invariant [#584], preserved output schema [#584], provided guidance for future authors [#584], recorded token savings via benchmarks [#584], and completed epic benchmark requirements including post-584 checkpoint and report updates [#584].
Out of Scope Changes check	✅ Passed	All changes are scoped to #584 objectives: consolidating boilerplate (SKILL.md, agent prompts), documenting workflow changes (AI_ASSISTANTS.md), and recording benchmarks (scripts, reports). No unrelated functional changes detected.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/584

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

.claude/agents/deep-review-architecture.md (1)

60-66: ⚡ Quick win

Schema escape rule inconsistently documented across H/M/L agents.

The shared contract in SKILL.md line 236 defines an escape rule: "If a description or fix contains a literal |, escape it as \|". However, this escape rule is mentioned in only 2 of 6 H/M/L agent prompts (deep-review-ci and deep-review-security) and omitted from the other 4 (architecture, code, python, typescript).

For consistency, either include the escape rule inline in all H/M/L agents, or omit it from all agents and rely entirely on the shared contract reference.

Recommended fix for consistency

Option 1 (preferred): Include escape rule in all H/M/L agents for completeness:

-Use the shared H/M/L schema:
+Use the shared H/M/L schema, escaping any literal `|` inside description or fix as `\|`:

Option 2: Remove escape rule from CI and security agents, rely on shared contract:

In .claude/agents/deep-review-ci.md and .claude/agents/deep-review-security.md:

-Use the shared H/M/L schema, escaping any literal `|` inside description or fix as `\|`:
+Use the shared H/M/L schema:

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/agents/deep-review-architecture.md around lines 60 - 66, The H/M/L
agent prompts are inconsistent about the escape rule; update the four agents
missing it (deep-review-architecture, deep-review-code, deep-review-python,
deep-review-typescript) to include the exact escape rule text from SKILL.md ("If
a description or fix contains a literal `|`, escape it as `\|`") so wording is
identical across all H/M/L agents; ensure the escape rule appears in the same
location/format as in deep-review-ci and deep-review-security and run a quick
grep to verify all six agent prompts contain the exact phrase.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.claude/agents/deep-review-architecture.md:
- Around line 60-66: The H/M/L agent prompts are inconsistent about the escape
rule; update the four agents missing it (deep-review-architecture,
deep-review-code, deep-review-python, deep-review-typescript) to include the
exact escape rule text from SKILL.md ("If a description or fix contains a
literal `|`, escape it as `\|`") so wording is identical across all H/M/L
agents; ensure the escape rule appears in the same location/format as in
deep-review-ci and deep-review-security and run a quick grep to verify all six
agent prompts contain the exact phrase.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b5f78b88-78c3-4c15-98fb-ca4a2384315f

📥 Commits

Reviewing files that changed from the base of the PR and between 48d1d3a and 86f9e57.

📒 Files selected for processing (19)

• .claude/agents/deep-review-architecture.md
• .claude/agents/deep-review-ci.md
• .claude/agents/deep-review-code.md
• .claude/agents/deep-review-docs.md
• .claude/agents/deep-review-project-checklist.md
• .claude/agents/deep-review-python.md
• .claude/agents/deep-review-qa.md
• .claude/agents/deep-review-security.md
• .claude/agents/deep-review-simplification.md
• .claude/agents/deep-review-typescript.md
• .claude/agents/deep-review-unit-test.md
• .claude/skills/deep-review-pro/SKILL.md
• docs/AI_ASSISTANTS.md
• docs/deep-review-pro-benchmark/README.md
• docs/deep-review-pro-benchmark/reports/587-epic-token-cost-matrix.json
• docs/deep-review-pro-benchmark/reports/587-epic-token-cost-matrix.md
• scripts/benchmark_deep_review_epic_matrix.py
• scripts/test_benchmark_deep_review_epic_matrix.py
• scripts/test_benchmark_deep_review_pro.py