fix(aicr-cross-review skill): consumer-less demotion; resumable Codex lane - #2102
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe workflow supports Codex job resumption with longer continuation waits and exposes Estimated code review effort: 4 (Complex) | ~45 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/aicr-cross-review/scripts/workflow.mjs:
- Around line 927-931: Move the zero-survivor guard containing
incomplete(zeroSurvivors(...)) into the function that owns integBatch and
demotedBatch, or propagate its result through that function’s caller. Preserve
the return incomplete(...) behavior so processing stops immediately; do not
leave a return at module scope or replace it with a bare call.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 8df0cc7b-9b84-4ca3-bdec-66f90d92a3a6
📒 Files selected for processing (2)
.agents/skills/aicr-cross-review/SKILL.md.agents/skills/aicr-cross-review/scripts/workflow.mjs
b612c97 to
65fb234
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/aicr-cross-review/scripts/workflow.mjs:
- Around line 295-307: Update both stale-worktree cleanup find commands in the
workflow to include -xdev alongside -depth -delete, ensuring cleanup cannot
descend into mounted filesystems. Revise the nearby explanation to state that
find is restricted from crossing filesystem boundaries, without changing the
existing prefix and age limits or replacing find with rm.
- Around line 925-929: Update the report-routing logic around intakeBatch and
demotedBatch so Integration Findings requires valid consumerPath and
consumerLine values. Detect the demotion flag and route those candidates to
ordinary findings even when sources contains "integration"; preserve normal
integration routing only for findings with a valid consumer pair.
In @.agents/skills/aicr-cross-review/SKILL.md:
- Line 122: The cleanup commands use non-portable find options. In
.agents/skills/aicr-cross-review/SKILL.md at lines 122-122 and 670-671, replace
both direct-file cleanup commands with find "$DIFFPATH" -delete; update the STEP
1 depth-limited reaper to use a BSD/macOS-compatible enumerator or explicitly
require GNU find. In .agents/skills/aicr-cross-review/scripts/workflow.mjs at
lines 292-292, make the corresponding direct-file cleanup change. Ensure reaper
failures are handled so stale-worktree cleanup is not silently skipped.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: e0d43f5c-6066-473f-8f79-0d09ec5b66e1
📒 Files selected for processing (2)
.agents/skills/aicr-cross-review/SKILL.md.agents/skills/aicr-cross-review/scripts/workflow.mjs
65fb234 to
8aae218
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
8aae218 to
de5d32d
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
de5d32d to
6c81fd3
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/aicr-cross-review/scripts/workflow.mjs:
- Line 46: Validate codexResumeJobId against the companion’s expected job-ID
format before it reaches the Codex resume/status command, reject invalid values,
and shell-quote the validated value during interpolation. Update the parsedArgs
handling and command construction in the workflow flow while leaving unrelated
arguments unchanged.
- Around line 213-217: Update the outer-timeout handling in
.agents/skills/aicr-cross-review/scripts/workflow.mjs lines 213-217 to preserve
the known Codex job ID in the unavailable result and require it for resumption;
update the corresponding protocol guidance in
.agents/skills/aicr-cross-review/SKILL.md lines 393-405, and make
FINDINGS_SCHEMA.jobId required for outer-timeout results so incomplete() emits
codexJobId.
In @.agents/skills/aicr-cross-review/SKILL.md:
- Around line 409-416: The resume example in the Codex recovery instructions
uses invalid object-spread syntax. Update the `Workflow` invocation in the
`incomplete`/`codexJobId` guidance to spread a valid identifier such as
`previousArgs` while preserving the existing arguments and `codexResumeJobId`
value.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 47e7e7b7-5387-4cdc-b111-b790ec45ac02
📒 Files selected for processing (2)
.agents/skills/aicr-cross-review/SKILL.md.agents/skills/aicr-cross-review/scripts/workflow.mjs
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
6c81fd3 to
cd49cd4
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/aicr-cross-review/SKILL.md:
- Around line 690-691: Make the final cleanup around the DIFFPATH removal
idempotent by tolerating an already-absent path, so `set -e` cannot prevent the
subsequent scoped `PRREF` and `BASEREF` deletion commands from running. Update
the “Clean up before finishing” instructions without changing the cleanup scope.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: aed2a065-f71b-4ad3-a62a-817ae08df5da
📒 Files selected for processing (2)
.agents/skills/aicr-cross-review/SKILL.md.agents/skills/aicr-cross-review/scripts/workflow.mjs
cd49cd4 to
7ad6044
Compare
7ad6044 to
3ccf82d
Compare
3ccf82d to
240783d
Compare
Pull request was converted to draft
240783d to
5202985
Compare
… lane Three cross-review skill fixes, all observed on real runs: Demote consumer-less integration findings, not the run. An integration-lane finding without a consumerPath/consumerLine pair is demoted to an ordinary finding (consumer fields stripped, flagged, no integration severity escalation) instead of dropped - so a legitimately consumer-less observation can no longer make the whole review return incomplete (observed on PR NVIDIA#2097: one real finding dropped, zero-survivor guard stopped the run with all four lanes ok and no report). Demotion is logged, demoted candidates carry a flag that reaches the cross-review candidate list and refuter prompts, and the minor-to-medium escalation now explicitly requires a verified consumer pair. The zero-survivor stop still fires when a non-empty integration result yields no accepted finding at all. Prompt-free lane operations. The CodeRabbit lane's worktree cleanup and stale-reaper now delete with 'git worktree remove' + 'find -depth -delete' instead of 'rm -rf', and the skill's diff-file cleanup follows suit: managed permission policies commonly gate rm behind a confirmation prompt (Bash(rm:*) ask rules match every sub-command and override allow rules), and a background lane blocked on a prompt stalls the review. STEP 1 now also probes whether ~/.coderabbit is sandbox-writable; STEP 2 runs sandboxed when the probe passes (no bypass approval prompt on machines that allowlist the path) and bypasses from the start otherwise, keeping the lane portable without guessing into the ten-minute sandbox-denial hang. Resumable Codex lane on exhausted wait budget. A real review job ran ~53 minutes on a +1951/-153 PR, past the previous three-wait (~27 min) continuation budget, so the lane correctly returned unavailable, the required-lane rule reported the whole run incomplete, and recovery had to be improvised with a hand-patched script. The budget is now five bounded waits (~45 min); the lane's result schema gains an optional jobId field the lane must set when a live job outlasts the budget; the workflow surfaces it top-level as codexJobId on incomplete results; and a new codexResumeJobId arg makes the resume first-class - the Codex lane's prompt gains a resume note telling it to fetch or wait on the existing job instead of re-dispatching, while the other three lanes' prompts stay byte-identical so their cached round-1 results replay on resume. Proven live on PR NVIDIA#2097 with zero re-dispatched work. SKILL.md is updated in lockstep. Signed-off-by: Yuan Chen <yuanchen97@gmail.com>
5202985 to
1ec58f5
Compare
Summary
Three fixes to the aicr-cross-review skill, all from failure modes observed on real runs:
consumerPath/consumerLinepair is now demoted to an ordinary finding (consumer fields stripped, flagged, no integration severity escalation) instead of dropped — so a legitimately consumer-less observation can no longer make the whole review returnincomplete.git worktree remove+find -xdev -depth -deleteinstead ofrm -rf, and sandbox bypass for thecoderabbit reviewcall is now probe-gated — machines whose sandbox allowlist covers~/.coderabbitrun it fully sandboxed with no bypass approval prompt.jobIdfield, the workflow surfaces it top-level ascodexJobIdonincompleteresults, and a newcodexResumeJobIdarg resumes the run against the existing job with no re-dispatch.Motivation / Context
Demotion: observed on PR #2097 — the integration lane returned exactly one finding, a real, evidenced observation that legitimately has no consumer. The consumer-coordinate filter dropped it, the zero-survivor guard saw a required lane contribute nothing, and the run stopped
incompletewith all four lanesok, ~550k subagent tokens spent, and no report produced. A consumer-less finding cannot be verified as an integration claim, but if it locates a defect (own path/line) with evidence it is a perfectly reviewable ordinary finding. Demotion keeps the lane's contribution and lets consensus run.Prompt-free ops: observed on a live run of this branch — managed (admin-deployed) permission policies commonly put
Bash(rm:*)in the ask list, which matches every sub-command of a compound call and overrides any allow rule, so each lane cleanup blocked on a manual confirmation. Separately, the unconditional sandbox bypass on thecoderabbit reviewcall costs a per-run approval prompt even on machines where~/.coderabbitis sandbox-writable.Resumable Codex lane: observed reviewing PR #2097 — a real Codex review job on a +1951/−153 diff ran ~53 minutes, past the three-wait budget (#2101) that was sized above the then-measured ~19-minute maximum. The lane correctly returned
unavailablewith the job id only in free text, the required-lane rule reported the whole runincomplete, and recovery had to be improvised: poll the job by hand, then resume with a hand-patched script telling the Codex lane to fetch the existing job. The recovery worked — cached lanes replayed, the finished job was fetched, zero work re-dispatched — so this change codifies it as a first-class path instead of an improvisation.Fixes: N/A
Related: #2101 (previous skill fix: Codex wait budget)
Type of Change
Component(s) Affected
cmd/aicr,pkg/cli)cmd/aicrd,pkg/server)pkg/recipe)pkg/bundler,pkg/component/*)pkg/collector,pkg/snapshotter)pkg/validator)pkg/errors,pkg/k8s)docs/,examples/).agents/skills/aicr-cross-review/)Implementation Notes
Demotion (workflow.mjs intake):
log()ed with locations, and each demoted candidate carries ademoted from integration claim (no consumer pair)flag that reaches the cross-review candidate list and refuter prompts.hasCoords(c.consumerPath, c.consumerLine)), so demoted findings cannot inherit the escalation reserved for proven broken consumers.Prompt-free lane operations (CodeRabbit protocol):
find … -xdev -depth -deleteinstead ofrm -rf; the skill's Phase 1/Phase 5 diff-file cleanup follows suit.find -xdev -depth -deleteremoves exactly the named tree with the same bounds —-xdevkeeps it on the filesystem it was pointed at — without matching anrmask rule. The rationale is documented at both sites so it doesn't get "simplified" back.~/.coderabbitis sandbox-writable (anif-wrapped write, safe underset -e) and echoesCR_SANDBOXED. STEP 2 runs sandboxed when the probe passed — no bypass, no approval prompt — and withdangerouslyDisableSandboxfrom the start otherwise. The probe replaces guessing: a sandboxed run on a denied machine does not fail fast, it hangs for the full ten-minute timebox, so try-sandboxed-first without a probe would cost a whole timebox per unprepared machine. The lane stays portable: contributors without the allowlist entry get exactly the previous behavior.Resumable Codex lane (workflow.mjs dispatch protocol + args):
.waitTimedOutis exhausted budget. Wait mechanics are unchanged: 540000 ms inner wait under a 600000 ms Bash cap, longer budgets always mean more calls, never longer ones.jobIdstring. The dispatch protocol requires the lane to set it whenever a dispatched job may still be live — a job that outlasts all five waits, and a wait call killed by the outer Bash timeout — since prose alone is not machine-recoverable; the job's last observed state goes instatusNote.incompleteresults surface a carriedjobIdtop-level ascodexJobId(alongsidereviewerStatus), so an orchestrator can resume mechanically without parsing free text.codexResumeJobIdarg prepends a resume note to the Codex lane's prompt: skip dispatch,statusthe given job id,resultit if terminal, otherwise wait on it under the normal continuation protocol; all other rules apply unchanged. Only the Codex prompt varies with the arg — the claude/coderabbit/integration prompts stay byte-identical so resume caching replays their round-1 results. SKILL.md documents the full recovery loop (pollcodexJobIduntil terminal, thenWorkflow({..., resumeFromRunId, args: {...prevArgs, codexResumeJobId}})). The arg is validated at intake against a plain job-id pattern and fails closed on shell metacharacters, since the lane interpolates it into a status command.SKILL.md's protocol rationale (finding-level exception paragraphs, escalation rule, sandbox-exception rules, cleanup commands, wait-budget arithmetic, recovery notes) is updated in lockstep, as the skill requires.Protocol-compliance hardening (from a live review-loop run where lane agents deviated from the written protocol):
rm -rfcleanup once blocked a run for hours on a managed-policy confirmation prompt; SKILL.md mirrors the rule.grep -c codexResumeJobId <scriptPath>must be non-zero before Phase 2, catching a stale/reverted script copy in a shared checkout (observed live); the check notes it must be updated if that identifier is renamed.Testing
Full
make qualifyskipped: the change touches only.agents/skills/aicr-cross-review/; tests, lint, and e2e cannot regress from it. Prompt byte-identity undercodexResumeJobIdwas verified statically — the claude/coderabbit/integration/cross/refute prompt builders reference neither the arg nor anything derived from it. Behavioral validation: the demotion path runs on the next cross-review whose integration lane reports a consumer-less finding (previously anincompletestop); the prompt-free cleanup and probe run on every CodeRabbit lane invocation; the resume path reproduces a live recovery already exercised end-to-end on PR #2097 (three cached lanes replayed, the ~53-minute job's result fetched, nothing re-dispatched).Risk Assessment
Rollout notes: Takes effect on the next skill invocation from an updated checkout.
Checklist
make testwith-race) — N/A, no Go changesmake lint) — N/A, no lintable sources touchedgit commit -S) — GPG signing info