feat(unic-pr-review): calibrate silent-failure-hunter for lost-signal/observability gaps

[orioltf]

Summary

• Adds three new detection bullets to the Silent Failure Hunter prompt (agents/silent-failure-hunter.md) covering lost-signal / observability gaps
• Adds two false-positive guard bullets to prevent over-flagging deliberate traced suppression and documented opt-outs
• No code changes, no version bump, no test changes needed — agent prompts are Markdown instructions to an LLM

Why

The #5612 parity comparison showed silent-failure-hunter returning 0 findings while the toolkit's equivalent correctly flagged a page_view analytics event being silently dropped on a cancelled route transition. The agent's "What to look for" section covered only error-handling antipatterns (catch blocks, promise rejections) — it had no concept of observability gaps in control flow.

Before / After example (PR #5612 — AnalyticsTracker.tsx, FZAG/dxp)

Pattern in the diff:

const handleRouteChangeComplete = (url: string) => {
  if (routeChangeCancelled) {
    return  // ← early exit; page_view never fires; no production trace of cancellation
  }
  // ...
  trackEvent('page_view', { url })
}

Before (unic v2.1.2):

{ "findings": [], "positiveObservations": [] }

After (with this change):

{
  "findings": [{
    "severity": "important",
    "confidence": 82,
    "filePath": "src/analytics/AnalyticsTracker.tsx",
    "startLine": 46,
    "title": "page_view lost on cancelled route — no production trace",
    "body": "The early return on routeChangeCancelled exits without firing trackEvent('page_view'). No alternative trace records the cancellation in production observability. Either emit a navigation_cancelled event in that branch, or add a structured log capturing the url and reason."
  }],
  "positiveObservations": []
}

False-positive guard validation

A route handler that fires trackEvent('navigation_cancelled', { url }) in the cancelled branch would not be flagged — that falls under the new "deliberate traced suppression" exclusion in "What NOT to look for".

Test plan

• pnpm --filter unic-pr-review typecheck — green (517 pass, 0 fail)
• pnpm --filter unic-pr-review test — green (517 pass, 0 fail)
• Manual: re-run on PR #5612 diff to confirm the observability gap finding surfaces

Closes #217

🤖 Generated with Claude Code

[orioltf]

🔍 Comprehensive PR Review

PR: #222 — feat(unic-pr-review): calibrate silent-failure-hunter for lost-signal/observability gaps
Reviewed by: code-review agent (4 agents skipped — prompt-only Markdown change)
Date: 2026-06-05

---

Summary

Prompt-only change to silent-failure-hunter.md — five new bullets covering lost-signal/observability gaps and false-positive guards. No .mjs code touched; error-handling, test-coverage, comment-quality, and docs-impact agents were skipped as not applicable.

Verdict: APPROVE (with one optional follow-up)

Severity	Count
🔴 CRITICAL	0
🟠 HIGH	0
🟡 MEDIUM	0
🟢 LOW	2

---

🟢 Low Issues (For Consideration)

1. Conceptual overlap between new bullets 1 and 2

📍 apps/claude-code/unic-pr-review/agents/silent-failure-hunter.md:42–44

Bullet 1 (abstract statement) and bullet 2 (concrete API examples: trackEvent, analytics.page, APM spans) describe the same antipattern from two angles. Low probability of LLM generating duplicate findings per location, but the risk exists.

View recommendation

Recommended: Keep as-is (Option B). The abstract+concrete two-framing is a deliberate LLM prompting strategy. If duplicate findings appear in manual testing on PR #5612, merge into a single bullet with inline examples.

---

2. No CHANGELOG entry for behavioral capability change

📍 apps/claude-code/unic-pr-review/CHANGELOG.md (unchanged)

The [Unreleased] section is empty. All recent releases (2.1.2, 2.1.1…) carried a CHANGELOG entry + patch bump even for smaller behavioural fixes. The agent's output changes visibly — new findings will surface where previously there were none.

View options

Option	Approach	Effort
A (recommended)	Add [Unreleased] entry + patch bump	LOW
B	Document "prompt-only = no entry" policy in AGENTS.md	LOW
C	Keep as-is	NONE

Recommendation: Option A — add a patch bump + CHANGELOG entry. The behaviour change is user-visible regardless of file type.

---

✅ What's Good

• False-positive guards are symmetric and precise — each new detection bullet has a matching exclusion in "What NOT to look for"
• PR description includes a clear before/after JSON example from PR #5612 — excellent documentation of the behavioural change
• Bullet 3 (dev-flag silences) extends scope cleanly without requiring coordination with other agents
• **different** emphasis in the false-positive guard is intentional and effective at distinguishing the one-word signal the LLM must hold

---

Next Steps

1. ✅ No CRITICAL or HIGH issues — no blocking auto-fixes needed
2. 📝 Decide on LOW Finding 2: add CHANGELOG entry (Option A) or skip
3. 🎯 Merge when ready

---

Reviewed by Archon comprehensive-pr-review workflow
Artifacts: /Users/oriol.torrent/.archon/workspaces/unic/unic-agents-plugins/artifacts/runs/9f2a389d1a4f53da2e4570e282c3826f/review/

[orioltf]

⚡ Self-Fix Report (Aggressive)

Status: COMPLETE
Pushed: ✅ Changes pushed to archon/task-feature-unic-pr-review-217-calibrate-lost-signal-o
Philosophy: Fix everything unless clearly a new concern

---

Fixes Applied (1 total)

Severity	Count
🔴 CRITICAL	0
🟠 HIGH	0
🟡 MEDIUM	0
🟢 LOW	1

View all fixes

• ✅ No CHANGELOG entry for behavioral capability change (apps/claude-code/unic-pr-review/CHANGELOG.md) — Added four-bullet [2.1.3] — 2026-06-05 entry documenting all new silent-failure-hunter detection patterns (lost-signal gaps, missing event-emission APIs, dev-flag silences, false-positive guards); ran pnpm bump patch to promote and sync plugin.json / marketplace.json / package.json to 2.1.3; verify:changelog CI check passes.

---

Tests Added

(none) — prompt-only PR; no .mjs code touched

---

Skipped (1)

Finding	Reason
Conceptual overlap between new "What to look for" bullets 1 and 2	Reviewer recommendation: keep as-is. The abstract + concrete two-framing is a deliberate LLM prompting strategy. Only merge if duplicate findings are observed in production on #5612.

---

Suggested Follow-up Issues

(none)

---

Validation

✅ verify:changelog | ✅ Lint (Biome ran during bump) | ✅ No code changed

Note: Another branch (archon/task-feature-unic-pr-review-216-two-phase-code-simplifi) also bumps to 2.1.3 — whichever merges second will need a minor version-conflict resolution.

---

Self-fix by Archon · aggressive mode · fixes pushed to archon/task-feature-unic-pr-review-217-calibrate-lost-signal-o

[Copilot]

Pull request overview

This PR calibrates the unic-pr-review Silent Failure Hunter agent prompt to detect “lost-signal / observability gap” patterns (early returns, fallbacks, cancellation paths, dev-only diagnostics) and adds explicit false-positive guards, then ships the change as version 2.1.3 with updated release metadata.

Changes:

• Expand silent-failure-hunter’s “What to look for” rubric to include production-observability gaps (missing analytics/telemetry/log/Sentry traces on non-success paths).
• Add “What NOT to look for” guardrails to avoid flagging deliberate traced suppression and explicit/documented telemetry opt-outs.
• Bump unic-pr-review to 2.1.3 and document the release in the changelog + plugin metadata.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
apps/claude-code/unic-pr-review/agents/silent-failure-hunter.md	Adds observability-gap detection bullets and false-positive exclusions to the agent prompt.
apps/claude-code/unic-pr-review/package.json	Bumps package version to 2.1.3.
apps/claude-code/unic-pr-review/CHANGELOG.md	Adds a 2.1.3 entry describing the new detection/guard behavior.
apps/claude-code/unic-pr-review/.claude-plugin/plugin.json	Bumps plugin version to 2.1.3.
apps/claude-code/unic-pr-review/.claude-plugin/marketplace.json	Bumps marketplace plugin version to 2.1.3.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.