evals: add edit/claude-tag-slack-fidelity — make a Slack-UI mock actually look like Slack

[nityeshaga]

What this eval tests

A new edit eval (edit-claude-tag-slack-fidelity). An arm receives a 12-slide deck that mocks the Slack desktop client built entirely from native PowerPoint shapes (workspace rail, channel sidebar, header, message list, reactions, composer) — but in its current form it does not convincingly read as Slack: a whole class of labels was silently dropped at compile, reaction pills are blue with no counts, the header is translucent and too tall, avatars are oversized/too round, most chrome slides have no composer, rail tiles render as bright white squares, etc.

The arm gets no checklist — just the brief "make it read as the real current Slack desktop UI, fix everything that gives it away, preserve the content and 12 slides, then prove your fixes in the render." It measures two things a clean patch can't fake:

1. Discovery — can a tool-holding agent figure out what makes a UI mock read as "not Slack," with no list?
2. Execution + verification — can it then carry a multi-slide, shared-component fidelity pass and confirm it actually landed in the PPTX render (recovered labels, recolored pills, resized header, added composers), rather than declaring success from a clean-looking source?

Input deck

evals/assets/claude-tag-slack-fidelity/slack-wireframe-input.pptx (12 slides, 16:9, 13.33×7.5in). Same-deck setup: both arms edit identical copies placed at {{WORKDIR}}/deck.pptx.

How it differs from claude-tag-review-loop

Both are answer-key-based edit evals on a "Claude Tag" deck, but they are distinct:

• claude-tag-review-loop hands a different buggy deck and tests find-and-fix of five generic render-only layout defects (a wrapping stat, white-on-white text, short header bar, clipped headline, overflow), some of which a linter flags.
• This task hands a Slack-UI wireframe and tests whether the arm can make a mock of a specific, well-known real app read as the genuine article — a component-level fidelity redesign against an external reference the arm must reconstruct from memory. Every tell here is no-machine-signal (no linter floor), so all of them require render-and-look plus Slack knowledge.

It does not modify or collide with the review-loop eval.

The judge's rubric

evals/judges/edit-auditor-claude-tag-slack-fidelity.md is an answer-key auditor (no design judges, like the review-loop auditor). It holds a prioritized tell manifest derived from a real-Slack-vs-deck audit:

• P0 (heaviest): recover the silently-dropped text (channel names, DM names, workspace name "Anthropic", channel-header titles, member count, reaction counts); neutralize reaction pills (gray, real emoji + count).
• P1: solid ~50px white header; ~40px/r8 square avatars; composers on every chrome slide (remove the banner in slide 3's composer slot).
• P2: muted rail-tile fills; neutral gray APP badge; channel-row sizing + unread bold; calm the paper watermark behind message panes; distinct admin/checklist icons.

Per arm it scores each tell fixed / attempted-but-broken / missed, computes a weighted P0/P1/P2 headline score, grades discovery breadth (how many of 10 tells found with no checklist) separately from fix correctness, checks consistency of shared-component fixes across all 12 slides, scores two traps (indexing; "clean patch ≠ fixed render"), and runs a luminance regression diff to catch lost content / new overflow / broken slide count. Rewards discovery + correct, consistent, render-verified execution; penalizes regressions and "looks fixed in the patch but not the render." Final per-arm grade A–F.

Sanity checks

• Asset opens: Presentation(...).slides → 12 slides, canvas 13.33×7.5in.
• Format parity confirmed against sibling edit-two-world-reskin / edit-retheme-and-insert tasks and the existing auditors ({{TOOLCHAIN_BLOCK}}/{{WORKDIR}}/{{JUDGING_DIR}} placeholders, 0-based indices, Process + Deliverables + structured changes.json log, A–F auditor grade).
• Registered in evals/README.md (tree) and evals/RUNBOOK.md (Phase 4 block).

Convention judgment calls

• The answer key is embedded inline in the auditor markdown (matching how edit-auditor-claude-tag and edit-auditor-two-world carry their manifests inline) rather than committed as a separate review file — no separate answer-key file is added.
• Asset lives in a subdir bundle claude-tag-slack-fidelity/ with a descriptive filename, consistent with the claude-tag-review-loop/ and every-consulting-rebuild/ asset layout.
• Branched off main so this PR is independent of the still-open claude-tag-review-loop PR.

🤖 Generated with Claude Code