feat: add user research workflow with skills, command, and agent

[mattthomps1]

Summary

Adds an optional Research phase to the compound engineering loop. Just as Plan grounds work in codebase context and Review catches issues before shipping, Research grounds product decisions in user evidence — personas, interview insights, and identified opportunities.

1 command, 3 skills, 1 agent. Integrates into brainstorm and plan workflows but is entirely optional — teams without research data experience zero behavioral change.

New Components

Component	Type	Description
/workflows:research	Command	Orchestrates 3 phases: plan research, process transcripts, build personas
research-plan	Skill	Create structured research plans with Teresa Torres methodology
transcript-insights	Skill	Process interview transcripts into tagged insight snapshots
persona-builder	Skill	Synthesize personas from interviews with confidence tracking
user-research-analyst	Agent	Search research artifacts for evidence relevant to planning

How It Fits the Loop

         ┌──────────────────────────────────────────┐
         │          COMPOUND ENGINEERING LOOP        │
         │                                           │
         │   Research (optional, feeds into Plan)    │
         │       ↓                                   │
         │   Plan → Work → Review → Compound         │
         └──────────────────────────────────────────┘

Key Decisions

1. "Do No Harm" — Producer Messages, Consumer Silences

The most important design constraint: existing workflows must behave identically when no research data exists.

How it works: user-research-analyst runs in parallel during brainstorm/plan. When docs/research/ is empty, the agent returns "No data found." But the consuming workflows don't surface that — they skip silently:

# In brainstorm.md:
If `user-research-analyst` returns relevant findings, briefly summarize them.
If no research data exists, skip the summary silently and proceed directly
to the collaborative dialogue — do not mention the absence of research
or suggest running `/workflows:research`.

Pattern: the producer handles absence with a message; the consumer handles absence with silence. This prevents nagging users who don't use research.

2. Privacy by Design

Transcripts contain PII and are gitignored by default. Only anonymized snapshots and personas get committed. Skills enforce PII stripping with MUST language, not advisory "consider" language.

Directory	Contents	Committed?
docs/research/plans/	Research plans with hypotheses	Yes
docs/research/transcripts/	Raw interview transcripts (PII)	No (gitignored)
docs/research/interviews/	Anonymized interview snapshots	Yes
docs/research/personas/	Living persona documents	Yes

3. No Hard Prerequisites Between Phases

Transcripts can be processed without creating a research plan first — select "Ad-hoc / no plan" during processing. The phase recommender prioritizes actionable data (unprocessed transcripts) over missing prerequisites (no plan).

4. Single Canonical Reference — Extensible by Design

discovery-playbook.md (Teresa Torres + Mom Test methodology, 414 lines / 10 sections) lives in research-plan/references/ and is referenced by the other two skills via relative path. No duplication.

5. Parallel Execution

user-research-analyst runs alongside repo-research-analyst (brainstorm) and learnings-researcher (plan). No serial bottleneck added.

Also Included

• Wired research context into existing /workflows:brainstorm and /workflows:plan commands
• Compound learnings doc: docs/solutions/integration-issues/adding-optional-workflow-phases-with-graceful-degradation.md
• Solution doc for workflow routing fix and transcript input mismatch
• Removed PII sample data and deduplicated discovery playbook during review

Test Plan

"Do no harm" tests (most important):

• Run /workflows:brainstorm with NO research data — verify zero behavioral change (no mention of research)
• Run /workflows:plan with NO research data — verify zero behavioral change

Feature tests:

• Run /workflows:research plan — verify research plan created at docs/research/plans/
• Run /workflows:research process with a transcript — verify anonymized snapshot at docs/research/interviews/
• Run /workflows:research process without a plan — verify "Ad-hoc / no plan" option works
• Run /workflows:research personas with existing snapshots — verify persona at docs/research/personas/
• Run /workflows:brainstorm WITH research data — verify personas/insights surfaced before dialogue
• Run /workflows:plan WITH research data — verify user evidence in Step 1.6 consolidation

Extensibility:

• Edit discovery-playbook.md — verify all 3 skills load updated content on next invocation

Verification:

• .gitignore includes docs/research/transcripts/*.md

Post-Deploy Monitoring & Validation

No additional operational monitoring required: plugin-only change with no runtime/server component. Existing workflows remain unaffected when docs/research/ is empty.

---

🤖 Generated with Claude Code

[kieranklaassen]

This is cool! LEt me try it out a bit!

[kieranklaassen]

Code Review: PR #194 — feat: add user research workflow

Review by: 5 parallel agents (pattern-recognition, architecture, simplicity, agent-native, security) + manual analysis

Overall Assessment

This is a well-designed PR that adds a cohesive user research workflow (1 command, 3 skills, 1 agent) to the compound engineering loop. The "Do No Harm" integration pattern is elegant — the producer handles absence with a message while the consumer handles it with silence. The architecture follows established plugin conventions (naming, frontmatter, directory structure, progressive disclosure). Component counts are accurate across all files.

However, the branch is stale relative to main and will silently regress features from v2.35.1 and v2.35.2 when merged. This is the critical blocking issue.

---

🔴 P1 — Critical (BLOCKS MERGE)

1. Stale branch will regress v2.35.1 and v2.35.2 features from main

The PR branch was forked before several important changes landed on main. The plan.md diff shows removals of features that exist on main but were never added to this branch:

Features that would be reverted:

• origin: frontmatter field in all 3 plan templates (v2.35.2)
• Brainstorm cross-check in Section 6 (v2.35.2)
• "Sources" sections linking plans to brainstorms (v2.35.2)
• "System-Wide Impact" sections in MORE and A LOT templates (v2.35.1)
• "Write Plan File" mandatory step before interactive prompts (v2.35.0 fix)
• Pipeline mode note for LFG/SLFG invocation (v2.35.0 fix)
• compound-engineering:workflow:spec-flow-analyzer qualified namespace (v2.35.0 fix — would revert to unqualified spec-flow-analyzer)

Version conflict: Main is at v2.35.2, this PR sets v2.35.0 — a downgrade.

Fix: Rebase on main, resolve conflicts in plan.md, and bump version to 2.36.0.

2. Incomplete .gitignore pattern — non-markdown transcripts not protected

docs/research/transcripts/*.md

This only ignores .md files. Transcripts saved as .txt, .docx, .json, or .csv (common for exports from Otter.ai, Rev, etc.) would be committed with full PII.

Fix: Change to:

docs/research/transcripts/*
!docs/research/transcripts/.gitkeep

---

🟡 P2 — Important (Should Fix)

3. .claude/settings.json should not be committed

This personal dev config forces compound-engineering@every-marketplace enablement on all repo clones. Similar to committing .vscode/settings.json with personal preferences. Remove from PR and add to .gitignore.

4. Anonymization is advisory, not enforced — PII could leak into committed files

The transcript-insights skill has a "Privacy Note" section at the bottom, but there's no dedicated anonymization pass before quotes are extracted in Step 4c. The AI processes the transcript top-to-bottom and may extract quotes before encountering the privacy instructions.

Fix: Add a "Step 4.0: Anonymization Pass" before insight extraction, and add - [ ] No real names, email addresses, or company names in output to both the transcript-insights and persona-builder Human Review Checklists.

5. Stale privacy language in research-plan/SKILL.md

Consider adding `docs/research/transcripts/` to `.gitignore` if transcripts
contain personally identifiable information.

The .gitignore entry already exists in this PR. "Consider adding" is stale and uses weak language. Update to: "Transcripts in docs/research/transcripts/ contain PII and MUST NOT be committed to version control."

6. No headless/batch mode for transcript processing (agent-native gap)

The write-side of this workflow requires 7+ AskUserQuestion calls per transcript (selection, plan linking, 5 metadata fields, handoff menu). When an orchestrating agent calls /workflows:research process, every decision point blocks on human input.

The read-side (user-research-analyst searching artifacts) is fully agent-native and well-designed. The inline transcript routing (Rule 1) is also a model of good agent-native design.

Recommendation for future: Add structured-input / headless paths so agents can pass metadata as arguments and skip interactive prompts. This is non-blocking for v1 since the primary use case is human-driven.

---

🔵 P3 — Nice to Have

7. Skills are large relative to the gold standard

Skill	Words	Gold Standard (dhh-rails-style)
research-plan	~1,400	~846
transcript-insights	~2,000	~846
persona-builder	~1,600	~846

Output template sections are ~40-50% of each file. These could move to references/output-template.md files for better progressive disclosure.

8. Discovery playbook could be trimmed

414 lines, but ~264 lines are non-actionable educational content (team structure advice, tool lists, bibliography, motivational sections). The skills already embed the relevant methodology. Consider trimming to ~150 lines covering just interview structure, snapshot format, transcript analysis, and OST framing.

9. Persona merge spec is over-engineered for v1

The merge specification has percentage thresholds (40/60 split detection), detailed field-by-field update tables, and hypothesis status definitions for a feature that will typically operate on 1-4 interviews. Consider collapsing to concise prose rules.

10. Agent search strategy over-prescribed

8 steps / 77 lines for what amounts to "grep the research directory for relevant files and summarize." The scoring rubric and fallback layers add complexity for a directory that will typically contain 0-10 files.

11. Plan docs should have status: completed

The two plan files in docs/plans/ still have status: active but the feature is built.

---

What's Working Well

• "Do No Harm" integration is elegant. The two-layer graceful degradation (producer handles absence with message, consumer handles with silence) ensures zero behavioral change for teams without research data.
• Parallel execution — user-research-analyst runs alongside repo-research-analyst and learnings-researcher with no serial bottleneck.
• Reference deduplication — single canonical discovery-playbook.md referenced by all 3 skills via relative paths. Avoids tripling 414 lines.
• Inline transcript routing (Rule 1) — accepts data, not decisions. An agent can pass a full transcript inline and the command saves + processes it without interaction.
• Privacy architecture — clean separation of raw transcripts (gitignored) from anonymized snapshots and personas (committed).
• All naming conventions, frontmatter, directory structure, component counts are correct and consistent with existing patterns.
• Context token budget — all descriptions within targets (100-300 for skills, 100-250 for agents, 40-120 for commands).
• Solution docs are excellent — the "Adding Optional Workflow Phases with Graceful Degradation" doc is a valuable pattern reference.

---

Review Agents Used

Agent	Focus
pattern-recognition-specialist	Codebase consistency, naming, conventions
architecture-strategist	Structural design, integration quality, coupling
code-simplicity-reviewer	YAGNI, over-engineering, LOC reduction
agent-native-reviewer	Agent accessibility, interactive blocking points
security-sentinel	PII handling, gitignore, anonymization, consent

Next Steps

1. Rebase on main — resolve conflicts in plan.md, preserve both research integration AND v2.35.1/v2.35.2 features
2. Bump version to 2.36.0 (minor: new functionality)
3. Fix gitignore pattern — docs/research/transcripts/* with !.gitkeep
4. Remove .claude/settings.json from PR
5. Update stale privacy language in research-plan/SKILL.md

[mattthomps1]

Thanks @kieranklaassen! Made some changes, will test in the morning and confirm it's good to re-review.

[mattthomps1]

Hi @kieranklaassen, addressed the P1 and P2 except for the headless mode. Will address that and some of the nice to have in future PR's. Also had the initial folder creation process mirror similar patterns of keeping them in the Skills rather than creating all at once.

Ready for review / merge!

[kieranklaassen]

Thanks, I'll try it out a bit myself!