Initiatives Project: validation tests + 30-day pilot review

[don-petry]

Tracks the validation playbook for the Initiatives Project pilot (#387) and the 30-day review checkpoint.

Validation tests (re-run on demand)

End-to-end smoke tests; re-run after any change to the workflow / scripts. Project link: https://github.com/orgs/petry-projects/projects/1

1. Add path (issue): open a throwaway issue in .github labeled dev-lead. Within ~30s an item with the issue number must appear on the board.
2. Noise gate: open a throwaway issue labeled dev-lead + compliance-audit. Workflow run must log Skip … has excluded label 'compliance-audit'; no item appears.
3. Add path (discussion): open a throwaway discussion in the Ideas category. A draft item titled [Discussion #N] … must appear with Source: <url> in the body.
4. Category gate: open a throwaway discussion in General. No draft must appear.
5. State machine (Option A): move the Ideas discussion to General. The draft for it must be deleted automatically (project count goes down by 1).
6. Cleanup-on-delete: delete the General discussion. Workflow logs not tracked no-op.

The 2026-06-07 smoke-test session passed all six. Workflow runs: 27093534727 / 27093534553 / 27093539930 / 27093539638 / 27093621644.

Bats suite (CI gate)

add-to-project-tests.yml runs on every PR touching the workflow / scripts / tests. 35 tests covering:

• All 4 corners of the discussion state machine + empty-category cleanup
• Paginated lookup with cursor-protocol assertion
• Title-prefix anchoring (Compliance: allow_auto_merge #42 vs docs(standards): warn that updateProjectV2Field replaces the full option list #420)
• Multi-match WARNING (does not crash via SIGPIPE)
• data.node:null → exit 75 with diagnostic
• Draft → Issue conversion via Issue.title fallback
• delete_project_item idempotency on Could not resolve
• Real-error propagation
• Null / non-array LABELS_JSON defense
• Programmer-error (64/65) vs clean-skip (1) exit-code distinction
• Source: URL preserved in draft body
• Invocation-count assertions on every happy path
• run --separate-stderr for stderr-aware error tests

30-day pilot review — target 2026-07-07

Decision the review must reach:

• Keep as-is — single Initiatives board, no further boards, multi-repo rollout deferred
• Expand — add Compliance program board, Fleet Monitor board; ship multi-repo rollout
• Roll back — pilot didn't deliver enough value to justify the maintenance overhead

Inputs to weigh at review time:

• How many items did the auto-add land vs. how many had to be added manually?
• Did the noise gate keep the board readable?
• Did anyone besides me actually use a view?
• Did the bats suite catch any regression in the 30 days?
• Are the 3 deferred items (multi-repo, generic reconcile, label-removal cleanup) blocking work?

Related

• Adopt GitHub Projects: org-level Initiatives pilot #387 — tracking issue (pilot kick-off)
• discussion Enable and adopt usage of GitHub Projects #386 — founding discussion
• this PR (TBD via auto-add) — multi-repo follow-on