diff --git a/PROJECT_STATUS.md b/PROJECT_STATUS.md index 36662d42..4365a710 100644 --- a/PROJECT_STATUS.md +++ b/PROJECT_STATUS.md @@ -53,6 +53,26 @@ Internal (fleet): with 913 Rust tests, 665 frontend tests plus the 20-scenario live qualification, lint, typecheck, production build, docs, and strict OpenSpec. +- **2026-08-10 — Revision-bound optimization contribution closeout:** extended + the local performance campaign with a mandatory post-promotion candidate + challenge and three closed CLI/MCP operations. The new local receipt binds + the kept campaign record, baseline, candidate commit, diff, optional T-Rex + preview evidence, and one canonical GitHub PR while preserving correctness, + performance, patch quality, head freshness, checks, review threads, + approvals, and merge authority as independent fail-closed gates. Candidate + source must be committed before promotion/challenge; stale heads, missing + required T-Rex evidence, unobserved checks, actionable feedback, and + symlink/path escapes cannot become ready. GitHub access is one fixed read-only + GraphQL query with explicit refresh—no polling, app install, comment, review + request, thread resolution, required check, merge, or deploy. Live read-only + dogfood against Marked PR #4048 caught and fixed a classifier error: Vercel + fork authorization is now `approval_required`, not a code failure; two older + inline threads remain visible as outdated. Raw evidence stays local and + upstream maintainers receive no additional workflow. Qualified candidate + comparisons retain target plus available smaller-input/allocation controls, + and the full Marked-shaped fixture proves reviewed-head invalidation, + simpler-candidate selection, bounded feedback learning, and regeneration of + a current receipt-backed publication projection. - **2026-08-09 — Shared lint baseline:** Adopted the Fleet Ultracite baseline for core TypeScript, React, and test code. Explicit compatibility exceptions preserve current behavior while 662 files pass with zero diagnostics; diff --git a/docs/development/performance.md b/docs/development/performance.md index c8d75f88..9c15966c 100644 --- a/docs/development/performance.md +++ b/docs/development/performance.md @@ -863,7 +863,7 @@ pnpm --silent runtime:campaign -- promote \ --hypothesis "Remove one redundant allocation without changing output." --json ``` -The same six operations are available from `runtime:mcp`. Start that server +The same six campaign operations are available from `runtime:mcp`. Start that server with `--repo /path/to/candidate --incumbent-repo /path/to/incumbent` so the promotion checkout is fixed outside tool arguments. The checked-in [agent program](../../scripts/runtime-failure-capsule/AUTONOMOUS_OPTIMIZATION_PROGRAM.md) @@ -881,6 +881,72 @@ finished campaign is a manual local cleanup choice. The engine installs nothing, invokes no production or cloud endpoint, and never commits, pushes, resets source, or manufactures a checkout. +### Candidate challenge and contribution closeout + +A campaign `keep` proves only the declared local scopes. Commit the candidate +before paired promotion so the kept record names the future PR head; uncommitted +source at challenge time fails closed. Before publishing, challenge that exact +retained SHA: + +```bash +pnpm --silent runtime:campaign -- challenge \ + --repo /path/to/candidate \ + --campaign .codevetter/optimization-campaigns/parser-loop \ + --selected-sequence 4 \ + --justification "The direct lookup adds no cache or fallback state." --json +``` + +The challenge binds the kept record, current commit, diff digest, complexity, +and deterministic risk observations. It requires either a directly comparable +qualified candidate or a bounded reason that a simpler comparison is not +applicable. Candidate comparison retains the largest scale or latency value as +the target and treats smaller scale points, bytes/op, and allocations/op as +controls when they exist. The simpler candidate must stay within the same 5% +tolerance on every retained metric. Risk tokens request evidence; they are not +a model score and cannot offset correctness or performance. + +Inspect one published pull request with the returned challenge path: + +```bash +pnpm --silent runtime:campaign -- inspect-contribution \ + --repo /path/to/candidate \ + --campaign .codevetter/optimization-campaigns/parser-loop \ + --challenge .codevetter/optimization-campaigns/parser-loop/closeout/challenge--.json \ + --pr https://github.com/owner/repository/pull/123 \ + --trex-policy optional --json +``` + +`trex-policy` is `optional`, `required`, or `not_applicable`. A supplied T-Rex +preview receipt is read from a contained local path and must identify the same +candidate SHA. CodeVetter does not launch T-Rex, a browser, preview, or hosted +load from this operation. + +The contribution receipt keeps correctness, performance, patch quality, T-Rex, +head freshness, checks, review threads, and merge authority as independent +gates. It distinguishes failed checks from fork-workflow approval, retains +outdated and resolved feedback without treating either as current work, and +reports `waiting_for_maintainer` when upstream owns merge. Use +`refresh-contribution` for one explicit reread; there is no polling. + +The append-only receipt also carries bounded maintainer-feedback learning when +a reviewed candidate is superseded: exact before/after SHAs and complexity, +the actionable thread, prior deterministic risk signals, the revised campaign +hypothesis, repeated local gates, and observed upstream disposition. This is +evidence from one contribution, not a universal style rule. + +`closeout/publication.json` is a concise projection of the newest current +receipt. A different PR head marks an existing projection `stale` while keeping +its original receipt digest; only a newly challenged, current receipt can +regenerate it. The receipt ledger remains the authority. + +The GitHub adapter uses one fixed read-only GraphQL query. It cannot comment, +resolve a thread, request review, approve, merge, deploy, install an app, create +a required check, or change pull-request metadata. Raw receipts remain local; +an author can copy a concise summary separately. + +The challenge, inspection, and refresh operations are also available through +`runtime:mcp`; their repository is fixed when the server starts. + ## 13. Qualifying a workload before profiling Runtime detection proves that a repository contains a supported toolchain; it diff --git a/docs/knowledge/performance-lab-publication.md b/docs/knowledge/performance-lab-publication.md index 739a1ab1..b8a35f89 100644 --- a/docs/knowledge/performance-lab-publication.md +++ b/docs/knowledge/performance-lab-publication.md @@ -63,23 +63,26 @@ disposition is retained in [markedjs/marked#4048](https://github.com/markedjs/marked/pull/4048) was the first external upstream PR created by this campaign. It optimizes reference-link -membership during the normal post-block inline phase while retaining the -dynamic direct-call path and clearing temporary state in `finally`. +membership by checking the existing link table directly instead of rebuilding +and linearly searching its keys for every queued inline source. The revised +patch contains no duplicate lexer state. -Five-sample paired verification measured: +Thirty alternating fresh-process pairs with per-process warmup measured: -| References | Baseline | Candidate | Change | +| References | Baseline | Candidate | Paired median change (95% interval) | | ---: | ---: | ---: | ---: | -| 100 | 1.577 ms/op | 0.869 ms/op | -44.9% | -| 500 | 15.472 ms/op | 3.326 ms/op | -78.5% | -| 2,000 | 217.253 ms/op | 12.761 ms/op | -94.126% | - -The endpoint exponent fell from 1.644 to 0.897. A representative Markdown -control moved +2.722%, below the 10% materiality threshold. Marked's complete -`npm test` passed: 190 unit tests, 1,779 specification tests, ESM/UMD/CJS and -type builds, lint, and generated-output checks. The PR is open and no longer a -draft as of this record. Its visible unit, specification, security, and Vercel -checks pass; upstream maintainers still own the merge decision. +| 100 | 0.805 ms/op | 0.489 ms/op | -39.2% (-40.3, -38.8) | +| 500 | 11.512 ms/op | 2.265 ms/op | -80.4% (-80.5, -80.2) | +| 2,000 | 164.115 ms/op | 10.095 ms/op | -93.8% (-93.9, -93.8) | + +A 2,000-paragraph no-definition control moved +1.3% with a +0.3% to +2.3% +interval. Reintroducing `Object.keys` or using `for...in` restored that fast path +but re-enumerated the full link table and erased most of the target improvement, +so neither experiment was retained. Marked's complete `npm test` passed: 190 +unit tests, 1,779 specification tests, ESM/UMD/CJS and type builds, lint, and +generated-output checks. The PR is open and no longer a draft. Two maintainer +threads on the earlier implementation are outdated. Snyk passes; Vercel's fork +preview requires upstream authorization and is not a code-test failure. ### qs @@ -143,6 +146,10 @@ optimization laboratory: 10. **Self-profiling** found and fixed repeated source-offset scans in V8 function coverage. The representative 91-document replay improved 18.11%; the 98.808% number belongs only to the adversarial regression fixture. +11. **Contribution closeout** now challenges retained patch complexity, binds + local and optional T-Rex evidence to the pull-request head, reads current + and outdated review threads, distinguishes failed checks from fork approval, + and stops at upstream ownership without posting or assigning maintainer work. The most important tooling improvements came from false or incomplete early results: startup-dominated tests now fail closed, Vitest names must identify one diff --git a/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/.openspec.yaml b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/.openspec.yaml new file mode 100644 index 00000000..d7bc0110 --- /dev/null +++ b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-08-10 diff --git a/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/design.md b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/design.md new file mode 100644 index 00000000..0d7c72d9 --- /dev/null +++ b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/design.md @@ -0,0 +1,172 @@ +## Context + +The local runtime campaign already keeps immutable manifests, append-only +experiment records, exact correctness results, paired performance evidence, +repository identities, and deterministic promotion decisions. It deliberately +does not own source edits or GitHub publication. T-Rex independently emits a +canonical change-plus-preview receipt with an exact source head and +deterministic browser-flow verdict. + +The missing boundary is between a campaign `keep` and an honest external +contribution closeout. See `proposal.md` and the optimization-contribution- +closeout specification. + +## Goals / Non-Goals + +**Goals:** + +- Make patch quality a required, evidence-backed gate after promotion. +- Bind local optimization and optional T-Rex evidence to one candidate SHA. +- Normalize enough GitHub state to distinguish code failure, review work, + approval, and upstream ownership. +- Keep the full loop available through deterministic CLI/MCP JSON without a + desktop dependency. + +**Non-Goals:** + +- Generating or editing patches, reviewing arbitrary style, or scoring code + quality with a model. +- Running T-Rex, deploying a preview, or load-testing hosted systems. +- Posting or resolving GitHub feedback, approving workflows, or merging PRs. +- Adding a second persistence system or modifying existing T-Rex receipts. + +## Decisions + +### 1. Add a companion closeout artifact instead of changing campaign v1 records + +The campaign ledger remains immutable and backward compatible. A new bounded +challenge artifact references one or two existing `keep` records by sequence, +record digest, repository revision, and diff digest. The contribution receipt +references the challenge and the selected record. + +Changing the closed v1 campaign record was rejected because old campaign +artifacts would become unreadable or require a migration that adds no runtime +evidence. Treating line count as part of the existing performance decision was +rejected because complexity must not compensate for correctness or speed. + +### 2. Use deterministic risk signals as prompts for evidence + +The challenge computes files and line movement from recorded campaign data and +scans the selected Git diff for bounded language-neutral risk tokens such as +new cache/state identifiers, `finally`/`defer` cleanup, and fallback branches. +Signals are observations, not a quality score. A flagged candidate needs either +a qualified simpler comparison or a bounded invariant justification. + +The comparison retains the largest scale or latency value as the target and +smaller scale points, bytes/op, and allocations/op as controls when the +promotion evidence contains them. A simpler candidate must stay inside the +same tolerance on every recorded value. This avoids accepting a fast target +that quietly regresses the no-op, small-input, or allocation surface. + +An AST dependency was rejected for the first slice because it would increase +packaging and language scope. An LLM quality verdict was rejected because it +would make the publication gate nondeterministic. + +### 3. Keep contribution inspection in a separate pure service + +`contribution-contracts.mjs` owns strict schemas and deterministic verdict +derivation. `contribution.mjs` loads campaign/challenge/T-Rex artifacts, +invokes a narrow injected GitHub inspector, normalizes observations, and writes +one atomic receipt under the campaign directory. Tests inject fixtures; the +default adapter calls local `gh` with argument arrays and a fixed GraphQL query. + +The service accepts one canonical PR URL and verifies that its repository and +head match the locally selected evidence. It does not use a general GitHub SDK +or add a production dependency. + +### 4. Compose T-Rex by receipt, not by execution + +The contribution input declares `optional`, `required`, or `not_applicable` and +may identify one contained T-Rex JSON receipt. The adapter reads only the fields +needed for source identity, verdict, preview identity, limitations, and receipt +identity. Unknown schema or malformed evidence becomes no-confidence. + +This keeps T-Rex beside performance while preserving authority: performance +cannot turn a failed browser flow into a pass, and T-Rex cannot claim a speedup. +Launching T-Rex from the runtime MCP was rejected because it would expand a +closed local performance server into a browser/network execution authority. + +### 5. Derive readiness lexicographically + +```mermaid +stateDiagram-v2 + [*] --> LocallyProven + LocallyProven --> QualityRequired: promotion keep + QualityRequired --> ContributionObserved: challenge qualified + ContributionObserved --> Stale: PR or T-Rex head differs + ContributionObserved --> ReviewAction: current maintainer thread + ContributionObserved --> ChecksPending: pending or approval required + ContributionObserved --> FlowBlocked: required T-Rex fails or is missing + ContributionObserved --> Ready: every configured gate passes + Ready --> WaitingForMaintainer: external merge authority + Ready --> Merged: current head merged + Stale --> ContributionObserved: new head locally reverified + ReviewAction --> ContributionObserved: revised head or resolved feedback + ChecksPending --> ContributionObserved: checks refresh + FlowBlocked --> ContributionObserved: current T-Rex receipt supplied +``` + +The receipt keeps all gates even when an earlier gate controls the overall +status. A weighted score was rejected because a large speedup must never offset +wrong behavior, stale identity, or actionable review. + +### 6. Make refresh explicit for the local MVP + +CLI and MCP expose challenge, inspect, and refresh operations. No daemon or +hosted webhook is added. Each refresh reads the current PR once and emits a new +current receipt while retaining earlier receipts as bounded history. + +Background polling was rejected because it creates hidden resource and API +cost, conflicts with the local-first product boundary, and is unnecessary to +prove the contract. + +### 7. Optimize contributor preparation, not maintainer ceremony + +Candidate challenge runs before publication whenever possible. GitHub refresh +only consumes evidence that already exists, and a maintainer-owned terminal +state ends in quiet waiting. The adapter contains no mutation command and the +MCP annotations remain read-only for inspection and refresh. + +Installing a GitHub App, creating a required status check, posting a large +benchmark report, automatically requesting re-review, or reminding maintainers +was rejected. Those actions externalize the contributor's verification cost +onto open-source projects. A future explicit author action may copy a concise +summary, but it is not part of this service. + +### 8. Derive learning and publication from receipt transitions + +When a reviewed candidate is superseded, the service reads the prior immutable +challenge and carries the actionable thread into a bounded learning record. It +records exact before/after revisions and complexity, prior risk signals, the +revised campaign hypothesis, repeated gate status, and current upstream +disposition. It does not infer a universal style rule or ask a maintainer to +classify the feedback. + +`publication.json` is a replaceable projection, not evidence authority. A +current receipt regenerates it. Head drift only marks the existing projection +stale while preserving its source digest; no stale receipt can silently become +the new publication. The append-only receipt ledger and immutable challenges +remain authoritative. + +## Risks / Trade-offs + +- **Diff-token signals can over-report complexity** → treat them as evidence + requests, never automatic rejection; allow explicit invariant justification. +- **GitHub status models vary across check runs and status contexts** → preserve + raw bounded observed kinds and normalize only the small readiness taxonomy. +- **T-Rex is not applicable to every Node or Go project** → require an explicit + policy and keep `not_applicable` distinct from missing evidence. +- **A PR can change immediately after inspection** → bind the receipt to the + observed head and require refresh before terminal closeout. +- **Repeated refreshes can consume GitHub API quota** → remain operator- + triggered, make normalization idempotent, and perform no background polling. +- **Issue #111 is broad** → ship one hermetic Marked-shaped fixture and closed + operations first; UI and hosted monitoring remain out of scope. + +## Migration Plan + +The change is additive. Existing campaign and T-Rex artifacts remain readable +and unchanged. New challenge and contribution files live under the existing +contained campaign directory. Rollback removes the new CLI/MCP operations and +artifacts without touching campaign history, T-Rex storage, desktop data, or +external pull requests. diff --git a/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/proposal.md b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/proposal.md new file mode 100644 index 00000000..f9143d27 --- /dev/null +++ b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/proposal.md @@ -0,0 +1,69 @@ +## Why + +The autonomous performance lab can prove that a candidate is correct for its +declared scopes and materially faster, but it can still preserve unnecessary +implementation complexity or describe an external pull request as healthy +without reading inline maintainer feedback. The Marked optimization exposed +both failures and also showed that performance, T-Rex flow verification, patch +quality, and upstream contribution state need independent, revision-bound +verdicts. + +## What Changes + +- Add a mandatory post-promotion candidate challenge that records observable + complexity, requires either a simpler-candidate comparison or a bounded + not-applicable reason, and refuses publication while patch quality is + unqualified. +- Add a closed, SHA-bound optimization contribution receipt that keeps + performance, correctness, patch quality, T-Rex, checks, reviews, approvals, + merge authority, and artifact freshness separate. +- Add read-only GitHub contribution inspection for checks and thread-aware + review evidence, including actionable current threads, outdated threads, + approval-required workflows, external-preview authorization, and terminal PR + state. +- Compose an existing T-Rex receipt as optional browser-flow correctness + evidence. When supplied, its source head must match the optimization + candidate and its failure or no-confidence verdict cannot be overridden by a + speedup. +- Invalidate derived contribution evidence when the PR head changes and retain + the prior receipt as history. +- Expose the new behavior through dependency-free JSON CLI and repository- + scoped MCP operations, with documentation and a Marked-shaped hermetic + qualification fixture. +- Keep the workflow entirely author-side by default: maintainers do not install + CodeVetter, add repository configuration, receive bot comments or reminders, + resolve CodeVetter-owned checks, or review raw local receipts. +- Keep the first implementation local and operator-triggered: it does not post + comments, resolve reviews, merge PRs, deploy previews, poll in the background, + or execute hosted load tests. + +## Capabilities + +### New Capabilities + +- `optimization-contribution-closeout`: Defines candidate simplification, + SHA-bound contribution receipts, thread-aware upstream inspection, optional + T-Rex evidence composition, invalidation, and honest terminal states. + +### Modified Capabilities + +None. T-Rex continues to produce its canonical receipt unchanged; this change +consumes that receipt without weakening the existing automatic-change- +verification contract. + +## Impact + +- Extends `scripts/runtime-failure-capsule/` campaign contracts, service, CLI, + MCP, tests, and operator documentation. +- Reads public or already-authorized GitHub pull-request metadata through the + existing local `gh` CLI; it adds no production dependency or credential + store. +- Reads an explicitly supplied repository-contained T-Rex receipt but does not + launch T-Rex or a preview itself. +- Adds no GitHub App, required check, webhook, maintainer configuration, bot + identity, or automated PR message. +- Adds no desktop route, database migration, hosted service, deployment, + production configuration, paid-model call, or automatic network workload. +- Implements the active scope of GitHub issue #111. Adjacent issues #52, #97, + and #105 remain independently evidence-gated and will be reconciled after + qualification rather than closed by association. diff --git a/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/specs/optimization-contribution-closeout/spec.md b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/specs/optimization-contribution-closeout/spec.md new file mode 100644 index 00000000..bf5e7f0d --- /dev/null +++ b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/specs/optimization-contribution-closeout/spec.md @@ -0,0 +1,200 @@ +## Purpose + +Define how a proven local optimization becomes a revision-bound, review-aware +contribution without conflating runtime improvement, correctness, patch quality, +T-Rex flow evidence, or upstream acceptance. + +## ADDED Requirements + +### Requirement: Candidate quality is challenged before contribution readiness +The system SHALL require a bounded candidate-quality challenge for the selected +promotion record before it can report an optimization contribution as ready. +The challenge MUST identify the selected candidate, record deterministic diff +complexity, and contain either a comparison with another qualified candidate or +an explicit bounded reason that a simpler-candidate attempt is not applicable. +When the qualified performance evidence contains smaller-input or secondary +metrics, the comparison MUST retain them as control metrics and apply the same +declared tolerance before preferring a candidate. + +#### Scenario: Simpler qualified candidate is available +- **WHEN** two correctness-preserving promotion records address the same + optimization scope and the simpler candidate remains inside the declared + performance tolerance +- **THEN** the challenge withholds readiness until the simpler candidate is + selected and records the target, control, and complexity basis + +#### Scenario: Simpler candidate regresses a control +- **WHEN** the simpler candidate remains inside tolerance on the target metric + but exceeds tolerance on a recorded smaller-input or secondary metric +- **THEN** patch quality remains `no_confidence` + +#### Scenario: Candidate adds defensive complexity +- **WHEN** the selected diff adds mutable state, caching, fallback, or cleanup + signals without a simpler comparison or explicit invariant +- **THEN** patch quality remains `no_confidence` and contribution readiness is + withheld + +#### Scenario: Simplification is not applicable +- **WHEN** the selected candidate adds no flagged defensive complexity and the + caller supplies a bounded not-applicable reason +- **THEN** the challenge may report `retained_with_justification` without + fabricating an alternative candidate + +### Requirement: Contribution receipts keep evidence gates independent +The system SHALL emit a closed, versioned contribution receipt bound to the +repository, pull request, baseline revision, candidate revision, selected +campaign record, and challenge evidence. Performance, correctness, patch +quality, T-Rex, checks, reviews, approvals, merge authority, pull-request state, +and artifact freshness MUST remain independent fields. + +#### Scenario: Performance is confirmed while review is actionable +- **WHEN** the selected candidate has qualified performance and correctness but + a current maintainer thread requires action +- **THEN** the receipt preserves `performance_confirmed` and reports + `review_action_required` without describing the contribution as ready + +#### Scenario: Repository checks require approval +- **WHEN** a current-head workflow reports approval-required and no repository + test has failed +- **THEN** checks report `approval_required` rather than `failed` + +### Requirement: GitHub inspection is thread-aware and read-only +The system SHALL inspect one canonical GitHub pull request through bounded +read-only operations and capture exact head identity, state, mergeability, +checks, reviews, and inline review threads. It MUST distinguish current, +outdated, and resolved threads and MUST NOT post comments, resolve threads, +approve, merge, deploy, or change repository settings. + +#### Scenario: Submitted review body is empty +- **WHEN** a maintainer review has an empty top-level body but contains current + unresolved inline comments +- **THEN** the comments are retained as actionable thread evidence + +#### Scenario: Feedback becomes outdated after revision +- **WHEN** the pull-request head changes and prior unresolved comments no longer + anchor to the current diff +- **THEN** the comments remain in history as outdated and do not independently + block the revised head + +### Requirement: Contribution evidence is invalidated by head drift +The system MUST compare the inspected pull-request head with the candidate SHA +bound to local evidence. A mismatch SHALL mark the receipt stale, preserve the +observed new head, and prevent a ready verdict until the new candidate is +locally reverified. + +#### Scenario: Pull request advances after verification +- **WHEN** the current pull-request head differs from the candidate SHA in the + challenge and performance evidence +- **THEN** artifact freshness reports `stale` and identifies the exact expected + and observed revisions + +#### Scenario: Reinspection is idempotent +- **WHEN** the same current pull-request evidence is inspected again +- **THEN** normalized receipt content remains stable apart from its explicit + observation timestamp + +### Requirement: Review learning and publication remain revision-bound +The system SHALL preserve current actionable maintainer feedback as bounded +learning only when a later locally verified candidate supersedes the reviewed +candidate. The record MUST retain the source and revised candidate revisions, +feedback location and summary, rejected deterministic risk signals, before and +after complexity, revised hypothesis when available, repeated correctness and +performance status, and observed upstream disposition. A concise publication +projection MUST be generated only from a current receipt; head drift MUST mark +an existing projection stale without replacing its source evidence. + +#### Scenario: Reviewed candidate is simplified +- **WHEN** a candidate with actionable inline feedback is replaced by a new + locally verified candidate for the same pull request +- **THEN** the current receipt preserves one revision-bound feedback-learning + record and the current publication projects it + +#### Scenario: Pull-request head advances before local reverification +- **WHEN** a current publication exists and GitHub reports a different head +- **THEN** the publication becomes `stale` and keeps its original source receipt + until a current challenged receipt regenerates it + +### Requirement: Existing T-Rex receipts compose without losing authority +The system SHALL optionally ingest one repository-contained canonical T-Rex +receipt under an explicit `optional`, `required`, or `not_applicable` policy. +When present, its source head MUST equal the optimization candidate SHA. T-Rex +failure, no-confidence, preview mismatch, and limitations MUST remain visible +and MUST NOT be overridden by performance evidence. + +#### Scenario: Matching T-Rex flow verification passes +- **WHEN** a required T-Rex receipt identifies the candidate head and reports + `passed_with_limits` +- **THEN** the contribution receipt records the pass and retains every T-Rex + limitation beside the other gates + +#### Scenario: T-Rex receipt belongs to an older head +- **WHEN** the T-Rex source head differs from the optimization candidate +- **THEN** T-Rex status and artifact freshness report `stale` and contribution + readiness is withheld + +#### Scenario: Required T-Rex evidence is absent +- **WHEN** policy is `required` and no receipt is supplied +- **THEN** T-Rex status reports `missing` and contribution readiness is withheld + +### Requirement: Contribution readiness is deterministic and conservative +The system SHALL derive one contribution status from the independent gates +using fail-closed precedence. `stale`, correctness failure, T-Rex failure, +actionable review, failing checks, approval-required checks, and pending checks +MUST prevent `ready`. Only a current open pull request with qualified local +evidence, acceptable patch quality, no current actionable review, passing +checks, satisfied configured T-Rex policy, and known merge authority MAY report +`ready`. + +#### Scenario: External maintainer owns merge +- **WHEN** every evidence gate passes but the contributor cannot merge the + upstream repository +- **THEN** the receipt reports `waiting_for_maintainer` rather than `complete` + +#### Scenario: Pull request is merged +- **WHEN** GitHub reports the current candidate head merged and all local + evidence remains current +- **THEN** the receipt may report terminal `merged` + +### Requirement: Machine operations remain bounded and local-first +The system SHALL expose candidate challenge, contribution inspection, and +receipt refresh through closed JSON CLI and repository-scoped MCP operations. +Inputs MUST use repository-contained paths and bounded strings. The first +implementation MUST NOT execute target workloads, launch T-Rex, install +dependencies, start background polling, contact non-GitHub services, or accept +arbitrary commands. + +#### Scenario: Unknown contribution argument is supplied +- **WHEN** a CLI or MCP caller supplies an unknown field, escaping path, or + unsupported policy +- **THEN** the operation fails before inspecting GitHub or writing evidence + +#### Scenario: No T-Rex receipt is configured +- **WHEN** policy is optional and no receipt path is supplied +- **THEN** the operation performs no preview or browser request and records the + missing optional evidence as a limitation + +### Requirement: Maintainer workflow does not expand by default +The system SHALL operate from the contributor's local checkout and existing +read-only pull-request evidence without requiring upstream maintainers to +install CodeVetter, grant an app permission, add configuration, inspect a raw +receipt, resolve a CodeVetter-owned check, or respond to automated messages. It +MUST NOT post comments, request review, send reminders, create required checks, +or modify pull-request metadata unless a future separately authorized operation +explicitly adds that mutation. + +#### Scenario: Contributor refreshes upstream evidence +- **WHEN** the contributor runs contribution inspection after a maintainer + review +- **THEN** CodeVetter reads and classifies the existing feedback locally without + notifying or assigning work to the maintainer + +#### Scenario: Contribution waits on upstream ownership +- **WHEN** every local gate passes and an upstream maintainer owns merge + authority +- **THEN** CodeVetter records `waiting_for_maintainer` without posting a reminder + or requesting another review + +#### Scenario: Author chooses to share evidence +- **WHEN** the contributor wants to update the pull request +- **THEN** CodeVetter may produce a concise copyable summary while keeping raw + receipts local, but this read-only MVP does not publish it automatically diff --git a/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/tasks.md b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/tasks.md new file mode 100644 index 00000000..142b7512 --- /dev/null +++ b/openspec/changes/archive/2026-08-10-add-optimization-contribution-closeout/tasks.md @@ -0,0 +1,34 @@ +## 1. Contracts and Candidate Challenge + +- [x] 1.1 Define strict versioned challenge and contribution receipt contracts with bounded inputs and independent gates +- [x] 1.2 Implement deterministic diff-risk observations, simpler-candidate comparison, and fail-closed patch-quality derivation +- [x] 1.3 Add focused contract and challenge tests, including defensive-complexity and no-alternative cases +- [x] 1.4 Retain target and control metrics when comparing qualified candidates + +## 2. Contribution Evidence + +- [x] 2.1 Implement repository-contained campaign, challenge, and candidate identity validation +- [x] 2.2 Implement read-only thread-aware GitHub normalization with head-drift invalidation +- [x] 2.3 Import optional, required, and not-applicable T-Rex receipts without launching browser or hosted work +- [x] 2.4 Derive conservative contribution readiness and append bounded revision-bound receipts +- [x] 2.5 Persist revision-bound feedback learning and current/stale publication projection + +## 3. Machine Interfaces + +- [x] 3.1 Add closed JSON CLI operations for candidate challenge, contribution inspection, and explicit refresh +- [x] 3.2 Add repository-scoped read-only MCP operations with schemas and annotations matching the CLI +- [x] 3.3 Prove malformed fields, escaping paths, mutation commands, polling, and non-GitHub network work are rejected or absent + +## 4. Qualification and Documentation + +- [x] 4.1 Add a hermetic Marked-shaped fixture covering current, outdated, and resolved feedback plus approval-required checks +- [x] 4.5 Extend the fixture through simplification, head invalidation, learning, and publication regeneration +- [x] 4.2 Add a SHA-matched T-Rex receipt fixture and verify performance and browser-flow gates remain independent +- [x] 4.3 Document the contributor-only workflow, concise evidence summary, limitations, and issue boundaries +- [x] 4.4 Run focused tests, the runtime capsule suite, MCP smoke tests, docs validation, and diff checks + +## 5. Closeout + +- [x] 5.1 Update project status and canonical documentation with the verified contribution-closeout contract +- [x] 5.2 Record independent closeout boundaries for issues 111, 52, 97, and 105 without closing adjacent work by association +- [x] 5.3 Prepare the completed change for canonical spec sync, archive, and publication diff --git a/openspec/specs/optimization-contribution-closeout/spec.md b/openspec/specs/optimization-contribution-closeout/spec.md new file mode 100644 index 00000000..49c030ae --- /dev/null +++ b/openspec/specs/optimization-contribution-closeout/spec.md @@ -0,0 +1,200 @@ +## Purpose + +Define how a proven local optimization becomes a revision-bound, review-aware +contribution without conflating runtime improvement, correctness, patch quality, +T-Rex flow evidence, or upstream acceptance. + +## Requirements + +### Requirement: Candidate quality is challenged before contribution readiness +The system SHALL require a bounded candidate-quality challenge for the selected +promotion record before it can report an optimization contribution as ready. +The challenge MUST identify the selected candidate, record deterministic diff +complexity, and contain either a comparison with another qualified candidate or +an explicit bounded reason that a simpler-candidate attempt is not applicable. +When the qualified performance evidence contains smaller-input or secondary +metrics, the comparison MUST retain them as control metrics and apply the same +declared tolerance before preferring a candidate. + +#### Scenario: Simpler qualified candidate is available +- **WHEN** two correctness-preserving promotion records address the same + optimization scope and the simpler candidate remains inside the declared + performance tolerance +- **THEN** the challenge withholds readiness until the simpler candidate is + selected and records the target, control, and complexity basis + +#### Scenario: Simpler candidate regresses a control +- **WHEN** the simpler candidate remains inside tolerance on the target metric + but exceeds tolerance on a recorded smaller-input or secondary metric +- **THEN** patch quality remains `no_confidence` + +#### Scenario: Candidate adds defensive complexity +- **WHEN** the selected diff adds mutable state, caching, fallback, or cleanup + signals without a simpler comparison or explicit invariant +- **THEN** patch quality remains `no_confidence` and contribution readiness is + withheld + +#### Scenario: Simplification is not applicable +- **WHEN** the selected candidate adds no flagged defensive complexity and the + caller supplies a bounded not-applicable reason +- **THEN** the challenge may report `retained_with_justification` without + fabricating an alternative candidate + +### Requirement: Contribution receipts keep evidence gates independent +The system SHALL emit a closed, versioned contribution receipt bound to the +repository, pull request, baseline revision, candidate revision, selected +campaign record, and challenge evidence. Performance, correctness, patch +quality, T-Rex, checks, reviews, approvals, merge authority, pull-request state, +and artifact freshness MUST remain independent fields. + +#### Scenario: Performance is confirmed while review is actionable +- **WHEN** the selected candidate has qualified performance and correctness but + a current maintainer thread requires action +- **THEN** the receipt preserves `performance_confirmed` and reports + `review_action_required` without describing the contribution as ready + +#### Scenario: Repository checks require approval +- **WHEN** a current-head workflow reports approval-required and no repository + test has failed +- **THEN** checks report `approval_required` rather than `failed` + +### Requirement: GitHub inspection is thread-aware and read-only +The system SHALL inspect one canonical GitHub pull request through bounded +read-only operations and capture exact head identity, state, mergeability, +checks, reviews, and inline review threads. It MUST distinguish current, +outdated, and resolved threads and MUST NOT post comments, resolve threads, +approve, merge, deploy, or change repository settings. + +#### Scenario: Submitted review body is empty +- **WHEN** a maintainer review has an empty top-level body but contains current + unresolved inline comments +- **THEN** the comments are retained as actionable thread evidence + +#### Scenario: Feedback becomes outdated after revision +- **WHEN** the pull-request head changes and prior unresolved comments no longer + anchor to the current diff +- **THEN** the comments remain in history as outdated and do not independently + block the revised head + +### Requirement: Contribution evidence is invalidated by head drift +The system MUST compare the inspected pull-request head with the candidate SHA +bound to local evidence. A mismatch SHALL mark the receipt stale, preserve the +observed new head, and prevent a ready verdict until the new candidate is +locally reverified. + +#### Scenario: Pull request advances after verification +- **WHEN** the current pull-request head differs from the candidate SHA in the + challenge and performance evidence +- **THEN** artifact freshness reports `stale` and identifies the exact expected + and observed revisions + +#### Scenario: Reinspection is idempotent +- **WHEN** the same current pull-request evidence is inspected again +- **THEN** normalized receipt content remains stable apart from its explicit + observation timestamp + +### Requirement: Review learning and publication remain revision-bound +The system SHALL preserve current actionable maintainer feedback as bounded +learning only when a later locally verified candidate supersedes the reviewed +candidate. The record MUST retain the source and revised candidate revisions, +feedback location and summary, rejected deterministic risk signals, before and +after complexity, revised hypothesis when available, repeated correctness and +performance status, and observed upstream disposition. A concise publication +projection MUST be generated only from a current receipt; head drift MUST mark +an existing projection stale without replacing its source evidence. + +#### Scenario: Reviewed candidate is simplified +- **WHEN** a candidate with actionable inline feedback is replaced by a new + locally verified candidate for the same pull request +- **THEN** the current receipt preserves one revision-bound feedback-learning + record and the current publication projects it + +#### Scenario: Pull-request head advances before local reverification +- **WHEN** a current publication exists and GitHub reports a different head +- **THEN** the publication becomes `stale` and keeps its original source receipt + until a current challenged receipt regenerates it + +### Requirement: Existing T-Rex receipts compose without losing authority +The system SHALL optionally ingest one repository-contained canonical T-Rex +receipt under an explicit `optional`, `required`, or `not_applicable` policy. +When present, its source head MUST equal the optimization candidate SHA. T-Rex +failure, no-confidence, preview mismatch, and limitations MUST remain visible +and MUST NOT be overridden by performance evidence. + +#### Scenario: Matching T-Rex flow verification passes +- **WHEN** a required T-Rex receipt identifies the candidate head and reports + `passed_with_limits` +- **THEN** the contribution receipt records the pass and retains every T-Rex + limitation beside the other gates + +#### Scenario: T-Rex receipt belongs to an older head +- **WHEN** the T-Rex source head differs from the optimization candidate +- **THEN** T-Rex status and artifact freshness report `stale` and contribution + readiness is withheld + +#### Scenario: Required T-Rex evidence is absent +- **WHEN** policy is `required` and no receipt is supplied +- **THEN** T-Rex status reports `missing` and contribution readiness is withheld + +### Requirement: Contribution readiness is deterministic and conservative +The system SHALL derive one contribution status from the independent gates +using fail-closed precedence. `stale`, correctness failure, T-Rex failure, +actionable review, failing checks, approval-required checks, and pending checks +MUST prevent `ready`. Only a current open pull request with qualified local +evidence, acceptable patch quality, no current actionable review, passing +checks, satisfied configured T-Rex policy, and known merge authority MAY report +`ready`. + +#### Scenario: External maintainer owns merge +- **WHEN** every evidence gate passes but the contributor cannot merge the + upstream repository +- **THEN** the receipt reports `waiting_for_maintainer` rather than `complete` + +#### Scenario: Pull request is merged +- **WHEN** GitHub reports the current candidate head merged and all local + evidence remains current +- **THEN** the receipt may report terminal `merged` + +### Requirement: Machine operations remain bounded and local-first +The system SHALL expose candidate challenge, contribution inspection, and +receipt refresh through closed JSON CLI and repository-scoped MCP operations. +Inputs MUST use repository-contained paths and bounded strings. The first +implementation MUST NOT execute target workloads, launch T-Rex, install +dependencies, start background polling, contact non-GitHub services, or accept +arbitrary commands. + +#### Scenario: Unknown contribution argument is supplied +- **WHEN** a CLI or MCP caller supplies an unknown field, escaping path, or + unsupported policy +- **THEN** the operation fails before inspecting GitHub or writing evidence + +#### Scenario: No T-Rex receipt is configured +- **WHEN** policy is optional and no receipt path is supplied +- **THEN** the operation performs no preview or browser request and records the + missing optional evidence as a limitation + +### Requirement: Maintainer workflow does not expand by default +The system SHALL operate from the contributor's local checkout and existing +read-only pull-request evidence without requiring upstream maintainers to +install CodeVetter, grant an app permission, add configuration, inspect a raw +receipt, resolve a CodeVetter-owned check, or respond to automated messages. It +MUST NOT post comments, request review, send reminders, create required checks, +or modify pull-request metadata unless a future separately authorized operation +explicitly adds that mutation. + +#### Scenario: Contributor refreshes upstream evidence +- **WHEN** the contributor runs contribution inspection after a maintainer + review +- **THEN** CodeVetter reads and classifies the existing feedback locally without + notifying or assigning work to the maintainer + +#### Scenario: Contribution waits on upstream ownership +- **WHEN** every local gate passes and an upstream maintainer owns merge + authority +- **THEN** CodeVetter records `waiting_for_maintainer` without posting a reminder + or requesting another review + +#### Scenario: Author chooses to share evidence +- **WHEN** the contributor wants to update the pull request +- **THEN** CodeVetter may produce a concise copyable summary while keeping raw + receipts local, but this read-only MVP does not publish it automatically diff --git a/scripts/runtime-failure-capsule/AUTONOMOUS_OPTIMIZATION_PROGRAM.md b/scripts/runtime-failure-capsule/AUTONOMOUS_OPTIMIZATION_PROGRAM.md index 8cc0225f..f3d69ad9 100644 --- a/scripts/runtime-failure-capsule/AUTONOMOUS_OPTIMIZATION_PROGRAM.md +++ b/scripts/runtime-failure-capsule/AUTONOMOUS_OPTIMIZATION_PROGRAM.md @@ -19,12 +19,31 @@ inspection, one hypothesis, and one bounded edit. worktree operation supplied by the host; never use destructive reset. 6. Treat `promising` only as permission to run paired promotion. Start the MCP server with `--incumbent-repo` pointing to an independently runnable checkout - whose source identity matches the recorded incumbent, then call + whose source identity matches the recorded incumbent. Have the operator + create the candidate commit before promotion; the campaign itself has no + commit authority. Then call `promote_optimization_candidate` with the same hypothesis. 7. Preserve a candidate only when promotion returns `keep`. Record what the benchmark covers, what it does not cover, elapsed local compute, and any limitations. A synthetic speedup is not a product-wide claim. -8. Repeat from status until a deterministic budget or plateau stops the +8. Before publication, call `challenge_optimization_candidate` for the exact + clean `keep` SHA. Supply either one directly comparable qualified candidate or a + bounded reason that a simpler comparison is not applicable. A speedup does + not compensate for unqualified patch complexity. Where promotion evidence + contains smaller-input or secondary metrics, the selected candidate must + remain inside tolerance on those controls as well as the target. +9. After the candidate has an exact commit and pull request, call + `inspect_optimization_contribution`. Keep optional T-Rex browser-flow + evidence independent and SHA-matched. Treat current review threads, stale + heads, failing or approval-required checks, and missing required T-Rex + evidence as separate blockers. +10. Refresh contribution evidence only when the operator asks. Do not poll, + comment, request review, resolve feedback, approve workflows, merge, deploy, + or ask maintainers to inspect CodeVetter's raw local receipts. A revised + current receipt may preserve prior actionable feedback as revision-bound + learning and regenerate the concise local publication projection; head drift + only marks an existing projection stale. +11. Repeat from status until a deterministic budget or plateau stops the campaign. Never call production endpoints, cloud runners, paid models, dependency installers, migration tools, commit, push, or deploy from this loop unless the operator separately authorizes them. diff --git a/scripts/runtime-failure-capsule/campaign-cli.mjs b/scripts/runtime-failure-capsule/campaign-cli.mjs index ff903bb6..f1b20bab 100644 --- a/scripts/runtime-failure-capsule/campaign-cli.mjs +++ b/scripts/runtime-failure-capsule/campaign-cli.mjs @@ -3,16 +3,35 @@ import { resolve } from 'node:path'; import { createOptimizationCampaignService } from './campaign.mjs'; +import { createOptimizationContributionService } from './contribution.mjs'; import { redactText } from './redact.mjs'; -const OPERATIONS = new Set(['init', 'baseline', 'screen', 'promote', 'inspect', 'status']); +const OPERATIONS = new Set([ + 'init', + 'baseline', + 'screen', + 'promote', + 'inspect', + 'status', + 'challenge', + 'inspect-contribution', + 'refresh-contribution', +]); export async function main(argv = process.argv.slice(2), overrides = {}) { let repositoryRoot = resolve(process.cwd()); try { const { operation, options } = parseArguments(argv); repositoryRoot = resolve(options.repo ?? process.cwd()); - const service = overrides.service ?? (await createOptimizationCampaignService(repositoryRoot)); + const contributionOperation = [ + 'challenge', + 'inspect-contribution', + 'refresh-contribution', + ].includes(operation); + const service = contributionOperation + ? (overrides.contributionService ?? + (await createOptimizationContributionService(repositoryRoot))) + : (overrides.service ?? (await createOptimizationCampaignService(repositoryRoot))); const input = { campaign_directory: required(options, 'campaign') }; let result; if (operation === 'init') result = await service.initialize(input); @@ -26,7 +45,30 @@ export async function main(argv = process.argv.slice(2), overrides = {}) { incumbent_repository: required(options, 'incumbent-repo'), }); } else if (operation === 'inspect') result = await service.inspect(input); - else result = await service.status(input); + else if (operation === 'status') result = await service.status(input); + else if (operation === 'challenge') { + result = await service.challenge({ + ...input, + selected_sequence: requiredInteger(options, 'selected-sequence'), + ...(options['comparison-sequence'] === undefined + ? {} + : { comparison_sequence: requiredInteger(options, 'comparison-sequence') }), + ...(options.justification === undefined + ? {} + : { simpler_not_applicable_reason: options.justification }), + }); + } else { + result = await service[operation === 'inspect-contribution' ? 'inspect' : 'refresh']({ + ...input, + challenge_path: required(options, 'challenge'), + pull_request_url: required(options, 'pr'), + trex_policy: required(options, 'trex-policy'), + ...(options['trex-receipt'] === undefined ? {} : { trex_receipt: options['trex-receipt'] }), + ...(options['trex-reason'] === undefined + ? {} + : { trex_not_applicable_reason: options['trex-reason'] }), + }); + } writeJson(result, overrides.stdout); return exitCode(result); } catch (error) { @@ -52,7 +94,7 @@ function parseArguments(argv) { const [operation, ...rest] = normalizedArguments; if (!OPERATIONS.has(operation)) { throw new Error( - 'usage: campaign-cli.mjs --campaign PATH [--repo PATH] [operation options] [--json]' + 'usage: campaign-cli.mjs --campaign PATH [--repo PATH] [operation options] [--json]' ); } const normalized = rest[0] === '--' ? rest.slice(1) : rest; @@ -80,13 +122,15 @@ function parseArguments(argv) { options[key] = value; index += 1; } - const allowed = new Set( - operation === 'screen' - ? ['repo', 'campaign', 'hypothesis', 'json'] - : operation === 'promote' - ? ['repo', 'campaign', 'hypothesis', 'incumbent-repo', 'json'] - : ['repo', 'campaign', 'json'] - ); + let operationOptions = []; + if (operation === 'screen') operationOptions = ['hypothesis']; + else if (operation === 'promote') operationOptions = ['hypothesis', 'incumbent-repo']; + else if (operation === 'challenge') { + operationOptions = ['selected-sequence', 'comparison-sequence', 'justification']; + } else if (['inspect-contribution', 'refresh-contribution'].includes(operation)) { + operationOptions = ['challenge', 'pr', 'trex-policy', 'trex-receipt', 'trex-reason']; + } + const allowed = new Set(['repo', 'campaign', 'json', ...operationOptions]); const unknown = Object.keys(options).find((key) => !allowed.has(key)); if (unknown) throw new Error(`unknown option for ${operation}: --${unknown}`); return { operation, options }; @@ -98,10 +142,26 @@ function required(options, key) { return value; } +function requiredInteger(options, key) { + const value = required(options, key); + if (!/^\d+$/.test(value)) throw new Error(`--${key} must be a non-negative integer`); + return Number(value); +} + function exitCode(result) { const decision = result?.record?.decision?.status ?? result?.latest?.decision?.status; if (['discard', 'crash'].includes(decision)) return 1; if (decision === 'no_confidence' || result?.status === 'no_confidence') return 2; + if (result?.challenge?.patch_quality?.status === 'no_confidence') return 2; + const contribution = result?.receipt?.status; + if ( + ['review_action_required', 'checks_failed', 'pull_request_not_ready'].includes(contribution) + ) { + return 1; + } + if (contribution && !['ready', 'waiting_for_maintainer', 'merged'].includes(contribution)) { + return 2; + } return 0; } diff --git a/scripts/runtime-failure-capsule/campaign.mjs b/scripts/runtime-failure-capsule/campaign.mjs index 91ab8493..ec740dfa 100644 --- a/scripts/runtime-failure-capsule/campaign.mjs +++ b/scripts/runtime-failure-capsule/campaign.mjs @@ -54,6 +54,7 @@ export async function createOptimizationCampaignService(repositoryRoot, override promote: (input) => promoteCampaign(root, input, dependencies), inspect: (input) => inspectCampaign(root, input, dependencies), status: (input) => statusCampaign(root, input, dependencies), + evidence: (input) => inspectCampaignEvidence(root, input), }; } @@ -371,6 +372,16 @@ async function statusCampaign(root, input, dependencies) { return deriveCampaignStatus(campaign.manifest, campaign.records, dependencies.now()); } +async function inspectCampaignEvidence(root, input) { + const campaign = await loadCampaign(root, requiredDirectory(input)); + if (!Number.isInteger(input.record_sequence) || input.record_sequence < 0) { + throw new Error('record_sequence must be a non-negative integer'); + } + const record = campaign.records[input.record_sequence]; + if (!record?.performance) throw new Error('campaign record has no performance evidence'); + return { record, evidence: await loadEvidence(campaign, record.performance) }; +} + export function deriveCampaignStatus(manifest, records, now = new Date()) { const incumbent = [...records] .reverse() @@ -764,7 +775,7 @@ function correctnessDecision(status) { }; } -async function inspectRepositoryState(root, manifest) { +export async function inspectRepositoryState(root, manifest) { const baseRevision = ( await runGit(root, ['rev-parse', `${manifest.repository_revision}^{commit}`]) ).stdout.trim(); diff --git a/scripts/runtime-failure-capsule/campaign.test.mjs b/scripts/runtime-failure-capsule/campaign.test.mjs index 63be1e68..2ddf313e 100644 --- a/scripts/runtime-failure-capsule/campaign.test.mjs +++ b/scripts/runtime-failure-capsule/campaign.test.mjs @@ -66,6 +66,51 @@ test('campaign CLI is closed, emits JSON, and preserves decision exit semantics' ); assert.equal(invalid, 2); assert.match(JSON.parse(output).error.message, /unknown option/); + + output = ''; + let challengeInput = null; + const contributionService = { + challenge: (input) => { + challengeInput = input; + return { challenge: { patch_quality: { status: 'retained_with_justification' } } }; + }, + }; + const challenged = await campaignCli( + [ + 'challenge', + '--campaign', + '.codevetter/optimization-campaigns/hermetic', + '--selected-sequence', + '4', + '--justification', + 'No simpler candidate is applicable.', + '--json', + ], + { contributionService, stdout } + ); + assert.equal(challenged, 0); + assert.equal(challengeInput.selected_sequence, 4); + assert.equal(challengeInput.simpler_not_applicable_reason, 'No simpler candidate is applicable.'); + + output = ''; + const rejectedContributionOption = await campaignCli( + [ + 'inspect-contribution', + '--campaign', + '.codevetter/optimization-campaigns/hermetic', + '--challenge', + '.codevetter/optimization-campaigns/hermetic/closeout/challenge.json', + '--pr', + 'https://github.com/example/repo/pull/1', + '--trex-policy', + 'optional', + '--command', + 'gh pr comment', + ], + { contributionService, stdout } + ); + assert.equal(rejectedContributionOption, 2); + assert.match(JSON.parse(output).error.message, /unknown option/); }); test('campaign rejects faster incorrect work, promotes paired evidence, resumes, and stops on plateau', async () => { diff --git a/scripts/runtime-failure-capsule/contribution-contracts.mjs b/scripts/runtime-failure-capsule/contribution-contracts.mjs new file mode 100644 index 00000000..b94919d8 --- /dev/null +++ b/scripts/runtime-failure-capsule/contribution-contracts.mjs @@ -0,0 +1,546 @@ +import { sha256, stableStringify } from './campaign-contracts.mjs'; + +export const CANDIDATE_CHALLENGE_SCHEMA_VERSION = 'optimization-candidate-challenge/v1'; +export const CONTRIBUTION_RECEIPT_SCHEMA_VERSION = 'optimization-contribution-receipt/v1'; +export const CONTRIBUTION_PUBLICATION_SCHEMA_VERSION = 'optimization-contribution-publication/v1'; + +export const CONTRIBUTION_LIMITS = Object.freeze({ + justificationCharacters: 1_000, + pullRequestUrlCharacters: 300, + receiptBytes: 1024 * 1024, + riskSignals: 16, + reviewThreads: 100, + checkRuns: 100, + feedbackLearning: 20, +}); + +const PATCH_QUALITY = new Set([ + 'simpler_candidate_selected', + 'retained_with_justification', + 'no_confidence', +]); +const CHECK_STATUSES = new Set([ + 'passed', + 'pending', + 'approval_required', + 'failed', + 'not_observed', +]); +const TREX_STATUSES = new Set([ + 'passed_with_limits', + 'failed', + 'no_confidence', + 'missing', + 'missing_optional', + 'not_applicable', + 'stale', +]); + +export function createDigestedArtifact(schemaVersion, payload, digestField) { + const value = { schema_version: schemaVersion, ...payload, [digestField]: null }; + value[digestField] = artifactDigest(value, digestField); + return value; +} + +export function artifactDigest(value, digestField) { + const payload = { ...value }; + delete payload[digestField]; + return sha256(stableStringify(payload)); +} + +export function assertCandidateChallenge(value) { + const errors = []; + object(value, 'challenge', errors); + if (errors.length === 0) { + closed( + value, + [ + 'schema_version', + 'campaign_id', + 'created_at', + 'candidate', + 'comparison', + 'diff_observations', + 'patch_quality', + 'challenge_digest', + ], + 'challenge', + errors + ); + exact(value.schema_version, CANDIDATE_CHALLENGE_SCHEMA_VERSION, 'schema_version', errors); + text(value.campaign_id, 'campaign_id', errors, 64); + timestamp(value.created_at, 'created_at', errors); + candidate(value.candidate, 'candidate', errors); + if (value.comparison !== null) candidate(value.comparison, 'comparison', errors); + diffObservations(value.diff_observations, errors); + patchQuality(value.patch_quality, errors); + digest(value.challenge_digest, 'challenge_digest', errors); + if ( + typeof value.challenge_digest === 'string' && + value.challenge_digest !== artifactDigest(value, 'challenge_digest') + ) { + errors.push('challenge_digest is invalid'); + } + } + if (errors.length > 0) throw new Error(`invalid candidate challenge: ${errors.join('; ')}`); + return value; +} + +export function assertContributionReceipt(value) { + const errors = []; + object(value, 'receipt', errors); + if (errors.length === 0) { + closed( + value, + [ + 'schema_version', + 'campaign_id', + 'observed_at', + 'pull_request', + 'evidence', + 'gates', + 'status', + 'limitations', + 'feedback_learning', + 'previous_receipt_digest', + 'receipt_digest', + ], + 'receipt', + errors + ); + exact(value.schema_version, CONTRIBUTION_RECEIPT_SCHEMA_VERSION, 'schema_version', errors); + text(value.campaign_id, 'campaign_id', errors, 64); + timestamp(value.observed_at, 'observed_at', errors); + pullRequest(value.pull_request, errors); + evidence(value.evidence, errors); + gates(value.gates, errors); + text(value.status, 'status', errors, 80); + strings(value.limitations, 'limitations', errors, 50, 500); + feedbackLearning(value.feedback_learning, errors); + if (value.previous_receipt_digest !== null) { + digest(value.previous_receipt_digest, 'previous_receipt_digest', errors); + } + digest(value.receipt_digest, 'receipt_digest', errors); + if ( + typeof value.receipt_digest === 'string' && + value.receipt_digest !== artifactDigest(value, 'receipt_digest') + ) { + errors.push('receipt_digest is invalid'); + } + const derived = deriveContributionStatus(value.gates, value.pull_request); + if (value.status !== derived) errors.push(`status must be derived as ${derived}`); + } + if (errors.length > 0) throw new Error(`invalid contribution receipt: ${errors.join('; ')}`); + return value; +} + +export function assertContributionPublication(value) { + const errors = []; + object(value, 'publication', errors); + if (errors.length === 0) { + closed( + value, + [ + 'schema_version', + 'campaign_id', + 'updated_at', + 'status', + 'source_receipt_digest', + 'candidate_revision', + 'pull_request', + 'summary', + 'feedback_learning', + 'stale_reason', + 'publication_digest', + ], + 'publication', + errors + ); + exact(value.schema_version, CONTRIBUTION_PUBLICATION_SCHEMA_VERSION, 'schema_version', errors); + text(value.campaign_id, 'campaign_id', errors, 64); + timestamp(value.updated_at, 'updated_at', errors); + if (!['current', 'stale'].includes(value.status)) errors.push('publication.status is invalid'); + digest(value.source_receipt_digest, 'source_receipt_digest', errors); + revision(value.candidate_revision, 'candidate_revision', errors); + pullRequest(value.pull_request, errors); + publicationSummary(value.summary, errors); + feedbackLearning(value.feedback_learning, errors); + if (value.stale_reason !== null) text(value.stale_reason, 'stale_reason', errors, 1_000); + digest(value.publication_digest, 'publication_digest', errors); + if ( + typeof value.publication_digest === 'string' && + value.publication_digest !== artifactDigest(value, 'publication_digest') + ) { + errors.push('publication_digest is invalid'); + } + } + if (errors.length > 0) throw new Error(`invalid contribution publication: ${errors.join('; ')}`); + return value; +} + +export function deriveContributionStatus(gates, pullRequest) { + if (gates?.freshness?.status === 'stale') return 'stale'; + if (gates?.correctness?.status !== 'passed') return 'correctness_not_proven'; + if (gates?.performance?.status !== 'confirmed') return 'performance_not_proven'; + if (!PATCH_QUALITY.has(gates?.patch_quality?.status)) return 'patch_quality_not_proven'; + if (gates?.patch_quality?.status === 'no_confidence') return 'patch_quality_not_proven'; + if (['failed', 'no_confidence', 'missing', 'stale'].includes(gates?.trex?.status)) { + return 'trex_blocked'; + } + if (gates?.approvals?.status === 'changes_requested') return 'review_action_required'; + if (gates?.reviews?.status === 'action_required') return 'review_action_required'; + if (gates?.checks?.status === 'failed') return 'checks_failed'; + if (gates?.checks?.status === 'approval_required') return 'checks_approval_required'; + if (gates?.checks?.status === 'pending') return 'checks_pending'; + if (gates?.checks?.status === 'not_observed') return 'checks_not_observed'; + if (pullRequest?.state === 'merged') return 'merged'; + if (pullRequest?.state !== 'open' || pullRequest?.is_draft) return 'pull_request_not_ready'; + if (gates?.merge_authority?.status === 'external_maintainer') return 'waiting_for_maintainer'; + if (gates?.merge_authority?.status !== 'contributor') return 'merge_authority_unknown'; + return 'ready'; +} + +function candidate(value, label, errors) { + if (!object(value, label, errors)) return; + closed( + value, + [ + 'sequence', + 'record_digest', + 'candidate_revision', + 'diff_digest', + 'complexity', + 'performance_metric', + 'control_metrics', + ], + label, + errors + ); + integer(value.sequence, `${label}.sequence`, errors, 0, 300); + digest(value.record_digest, `${label}.record_digest`, errors); + revision(value.candidate_revision, `${label}.candidate_revision`, errors); + digest(value.diff_digest, `${label}.diff_digest`, errors); + complexity(value.complexity, `${label}.complexity`, errors); + if (value.performance_metric !== null) { + performanceMetric(value.performance_metric, `${label}.performance_metric`, errors); + } + if (!Array.isArray(value.control_metrics) || value.control_metrics.length > 16) { + errors.push(`${label}.control_metrics is invalid`); + } else { + value.control_metrics.forEach((metric, index) => + performanceMetric(metric, `${label}.control_metrics[${index}]`, errors, { label: true }) + ); + } +} + +function performanceMetric(value, label, errors, options = {}) { + if (!object(value, label, errors)) return; + closed( + value, + options.label ? ['label', 'kind', 'value', 'unit'] : ['kind', 'value', 'unit'], + label, + errors + ); + if (options.label) text(value.label, `${label}.label`, errors, 80); + text(value.kind, `${label}.kind`, errors, 80); + number(value.value, `${label}.value`, errors); + text(value.unit, `${label}.unit`, errors, 40); +} + +function complexity(value, label, errors) { + if (!object(value, label, errors)) return; + closed( + value, + ['files_changed', 'added_lines', 'deleted_lines', 'delta_added_lines', 'delta_deleted_lines'], + label, + errors + ); + for (const key of Object.keys(value)) + integer(value[key], `${label}.${key}`, errors, -1_000_000, 1_000_000); +} + +function diffObservations(value, errors) { + if (!object(value, 'diff_observations', errors)) return; + closed(value, ['changed_files', 'diff_digest', 'risk_signals'], 'diff_observations', errors); + strings(value.changed_files, 'diff_observations.changed_files', errors, 64, 300); + digest(value.diff_digest, 'diff_observations.diff_digest', errors); + if ( + !Array.isArray(value.risk_signals) || + value.risk_signals.length > CONTRIBUTION_LIMITS.riskSignals + ) { + errors.push('diff_observations.risk_signals is invalid'); + } else { + value.risk_signals.forEach((signal, index) => { + const label = `diff_observations.risk_signals[${index}]`; + if (!object(signal, label, errors)) return; + closed(signal, ['kind', 'occurrences'], label, errors); + text(signal.kind, `${label}.kind`, errors, 80); + integer(signal.occurrences, `${label}.occurrences`, errors, 1, 10_000); + }); + } +} + +function patchQuality(value, errors) { + if (!object(value, 'patch_quality', errors)) return; + closed(value, ['status', 'reason', 'justification'], 'patch_quality', errors); + if (!PATCH_QUALITY.has(value.status)) errors.push('patch_quality.status is invalid'); + text(value.reason, 'patch_quality.reason', errors, 1_000); + if (value.justification !== null) { + text( + value.justification, + 'patch_quality.justification', + errors, + CONTRIBUTION_LIMITS.justificationCharacters + ); + } +} + +function pullRequest(value, errors) { + if (!object(value, 'pull_request', errors)) return; + closed( + value, + ['url', 'repository', 'number', 'head_sha', 'base_sha', 'state', 'is_draft', 'mergeable'], + 'pull_request', + errors + ); + text(value.url, 'pull_request.url', errors, CONTRIBUTION_LIMITS.pullRequestUrlCharacters); + text(value.repository, 'pull_request.repository', errors, 200); + integer(value.number, 'pull_request.number', errors, 1, 10_000_000); + revision(value.head_sha, 'pull_request.head_sha', errors); + revision(value.base_sha, 'pull_request.base_sha', errors); + if (!['open', 'closed', 'merged'].includes(value.state)) + errors.push('pull_request.state is invalid'); + if (typeof value.is_draft !== 'boolean') errors.push('pull_request.is_draft must be boolean'); + text(value.mergeable, 'pull_request.mergeable', errors, 40); +} + +function evidence(value, errors) { + if (!object(value, 'evidence', errors)) return; + closed( + value, + [ + 'campaign_record_digest', + 'baseline_revision', + 'candidate_revision', + 'candidate_diff_digest', + 'challenge_path', + 'challenge_digest', + 'trex_path', + ], + 'evidence', + errors + ); + digest(value.campaign_record_digest, 'evidence.campaign_record_digest', errors); + revision(value.baseline_revision, 'evidence.baseline_revision', errors); + revision(value.candidate_revision, 'evidence.candidate_revision', errors); + digest(value.candidate_diff_digest, 'evidence.candidate_diff_digest', errors); + safePath(value.challenge_path, 'evidence.challenge_path', errors); + digest(value.challenge_digest, 'evidence.challenge_digest', errors); + if (value.trex_path !== null) safePath(value.trex_path, 'evidence.trex_path', errors); +} + +function feedbackLearning(value, errors) { + if (!Array.isArray(value) || value.length > CONTRIBUTION_LIMITS.feedbackLearning) { + errors.push('feedback_learning is invalid'); + return; + } + value.forEach((entry, index) => { + const label = `feedback_learning[${index}]`; + if (!object(entry, label, errors)) return; + closed( + entry, + [ + 'source_candidate_revision', + 'revised_candidate_revision', + 'feedback', + 'rejected_patterns', + 'before_complexity', + 'after_complexity', + 'revised_hypothesis', + 'correctness_status', + 'performance_status', + 'upstream_disposition', + ], + label, + errors + ); + revision(entry.source_candidate_revision, `${label}.source_candidate_revision`, errors); + revision(entry.revised_candidate_revision, `${label}.revised_candidate_revision`, errors); + if ( + !Array.isArray(entry.feedback) || + entry.feedback.length > CONTRIBUTION_LIMITS.reviewThreads + ) { + errors.push(`${label}.feedback is invalid`); + } else { + entry.feedback.forEach((observation, observationIndex) => { + const observationLabel = `${label}.feedback[${observationIndex}]`; + if (!object(observation, observationLabel, errors)) return; + closed(observation, ['author', 'path', 'line', 'summary'], observationLabel, errors); + text(observation.author, `${observationLabel}.author`, errors, 100); + text(observation.path, `${observationLabel}.path`, errors, 300); + if (observation.line !== null) + integer(observation.line, `${observationLabel}.line`, errors, 1, 10_000_000); + text(observation.summary, `${observationLabel}.summary`, errors, 300); + }); + } + strings(entry.rejected_patterns, `${label}.rejected_patterns`, errors, 16, 80); + complexity(entry.before_complexity, `${label}.before_complexity`, errors); + complexity(entry.after_complexity, `${label}.after_complexity`, errors); + if (entry.revised_hypothesis !== null) + text(entry.revised_hypothesis, `${label}.revised_hypothesis`, errors, 1_000); + text(entry.correctness_status, `${label}.correctness_status`, errors, 40); + text(entry.performance_status, `${label}.performance_status`, errors, 40); + text(entry.upstream_disposition, `${label}.upstream_disposition`, errors, 80); + }); +} + +function publicationSummary(value, errors) { + if (!object(value, 'summary', errors)) return; + closed(value, ['performance', 'correctness', 'patch_quality', 'contribution'], 'summary', errors); + text(value.performance, 'summary.performance', errors, 80); + text(value.correctness, 'summary.correctness', errors, 80); + text(value.patch_quality, 'summary.patch_quality', errors, 80); + text(value.contribution, 'summary.contribution', errors, 80); +} + +function gates(value, errors) { + if (!object(value, 'gates', errors)) return; + closed( + value, + [ + 'freshness', + 'correctness', + 'performance', + 'patch_quality', + 'trex', + 'checks', + 'reviews', + 'approvals', + 'merge_authority', + ], + 'gates', + errors + ); + simpleGate(value.freshness, 'freshness', new Set(['current', 'stale']), errors); + simpleGate( + value.correctness, + 'correctness', + new Set(['passed', 'failed', 'no_confidence']), + errors + ); + simpleGate(value.performance, 'performance', new Set(['confirmed', 'no_confidence']), errors); + simpleGate(value.patch_quality, 'patch_quality', PATCH_QUALITY, errors); + simpleGate(value.trex, 'trex', TREX_STATUSES, errors, ['policy', 'limitations']); + simpleGate(value.checks, 'checks', CHECK_STATUSES, errors, ['observations']); + simpleGate(value.reviews, 'reviews', new Set(['clear', 'action_required']), errors, [ + 'current_threads', + 'outdated_threads', + 'resolved_threads', + 'observations', + ]); + simpleGate( + value.approvals, + 'approvals', + new Set(['approved', 'not_observed', 'changes_requested']), + errors, + ['observations'] + ); + simpleGate( + value.merge_authority, + 'merge_authority', + new Set(['contributor', 'external_maintainer', 'unknown']), + errors + ); +} + +function simpleGate(value, label, statuses, errors, optional = []) { + if (!object(value, `gates.${label}`, errors)) return; + closed(value, ['status', 'reason', ...optional], `gates.${label}`, errors); + if (!statuses.has(value.status)) errors.push(`gates.${label}.status is invalid`); + text(value.reason, `gates.${label}.reason`, errors, 1_000); + for (const key of optional) { + if (key === 'policy') text(value[key], `gates.${label}.${key}`, errors, 40); + else if (key === 'limitations') strings(value[key], `gates.${label}.${key}`, errors, 50, 500); + else if (key === 'observations') { + if (!Array.isArray(value[key]) || value[key].length > CONTRIBUTION_LIMITS.checkRuns) { + errors.push(`gates.${label}.${key} is invalid`); + } + } else + integer(value[key], `gates.${label}.${key}`, errors, 0, CONTRIBUTION_LIMITS.reviewThreads); + } +} + +function object(value, label, errors) { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + errors.push(`${label} must be an object`); + return false; + } + return true; +} + +function closed(value, allowed, label, errors) { + for (const key of Object.keys(value)) { + if (!allowed.includes(key)) errors.push(`${label} has unknown field: ${key}`); + } + for (const key of allowed) { + if (!Object.hasOwn(value, key)) errors.push(`${label} is missing field: ${key}`); + } +} + +function exact(value, expected, label, errors) { + if (value !== expected) errors.push(`${label} must equal ${expected}`); +} + +function text(value, label, errors, maximum) { + if (typeof value !== 'string' || value.trim() === '' || value.length > maximum) { + errors.push(`${label} must be non-empty text no longer than ${maximum}`); + } +} + +function strings(value, label, errors, maximumItems, maximumCharacters) { + if (!Array.isArray(value) || value.length > maximumItems) { + errors.push(`${label} must be an array with at most ${maximumItems} items`); + return; + } + value.forEach((entry, index) => text(entry, `${label}[${index}]`, errors, maximumCharacters)); +} + +function integer(value, label, errors, minimum, maximum) { + if (!Number.isInteger(value) || value < minimum || value > maximum) { + errors.push(`${label} must be an integer between ${minimum} and ${maximum}`); + } +} + +function number(value, label, errors) { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) { + errors.push(`${label} must be a finite non-negative number`); + } +} + +function timestamp(value, label, errors) { + if (typeof value !== 'string' || Number.isNaN(Date.parse(value))) + errors.push(`${label} is invalid`); +} + +function revision(value, label, errors) { + if (typeof value !== 'string' || !/^[0-9a-f]{40,64}$/i.test(value)) { + errors.push(`${label} must be a full Git revision`); + } +} + +function digest(value, label, errors) { + if (typeof value !== 'string' || !/^[0-9a-f]{64}$/i.test(value)) { + errors.push(`${label} must be a SHA-256 digest`); + } +} + +function safePath(value, label, errors) { + if ( + typeof value !== 'string' || + value.startsWith('/') || + value.includes('\\') || + value.split('/').some((part) => ['', '.', '..'].includes(part)) + ) { + errors.push(`${label} must be a contained repository-relative path`); + } +} diff --git a/scripts/runtime-failure-capsule/contribution.mjs b/scripts/runtime-failure-capsule/contribution.mjs new file mode 100644 index 00000000..4c2bacf1 --- /dev/null +++ b/scripts/runtime-failure-capsule/contribution.mjs @@ -0,0 +1,1122 @@ +import { spawn } from 'node:child_process'; +import { mkdir, readFile, realpath, rename, rm, stat, writeFile } from 'node:fs/promises'; +import { basename, dirname, isAbsolute, relative, resolve, sep } from 'node:path'; + +import { sha256, stableStringify } from './campaign-contracts.mjs'; +import { createOptimizationCampaignService, inspectRepositoryState } from './campaign.mjs'; +import { + CANDIDATE_CHALLENGE_SCHEMA_VERSION, + CONTRIBUTION_LIMITS, + CONTRIBUTION_PUBLICATION_SCHEMA_VERSION, + CONTRIBUTION_RECEIPT_SCHEMA_VERSION, + assertCandidateChallenge, + assertContributionPublication, + assertContributionReceipt, + createDigestedArtifact, + deriveContributionStatus, +} from './contribution-contracts.mjs'; +import { redactText } from './redact.mjs'; + +const CLOSEOUT_DIRECTORY = 'closeout'; +const RECEIPT_LEDGER = 'contributions.ndjson'; +const PUBLICATION_FILE = 'publication.json'; +const TREX_POLICIES = new Set(['optional', 'required', 'not_applicable']); +const SIMPLER_TOLERANCE_PERCENT = 5; + +const RISK_PATTERNS = Object.freeze([ + ['mutable_state', /^\+.*\b(?:cache|cached|memoized|state)\b/im], + ['class_member_state', /^\+\s*(?:private|protected)\s+(?:readonly\s+)?\w+\s*[:=]/im], + ['cleanup_path', /^\+.*\b(?:finally|defer)\b/im], + ['fallback_path', /^\+.*\b(?:fallback|else)\b/im], + ['new_branch', /^\+.*\b(?:if\s*\(|switch\s*\()/im], + [ + 'public_signature', + /^\+.*\b(?:export\s+(?:async\s+)?(?:function|class|const|let|var)|public\s+\w+)/im, + ], +]); + +export async function createOptimizationContributionService(repositoryRoot, overrides = {}) { + const root = await realpath(resolve(repositoryRoot)); + const dependencies = { + now: () => new Date(), + campaignService: await createOptimizationCampaignService(root), + inspectRepositoryState, + inspectCandidateCommit, + readCandidateDiff, + githubInspector: inspectGitHubPullRequest, + ...overrides, + }; + return { + challenge: (input) => challengeCandidate(root, input, dependencies), + inspect: (input) => inspectContribution(root, input, dependencies), + refresh: (input) => inspectContribution(root, input, dependencies), + }; +} + +async function challengeCandidate(root, input, dependencies) { + closedInput( + input, + ['campaign_directory', 'selected_sequence'], + ['comparison_sequence', 'simpler_not_applicable_reason'] + ); + const campaign = await dependencies.campaignService.inspect({ + campaign_directory: input.campaign_directory, + }); + const selectedRecord = qualifiedPromotion(campaign.records, input.selected_sequence, 'selected'); + const currentRepository = await dependencies.inspectRepositoryState(root, campaign.manifest); + if (currentRepository.diff_digest !== selectedRecord.repository.diff_digest) { + throw new Error('current candidate diff does not match the selected promotion record'); + } + if (currentRepository.revision !== selectedRecord.repository.revision) { + throw new Error('current candidate revision does not match the selected promotion record'); + } + const commitState = await dependencies.inspectCandidateCommit(root, campaign.manifest); + if (!commitState.clean) { + throw new Error( + `candidate must be committed before challenge; uncommitted files: ${commitState.changed_files.slice(0, 8).join(', ')}` + ); + } + const selectedEvidence = await dependencies.campaignService.evidence({ + campaign_directory: input.campaign_directory, + record_sequence: selectedRecord.sequence, + }); + const selected = challengeCandidateIdentity( + selectedRecord, + currentRepository.revision, + performanceMetrics(selectedEvidence.evidence) + ); + let comparison = null; + let comparisonEvidence = null; + if (input.comparison_sequence !== undefined) { + const comparisonRecord = qualifiedPromotion( + campaign.records, + input.comparison_sequence, + 'comparison' + ); + comparisonEvidence = await dependencies.campaignService.evidence({ + campaign_directory: input.campaign_directory, + record_sequence: comparisonRecord.sequence, + }); + comparison = challengeCandidateIdentity( + comparisonRecord, + comparisonRecord.repository.revision, + performanceMetrics(comparisonEvidence.evidence) + ); + } + const diff = await dependencies.readCandidateDiff(root, campaign.manifest, currentRepository); + const observations = { + changed_files: currentRepository.changed_files, + diff_digest: sha256(diff), + risk_signals: riskSignals(diff, currentRepository.changed_files), + }; + const patchQuality = derivePatchQuality({ + selected, + comparison, + selectedEvidence: selectedEvidence.evidence, + comparisonEvidence: comparisonEvidence?.evidence, + riskSignals: observations.risk_signals, + justification: input.simpler_not_applicable_reason, + root, + }); + const challenge = createDigestedArtifact( + CANDIDATE_CHALLENGE_SCHEMA_VERSION, + { + campaign_id: campaign.manifest.campaign_id, + created_at: dependencies.now().toISOString(), + candidate: selected, + comparison, + diff_observations: observations, + patch_quality: patchQuality, + }, + 'challenge_digest' + ); + assertCandidateChallenge(challenge); + const directory = safeCampaignDirectory(input.campaign_directory); + const canonicalCampaign = await realpath(resolve(root, directory)); + assertContained(root, canonicalCampaign, 'campaign directory'); + const path = `${directory}/${CLOSEOUT_DIRECTORY}/challenge-${selected.candidate_revision.slice(0, 12)}-${challenge.challenge_digest.slice(0, 12)}.json`; + await writeContainedArtifact(root, path, `${stableStringify(challenge)}\n`, canonicalCampaign); + return { path, challenge }; +} + +async function inspectContribution(root, input, dependencies) { + closedInput( + input, + ['campaign_directory', 'challenge_path', 'pull_request_url', 'trex_policy'], + ['trex_receipt', 'trex_not_applicable_reason'] + ); + if (!TREX_POLICIES.has(input.trex_policy)) throw new Error('trex_policy is unsupported'); + const campaignDirectory = safeCampaignDirectory(input.campaign_directory); + const canonicalCampaign = await realpath(resolve(root, campaignDirectory)); + assertContained(root, canonicalCampaign, 'campaign directory'); + const challengePath = safeContainedPath(input.challenge_path, 'challenge_path'); + if (!challengePath.startsWith(`${campaignDirectory}/${CLOSEOUT_DIRECTORY}/`)) { + throw new Error('challenge_path must stay inside the selected campaign closeout directory'); + } + const trexPath = input.trex_receipt + ? safeContainedPath(input.trex_receipt, 'trex_receipt') + : null; + validateTrexPolicy(input, trexPath); + const challenge = await readJsonArtifact( + root, + challengePath, + CONTRIBUTION_LIMITS.receiptBytes, + canonicalCampaign + ); + assertCandidateChallenge(challenge); + const campaign = await dependencies.campaignService.inspect({ + campaign_directory: campaignDirectory, + }); + if (challenge.campaign_id !== campaign.manifest.campaign_id) { + throw new Error('challenge campaign identity does not match the selected campaign'); + } + const selectedRecord = qualifiedPromotion( + campaign.records, + challenge.candidate.sequence, + 'challenge candidate' + ); + if (selectedRecord.record_digest !== challenge.candidate.record_digest) { + throw new Error('challenge campaign record digest is stale'); + } + const pullRequest = normalizeGitHubEvidence( + await dependencies.githubInspector(input.pull_request_url) + ); + if (pullRequest.identity.url !== canonicalPullRequestUrl(input.pull_request_url)) { + throw new Error('GitHub returned a different pull request identity'); + } + const trex = await importTrexGate(root, input, challenge.candidate.candidate_revision, trexPath); + const gates = contributionGates(challenge, selectedRecord, pullRequest, trex); + const ledgerPath = `${campaignDirectory}/${CLOSEOUT_DIRECTORY}/${RECEIPT_LEDGER}`; + const publicationPath = `${campaignDirectory}/${CLOSEOUT_DIRECTORY}/${PUBLICATION_FILE}`; + const previous = await readReceiptLedger(root, ledgerPath, canonicalCampaign); + const feedbackLearning = await deriveFeedbackLearning({ + root, + previous, + challenge, + selectedRecord, + pullRequest, + gates, + canonicalCampaign, + }); + const receiptPayload = { + campaign_id: campaign.manifest.campaign_id, + observed_at: dependencies.now().toISOString(), + pull_request: pullRequest.identity, + evidence: { + campaign_record_digest: selectedRecord.record_digest, + baseline_revision: campaign.manifest.repository_revision, + candidate_revision: challenge.candidate.candidate_revision, + candidate_diff_digest: challenge.candidate.diff_digest, + challenge_path: challengePath, + challenge_digest: challenge.challenge_digest, + trex_path: trexPath, + }, + gates, + status: deriveContributionStatus(gates, pullRequest.identity), + limitations: trex.limitations, + feedback_learning: feedbackLearning, + previous_receipt_digest: previous.at(-1)?.receipt_digest ?? null, + }; + const receipt = createDigestedArtifact( + CONTRIBUTION_RECEIPT_SCHEMA_VERSION, + receiptPayload, + 'receipt_digest' + ); + assertContributionReceipt(receipt); + await appendReceipt(root, ledgerPath, previous, receipt, canonicalCampaign); + const publication = await updatePublication(root, publicationPath, receipt, canonicalCampaign); + return { receipt_path: ledgerPath, receipt, publication_path: publicationPath, publication }; +} + +function qualifiedPromotion(records, sequence, label) { + if (!Number.isInteger(sequence) || sequence < 0) { + throw new Error(`${label}_sequence must be a non-negative integer`); + } + const record = records[sequence]; + if (!record || record.sequence !== sequence) throw new Error(`${label} record is unavailable`); + if (record.kind !== 'promotion' || record.decision.status !== 'keep') { + throw new Error(`${label} record must be a qualified keep promotion`); + } + if (!record.repository?.diff_digest) + throw new Error(`${label} repository identity is incomplete`); + return record; +} + +function challengeCandidateIdentity(record, revision, metrics) { + return { + sequence: record.sequence, + record_digest: record.record_digest, + candidate_revision: revision, + diff_digest: record.repository.diff_digest, + complexity: record.complexity, + performance_metric: metrics.target, + control_metrics: metrics.controls, + }; +} + +function performanceMetrics(evidence) { + const verification = evidence?.verification; + const scale = verification?.observed?.find( + (observation) => observation.kind === 'scale_point_comparison' + ); + if (scale?.points?.length > 0) { + const target = scale.points.at(-1); + return { + target: { + kind: 'largest_scale_point', + value: target.current, + unit: target.unit, + }, + controls: scale.points.slice(0, -1).map((point) => ({ + label: `input:${point.input}`, + kind: 'scale_control_point', + value: point.current, + unit: point.unit, + })), + }; + } + const go = verification?.observed?.find( + (observation) => observation.kind === 'go_benchmark_comparison' + ); + if (go?.metrics?.ns_per_op?.current !== undefined) { + return { + target: { kind: 'go_ns_per_op', value: go.metrics.ns_per_op.current, unit: 'ns/op' }, + controls: ['bytes_per_op', 'allocs_per_op'].flatMap((name) => + go.metrics?.[name]?.current === undefined + ? [] + : [ + { + label: name, + kind: `go_${name}`, + value: go.metrics[name].current, + unit: name === 'bytes_per_op' ? 'B/op' : 'allocs/op', + }, + ] + ), + }; + } + const wall = verification?.observed?.find( + (observation) => observation.kind === 'wall_time_comparison' + ); + if (wall?.comparison?.current_median_ms !== undefined) { + return { + target: { kind: 'wall_time', value: wall.comparison.current_median_ms, unit: 'ms' }, + controls: [], + }; + } + return { target: null, controls: [] }; +} + +function derivePatchQuality({ + selected, + comparison, + selectedEvidence, + comparisonEvidence, + riskSignals: signals, + justification, + root, +}) { + const sanitizedJustification = sanitizeJustification(justification, root); + if (comparison) { + const sameWorkload = + selectedEvidence?.verification?.workload_identity?.digest && + selectedEvidence.verification.workload_identity.digest === + comparisonEvidence?.verification?.workload_identity?.digest; + const comparable = + sameWorkload && + selected.performance_metric && + comparison.performance_metric && + selected.performance_metric.kind === comparison.performance_metric.kind && + selected.performance_metric.unit === comparison.performance_metric.unit && + compatibleControls(selected.control_metrics, comparison.control_metrics); + if (!comparable) { + return { + status: 'no_confidence', + reason: 'Candidate performance evidence is not directly comparable on one workload.', + justification: sanitizedJustification, + }; + } + const selectedMovement = codeMovement(selected.complexity); + const comparisonMovement = codeMovement(comparison.complexity); + const withinTolerance = + selected.performance_metric.value <= + comparison.performance_metric.value * (1 + SIMPLER_TOLERANCE_PERCENT / 100) && + selected.control_metrics.every( + (metric, index) => + metric.value <= + comparison.control_metrics[index].value * (1 + SIMPLER_TOLERANCE_PERCENT / 100) + ); + if (selectedMovement <= comparisonMovement && withinTolerance) { + return { + status: 'simpler_candidate_selected', + reason: `Selected candidate is no more complex and remains within ${SIMPLER_TOLERANCE_PERCENT}% on the target and ${selected.control_metrics.length} recorded control metric(s).`, + justification: sanitizedJustification, + }; + } + return { + status: 'no_confidence', + reason: + 'A qualified comparison is simpler or the selected candidate misses the performance tolerance.', + justification: sanitizedJustification, + }; + } + if (!sanitizedJustification) { + return { + status: 'no_confidence', + reason: + signals.length > 0 + ? 'The diff adds defensive complexity without a qualified comparison or invariant justification.' + : 'A bounded reason is required when no simpler-candidate comparison is supplied.', + justification: null, + }; + } + return { + status: 'retained_with_justification', + reason: + signals.length > 0 + ? 'Risk signals are retained as observations and covered by the supplied invariant justification.' + : 'No deterministic defensive-complexity signal was observed; the missing comparison is explicitly bounded.', + justification: sanitizedJustification, + }; +} + +function compatibleControls(selected, comparison) { + return ( + selected.length === comparison.length && + selected.every( + (metric, index) => + metric.label === comparison[index].label && + metric.kind === comparison[index].kind && + metric.unit === comparison[index].unit + ) + ); +} + +function riskSignals(diff, changedFiles) { + const sourceSignals = RISK_PATTERNS.flatMap(([kind, pattern]) => { + const flags = pattern.flags.includes('g') ? pattern.flags : `${pattern.flags}g`; + const matches = String(diff).match(new RegExp(pattern.source, flags)); + return matches?.length ? [{ kind, occurrences: matches.length }] : []; + }); + if ( + changedFiles.some((path) => + /(?:^|\/)(?:package\.json|go\.mod|Cargo\.toml|requirements\.txt|pyproject\.toml)$/.test(path) + ) + ) { + sourceSignals.push({ kind: 'dependency_manifest', occurrences: 1 }); + } + return sourceSignals; +} + +function codeMovement(complexity) { + return Math.max(0, complexity.added_lines) + Math.max(0, complexity.deleted_lines); +} + +function sanitizeJustification(value, root) { + if (value === undefined || value === null || String(value).trim() === '') return null; + if (typeof value !== 'string' || value.length > CONTRIBUTION_LIMITS.justificationCharacters) { + throw new Error('simpler_not_applicable_reason is too long'); + } + return redactText(value.trim(), { + repositoryRoot: root, + limit: CONTRIBUTION_LIMITS.justificationCharacters, + }).text; +} + +async function deriveFeedbackLearning({ + root, + previous, + challenge, + selectedRecord, + pullRequest, + gates, + canonicalCampaign, +}) { + const latest = previous.at(-1); + const carried = latest?.feedback_learning ?? []; + if (latest && latest.pull_request.url !== pullRequest.identity.url) { + throw new Error('contribution ledger belongs to a different pull request'); + } + if (!latest || latest.evidence.candidate_revision === challenge.candidate.candidate_revision) { + return carried; + } + const source = previous.findLast( + (receipt) => + receipt.evidence.candidate_revision !== challenge.candidate.candidate_revision && + receipt.gates.reviews.observations.some( + (observation) => + observation.kind === 'thread' && !observation.resolved && !observation.outdated + ) + ); + if (!source) return carried; + const feedback = source.gates.reviews.observations + .filter( + (observation) => + observation.kind === 'thread' && !observation.resolved && !observation.outdated + ) + .map((observation) => ({ + author: observation.author, + path: observation.path, + line: observation.line ?? null, + summary: observation.summary, + })); + if (feedback.length === 0) return carried; + const priorChallenge = await readJsonArtifact( + root, + source.evidence.challenge_path, + CONTRIBUTION_LIMITS.receiptBytes, + canonicalCampaign + ); + assertCandidateChallenge(priorChallenge); + if (priorChallenge.challenge_digest !== source.evidence.challenge_digest) { + throw new Error('previous contribution challenge digest does not match its receipt'); + } + const learning = { + source_candidate_revision: source.evidence.candidate_revision, + revised_candidate_revision: challenge.candidate.candidate_revision, + feedback, + rejected_patterns: priorChallenge.diff_observations.risk_signals.map((signal) => signal.kind), + before_complexity: priorChallenge.candidate.complexity, + after_complexity: challenge.candidate.complexity, + revised_hypothesis: + typeof selectedRecord.hypothesis === 'string' && selectedRecord.hypothesis.trim() + ? sanitizeJustification(selectedRecord.hypothesis, root) + : null, + correctness_status: gates.correctness.status, + performance_status: gates.performance.status, + upstream_disposition: `${pullRequest.identity.state}:${deriveContributionStatus(gates, pullRequest.identity)}`, + }; + return [...carried, learning].slice(-CONTRIBUTION_LIMITS.feedbackLearning); +} + +async function updatePublication(root, path, receipt, canonicalCampaign) { + const existing = await readPublication(root, path, canonicalCampaign); + let publication; + if (receipt.gates.freshness.status === 'current') { + publication = createDigestedArtifact( + CONTRIBUTION_PUBLICATION_SCHEMA_VERSION, + { + campaign_id: receipt.campaign_id, + updated_at: receipt.observed_at, + status: 'current', + source_receipt_digest: receipt.receipt_digest, + candidate_revision: receipt.evidence.candidate_revision, + pull_request: receipt.pull_request, + summary: { + performance: receipt.gates.performance.status, + correctness: receipt.gates.correctness.status, + patch_quality: receipt.gates.patch_quality.status, + contribution: receipt.status, + }, + feedback_learning: receipt.feedback_learning, + stale_reason: null, + }, + 'publication_digest' + ); + } else if (existing) { + publication = createDigestedArtifact( + CONTRIBUTION_PUBLICATION_SCHEMA_VERSION, + { + campaign_id: existing.value.campaign_id, + updated_at: receipt.observed_at, + status: 'stale', + source_receipt_digest: existing.value.source_receipt_digest, + candidate_revision: existing.value.candidate_revision, + pull_request: existing.value.pull_request, + summary: existing.value.summary, + feedback_learning: existing.value.feedback_learning, + stale_reason: receipt.gates.freshness.reason, + }, + 'publication_digest' + ); + } else { + return null; + } + assertContributionPublication(publication); + await replaceContainedArtifact( + root, + path, + `${stableStringify(publication)}\n`, + existing?.source ?? '', + canonicalCampaign + ); + return publication; +} + +async function readPublication(root, path, containmentRoot) { + const absolute = resolve(root, path); + assertContained(root, absolute, path); + try { + const canonical = await realpath(absolute); + assertContained(containmentRoot, canonical, path); + const details = await stat(canonical); + if (!details.isFile() || details.size > CONTRIBUTION_LIMITS.receiptBytes) { + throw new Error('contribution publication is unavailable or oversized'); + } + const source = await readFile(canonical, 'utf8'); + const value = JSON.parse(source); + assertContributionPublication(value); + return { source, value }; + } catch (error) { + if (error.code === 'ENOENT') return null; + if (error instanceof SyntaxError) throw new Error('contribution publication is not valid JSON'); + throw error; + } +} + +function contributionGates(challenge, record, pullRequest, trex) { + const freshness = + pullRequest.identity.head_sha === challenge.candidate.candidate_revision + ? { + status: 'current', + reason: 'Pull-request head matches the locally challenged candidate revision.', + } + : { + status: 'stale', + reason: `Expected ${challenge.candidate.candidate_revision}; observed ${pullRequest.identity.head_sha}.`, + }; + const correctnessPassed = + Array.isArray(record.correctness) && + record.correctness.length > 0 && + record.correctness.every((result) => result.status === 'passed'); + return { + freshness, + correctness: correctnessPassed + ? { status: 'passed', reason: 'Every correctness result in the selected promotion passed.' } + : { status: 'no_confidence', reason: 'Selected promotion correctness is incomplete.' }, + performance: + record.decision.status === 'keep' + ? { status: 'confirmed', reason: 'Selected campaign promotion met the keep policy.' } + : { status: 'no_confidence', reason: 'Selected campaign promotion is not a keep.' }, + patch_quality: { + status: challenge.patch_quality.status, + reason: challenge.patch_quality.reason, + }, + trex: { + status: trex.status, + reason: trex.reason, + policy: trex.policy, + limitations: trex.limitations, + }, + checks: pullRequest.checks, + reviews: pullRequest.reviews, + approvals: pullRequest.approvals, + merge_authority: pullRequest.mergeAuthority, + }; +} + +async function importTrexGate(root, input, candidateRevision, path) { + if (input.trex_policy === 'not_applicable') { + return { + status: 'not_applicable', + policy: input.trex_policy, + reason: sanitizeJustification(input.trex_not_applicable_reason, root), + limitations: [], + }; + } + if (!path) { + return { + status: input.trex_policy === 'required' ? 'missing' : 'missing_optional', + policy: input.trex_policy, + reason: + input.trex_policy === 'required' + ? 'Required T-Rex evidence was not supplied.' + : 'Optional T-Rex evidence was not supplied.', + limitations: + input.trex_policy === 'optional' ? ['Optional browser-flow evidence is absent.'] : [], + }; + } + let receipt; + try { + receipt = await readJsonArtifact(root, path, CONTRIBUTION_LIMITS.receiptBytes); + } catch (error) { + return { + status: 'no_confidence', + policy: input.trex_policy, + reason: 'T-Rex receipt could not be read as bounded JSON.', + limitations: [error.message], + }; + } + const limitations = Array.isArray(receipt.limitations) + ? receipt.limitations.filter((entry) => typeof entry === 'string').slice(0, 50) + : []; + if ( + receipt.schema_version !== 1 || + typeof receipt.run_id !== 'string' || + !/^[0-9a-f]{40,64}$/i.test(receipt.source?.head_sha ?? '') || + !['passed_with_limits', 'failed', 'no_confidence'].includes(receipt.verdict) || + !['verified', 'claimed', 'mismatch'].includes(receipt.preview?.status) + ) { + return { + status: 'no_confidence', + policy: input.trex_policy, + reason: 'T-Rex receipt schema or required evidence is unsupported.', + limitations, + }; + } + if (receipt.source.head_sha !== candidateRevision || receipt.preview.status === 'mismatch') { + return { + status: 'stale', + policy: input.trex_policy, + reason: 'T-Rex source or preview identity does not match the challenged candidate.', + limitations, + }; + } + return { + status: receipt.verdict, + policy: input.trex_policy, + reason: receipt.summary || `T-Rex reported ${receipt.verdict}.`, + limitations, + }; +} + +function validateTrexPolicy(input, trexPath) { + if (input.trex_policy === 'not_applicable') { + if (trexPath) throw new Error('trex_receipt is incompatible with not_applicable policy'); + if (!input.trex_not_applicable_reason) { + throw new Error('trex_not_applicable_reason is required for not_applicable policy'); + } + } else if (input.trex_not_applicable_reason !== undefined) { + throw new Error('trex_not_applicable_reason requires not_applicable policy'); + } +} + +export function normalizeGitHubEvidence(value) { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + throw new Error('GitHub inspection returned invalid evidence'); + } + const identity = { + url: canonicalPullRequestUrl(value.url), + repository: value.repository, + number: value.number, + head_sha: value.head_sha, + base_sha: value.base_sha, + state: String(value.state).toLowerCase(), + is_draft: Boolean(value.is_draft), + mergeable: String(value.mergeable ?? 'unknown').toLowerCase(), + }; + const checks = normalizeChecks(value.checks ?? []); + const threads = (value.threads ?? []).slice(0, CONTRIBUTION_LIMITS.reviewThreads); + const current = threads.filter((thread) => !thread.resolved && !thread.outdated); + const outdated = threads.filter((thread) => thread.outdated); + const resolved = threads.filter((thread) => thread.resolved); + const observations = [ + ...(value.reviews ?? []).slice(0, CONTRIBUTION_LIMITS.reviewThreads).map((review) => ({ + kind: 'review', + author: bounded(review.author, 100), + state: bounded(review.state, 40), + summary: bounded(review.body || '', 300), + })), + ...threads.map((thread) => ({ + kind: 'thread', + author: bounded(thread.author, 100), + path: bounded(thread.path, 300), + line: Number.isInteger(thread.line) ? thread.line : null, + summary: bounded(thread.body, 300), + resolved: Boolean(thread.resolved), + outdated: Boolean(thread.outdated), + })), + ]; + const reviews = { + status: current.length > 0 ? 'action_required' : 'clear', + reason: + current.length > 0 + ? `${current.length} current unresolved maintainer thread(s) require contributor action.` + : 'No current unresolved review thread requires action.', + current_threads: current.length, + outdated_threads: outdated.length, + resolved_threads: resolved.length, + observations, + }; + const reviewStates = (value.reviews ?? []).map((review) => String(review.state).toUpperCase()); + const approvals = reviewStates.includes('CHANGES_REQUESTED') + ? { + status: 'changes_requested', + reason: 'A submitted review requests changes.', + observations: observations.filter((observation) => observation.kind === 'review'), + } + : reviewStates.includes('APPROVED') + ? { + status: 'approved', + reason: 'At least one approval is observed.', + observations: observations.filter((observation) => observation.kind === 'review'), + } + : { + status: 'not_observed', + reason: 'No approval or change-request review is observed.', + observations: observations.filter((observation) => observation.kind === 'review'), + }; + const permission = String(value.viewer_permission ?? '').toUpperCase(); + const mergeAuthority = ['ADMIN', 'MAINTAIN', 'WRITE'].includes(permission) + ? { status: 'contributor', reason: `GitHub viewer permission is ${permission}.` } + : ['READ', 'TRIAGE'].includes(permission) + ? { status: 'external_maintainer', reason: `GitHub viewer permission is ${permission}.` } + : { status: 'unknown', reason: 'GitHub merge authority could not be established.' }; + return { identity, checks, reviews, approvals, mergeAuthority }; +} + +function normalizeChecks(values) { + const observations = values.slice(0, CONTRIBUTION_LIMITS.checkRuns).map((check) => ({ + name: bounded(check.name, 160), + status: bounded(check.status, 40), + conclusion: bounded(check.conclusion, 40), + details_url: bounded(check.details_url, 300), + })); + const conclusions = observations.map((check) => check.conclusion.toLowerCase()); + const statuses = observations.map((check) => check.status.toLowerCase()); + const approvalRequired = observations.some( + (check) => + ['action_required', 'approval_required'].includes(check.conclusion.toLowerCase()) || + /(?:authorize|approval_required|actions\/runs\/[^/]+\/approve)/i.test(check.details_url) + ); + const failed = observations.some( + (check) => + ['failure', 'failed', 'error', 'cancelled', 'timed_out'].includes( + check.conclusion.toLowerCase() + ) && !/(?:authorize|approval_required|actions\/runs\/[^/]+\/approve)/i.test(check.details_url) + ); + let status = 'passed'; + let reason = 'All observed checks passed.'; + if (observations.length === 0) { + status = 'not_observed'; + reason = 'No current-head check was observed.'; + } else if (failed) { + status = 'failed'; + reason = 'At least one current-head check failed.'; + } else if (approvalRequired) { + status = 'approval_required'; + reason = 'A current-head workflow requires repository approval.'; + } else if ( + statuses.some((value) => !['completed', 'success'].includes(value)) || + conclusions.some((value) => ['', 'pending', 'queued', 'in_progress'].includes(value)) + ) { + status = 'pending'; + reason = 'At least one current-head check is pending.'; + } + return { status, reason, observations }; +} + +export async function inspectGitHubPullRequest(url) { + const parsed = parsePullRequestUrl(url); + const query = `query($owner:String!,$name:String!,$number:Int!){repository(owner:$owner,name:$name){viewerPermission pullRequest(number:$number){url number state isDraft mergeable headRefOid baseRefOid reviews(first:100){nodes{author{login}state body}} reviewThreads(first:100){nodes{isResolved isOutdated comments(first:20){nodes{author{login}body path line}}}} commits(last:1){nodes{commit{statusCheckRollup{contexts(first:100){nodes{__typename ... on CheckRun{name status conclusion detailsUrl} ... on StatusContext{context state targetUrl}}}}}}}}}}`; + const output = await runCommand('gh', [ + 'api', + 'graphql', + '-f', + `query=${query}`, + '-F', + `owner=${parsed.owner}`, + '-F', + `name=${parsed.name}`, + '-F', + `number=${parsed.number}`, + ]); + let payload; + try { + payload = JSON.parse(output); + } catch { + throw new Error('GitHub inspection did not return JSON'); + } + const repository = payload?.data?.repository; + const pr = repository?.pullRequest; + if (!pr) throw new Error('GitHub pull request was not found'); + const contexts = pr.commits?.nodes?.[0]?.commit?.statusCheckRollup?.contexts?.nodes ?? []; + return { + url: pr.url, + repository: `${parsed.owner}/${parsed.name}`, + number: pr.number, + head_sha: pr.headRefOid, + base_sha: pr.baseRefOid, + state: pr.state, + is_draft: pr.isDraft, + mergeable: pr.mergeable, + viewer_permission: repository.viewerPermission, + checks: contexts.map((context) => + context.__typename === 'StatusContext' + ? { + name: context.context, + status: context.state === 'PENDING' ? 'pending' : 'completed', + conclusion: context.state.toLowerCase(), + details_url: context.targetUrl ?? '', + } + : { + name: context.name, + status: context.status, + conclusion: context.conclusion ?? '', + details_url: context.detailsUrl ?? '', + } + ), + reviews: (pr.reviews?.nodes ?? []).map((review) => ({ + author: review.author?.login ?? 'unknown', + state: review.state, + body: review.body ?? '', + })), + threads: (pr.reviewThreads?.nodes ?? []).map((thread) => { + const comment = thread.comments?.nodes?.at(-1) ?? {}; + return { + resolved: thread.isResolved, + outdated: thread.isOutdated, + author: comment.author?.login ?? 'unknown', + path: comment.path ?? '', + line: comment.line ?? null, + body: comment.body ?? '', + }; + }), + }; +} + +async function readCandidateDiff(root, manifest, repository) { + let source = await runCommand('git', [ + '-C', + root, + 'diff', + '--no-ext-diff', + '--unified=0', + manifest.repository_revision, + '--', + ...repository.changed_files, + ]); + const untracked = new Set( + (await runCommand('git', ['-C', root, 'ls-files', '--others', '--exclude-standard', '-z'])) + .split('\0') + .filter(Boolean) + ); + for (const path of repository.changed_files.filter((candidate) => untracked.has(candidate))) { + const absolute = resolve(root, path); + assertContained(root, absolute, 'untracked candidate source'); + const details = await stat(absolute); + if (!details.isFile()) throw new Error(`untracked candidate is not a regular file: ${path}`); + const content = await readFile(absolute, 'utf8'); + source += `\n+++ b/${path}\n${content + .split(/\r?\n/) + .map((line) => `+${line}`) + .join('\n')}\n`; + if (Buffer.byteLength(source) > CONTRIBUTION_LIMITS.receiptBytes) { + throw new Error('candidate diff exceeds contribution evidence bounds'); + } + } + if (Buffer.byteLength(source) > CONTRIBUTION_LIMITS.receiptBytes) { + throw new Error('candidate diff exceeds contribution evidence bounds'); + } + return source; +} + +async function inspectCandidateCommit(root, manifest) { + const outputs = await Promise.all([ + runCommand('git', ['-C', root, 'diff', '--name-only', '-z', 'HEAD', '--']), + runCommand('git', ['-C', root, 'diff', '--cached', '--name-only', '-z', 'HEAD', '--']), + runCommand('git', ['-C', root, 'ls-files', '--others', '--exclude-standard', '-z']), + ]); + const changed = [ + ...new Set(outputs.flatMap((output) => output.split('\0').filter(Boolean))), + ].filter( + (path) => + path !== manifest.artifact_directory && !path.startsWith(`${manifest.artifact_directory}/`) + ); + return { clean: changed.length === 0, changed_files: changed.sort() }; +} + +function parsePullRequestUrl(value) { + const canonical = canonicalPullRequestUrl(value); + const match = /^https:\/\/github\.com\/([^/]+)\/([^/]+)\/pull\/(\d+)$/.exec(canonical); + return { owner: match[1], name: match[2], number: Number(match[3]) }; +} + +function canonicalPullRequestUrl(value) { + if (typeof value !== 'string' || value.length > CONTRIBUTION_LIMITS.pullRequestUrlCharacters) { + throw new Error('pull_request_url is invalid'); + } + const match = /^https:\/\/github\.com\/([^/?#]+)\/([^/?#]+)\/pull\/(\d+)\/?$/.exec(value); + if (!match) throw new Error('pull_request_url must be a canonical GitHub pull request URL'); + return `https://github.com/${match[1]}/${match[2]}/pull/${Number(match[3])}`; +} + +function closedInput(value, required, optional) { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + throw new Error('contribution input must be an object'); + } + const allowed = new Set([...required, ...optional]); + const unknown = Object.keys(value).filter((key) => !allowed.has(key)); + const missing = required.filter( + (key) => value[key] === undefined || value[key] === null || value[key] === '' + ); + if (unknown.length > 0) throw new Error(`unknown contribution field: ${unknown.join(', ')}`); + if (missing.length > 0) throw new Error(`missing contribution field: ${missing.join(', ')}`); +} + +function safeCampaignDirectory(value) { + const path = safeContainedPath(value, 'campaign_directory'); + if (!path.startsWith('.codevetter/optimization-campaigns/')) { + throw new Error('campaign_directory must be under .codevetter/optimization-campaigns/'); + } + return path; +} + +function safeContainedPath(value, label) { + if ( + typeof value !== 'string' || + isAbsolute(value) || + value.includes('\\') || + value.split('/').some((part) => ['', '.', '..'].includes(part)) + ) { + throw new Error(`${label} must be a contained repository-relative POSIX path`); + } + return value; +} + +async function readJsonArtifact(root, path, maximumBytes, containmentRoot = root) { + const absolute = resolve(root, path); + assertContained(root, absolute, path); + const canonical = await realpath(absolute); + assertContained(containmentRoot, canonical, path); + const details = await stat(canonical); + if (!details.isFile() || details.size > maximumBytes) + throw new Error(`${path} is unavailable or oversized`); + try { + return JSON.parse(await readFile(canonical, 'utf8')); + } catch { + throw new Error(`${path} is not valid JSON`); + } +} + +async function writeContainedArtifact(root, path, source, containmentRoot = root) { + const absolute = resolve(root, path); + assertContained(root, absolute, path); + if (Buffer.byteLength(source) > CONTRIBUTION_LIMITS.receiptBytes) { + throw new Error('contribution artifact exceeds size limit'); + } + await mkdir(dirname(absolute), { recursive: true }); + const canonicalParent = await realpath(dirname(absolute)); + assertContained(containmentRoot, canonicalParent, path); + const target = resolve(canonicalParent, basename(absolute)); + try { + const existing = await readFile(target, 'utf8'); + if (existing === source) return; + throw new Error('contribution artifact already exists with different content'); + } catch (error) { + if (error.code !== 'ENOENT') throw error; + } + const temporary = `${target}.codevetter-${process.pid}.tmp`; + try { + await writeFile(temporary, source, { flag: 'wx' }); + await rename(temporary, target); + } finally { + await rm(temporary, { force: true }); + } +} + +async function replaceContainedArtifact(root, path, source, expected, containmentRoot = root) { + const absolute = resolve(root, path); + assertContained(root, absolute, path); + if (Buffer.byteLength(source) > CONTRIBUTION_LIMITS.receiptBytes) { + throw new Error('contribution artifact exceeds size limit'); + } + await mkdir(dirname(absolute), { recursive: true }); + const canonicalParent = await realpath(dirname(absolute)); + assertContained(containmentRoot, canonicalParent, path); + const target = resolve(canonicalParent, basename(absolute)); + let current = ''; + try { + current = await readFile(target, 'utf8'); + } catch (error) { + if (error.code !== 'ENOENT') throw error; + } + if (current !== expected) throw new Error('contribution publication changed during refresh'); + const temporary = `${target}.codevetter-${process.pid}.tmp`; + try { + await writeFile(temporary, source, { flag: 'wx' }); + await rename(temporary, target); + } finally { + await rm(temporary, { force: true }); + } +} + +async function readReceiptLedger(root, path, containmentRoot = root) { + const absolute = resolve(root, path); + assertContained(root, absolute, path); + let source = ''; + try { + const canonical = await realpath(absolute); + assertContained(containmentRoot, canonical, path); + const details = await stat(canonical); + if (!details.isFile() || details.size > CONTRIBUTION_LIMITS.receiptBytes) { + throw new Error('contribution receipt ledger is unavailable or oversized'); + } + source = await readFile(canonical, 'utf8'); + } catch (error) { + if (error.code === 'ENOENT') return []; + throw error; + } + const receipts = source + .split(/\r?\n/) + .filter(Boolean) + .map((line) => JSON.parse(line)); + let previous = null; + for (const receipt of receipts) { + assertContributionReceipt(receipt); + if (receipt.previous_receipt_digest !== previous) + throw new Error('contribution receipt chain is broken'); + previous = receipt.receipt_digest; + } + return receipts; +} + +async function appendReceipt(root, path, previous, receipt, containmentRoot = root) { + const absolute = resolve(root, path); + assertContained(root, absolute, path); + await mkdir(dirname(absolute), { recursive: true }); + const canonicalParent = await realpath(dirname(absolute)); + assertContained(containmentRoot, canonicalParent, path); + const target = resolve(canonicalParent, basename(absolute)); + const existing = previous.map((entry) => `${stableStringify(entry)}\n`).join(''); + const next = `${existing}${stableStringify(receipt)}\n`; + if (Buffer.byteLength(next) > CONTRIBUTION_LIMITS.receiptBytes) { + throw new Error('contribution receipt ledger size limit exceeded'); + } + let current = ''; + try { + current = await readFile(target, 'utf8'); + } catch (error) { + if (error.code !== 'ENOENT') throw error; + } + if (current !== existing) throw new Error('contribution receipt ledger changed during refresh'); + const temporary = `${target}.codevetter-${process.pid}.tmp`; + try { + await writeFile(temporary, next, { flag: 'wx' }); + await rename(temporary, target); + } finally { + await rm(temporary, { force: true }); + } +} + +function assertContained(root, path, label) { + const rel = relative(root, path); + if (rel === '' || rel === '..' || rel.startsWith(`..${sep}`) || isAbsolute(rel)) { + throw new Error(`${label} escapes the repository`); + } +} + +function bounded(value, maximum) { + return typeof value === 'string' ? value.slice(0, maximum) : ''; +} + +function runCommand(command, args) { + return new Promise((resolvePromise, reject) => { + const child = spawn(command, args, { stdio: ['ignore', 'pipe', 'pipe'] }); + let stdout = ''; + let stderr = ''; + child.stdout.on('data', (chunk) => { + if (stdout.length <= CONTRIBUTION_LIMITS.receiptBytes) stdout += chunk; + }); + child.stderr.on('data', (chunk) => { + if (stderr.length <= 4_000) stderr += chunk; + }); + child.on('error', reject); + child.on('close', (code) => { + if (code !== 0) reject(new Error(`${command} inspection failed: ${stderr.trim()}`)); + else if (Buffer.byteLength(stdout) > CONTRIBUTION_LIMITS.receiptBytes) { + reject(new Error(`${command} inspection output exceeded the evidence bound`)); + } else resolvePromise(stdout); + }); + }); +} diff --git a/scripts/runtime-failure-capsule/contribution.test.mjs b/scripts/runtime-failure-capsule/contribution.test.mjs new file mode 100644 index 00000000..64018c81 --- /dev/null +++ b/scripts/runtime-failure-capsule/contribution.test.mjs @@ -0,0 +1,588 @@ +import assert from 'node:assert/strict'; +import { mkdtemp, mkdir, readFile, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import test from 'node:test'; + +import { + assertCandidateChallenge, + assertContributionReceipt, + deriveContributionStatus, +} from './contribution-contracts.mjs'; +import { createOptimizationContributionService, normalizeGitHubEvidence } from './contribution.mjs'; + +const BASE = 'a'.repeat(40); +const HEAD = 'b'.repeat(40); +const OLD_HEAD = 'c'.repeat(40); +const DIFF = 'd'.repeat(64); +const RECORD = 'e'.repeat(64); +const WORKLOAD = 'f'.repeat(64); + +test('candidate challenge records diff risk and requires an explicit bounded reason', async (context) => { + const fixture = await contributionFixture(context); + const service = await createOptimizationContributionService(fixture.root, fixture.dependencies); + + const unqualified = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + }); + assert.equal(unqualified.challenge.patch_quality.status, 'no_confidence'); + assert.deepEqual( + unqualified.challenge.diff_observations.risk_signals.map((signal) => signal.kind), + ['mutable_state', 'cleanup_path', 'fallback_path', 'new_branch'] + ); + assertCandidateChallenge(unqualified.challenge); + + fixture.dependencies.now = () => new Date('2026-08-10T00:00:01.000Z'); + const qualified = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + simpler_not_applicable_reason: + 'The direct own-property lookup adds no cache; the branch preserves the empty-table fast path.', + }); + assert.equal(qualified.challenge.patch_quality.status, 'retained_with_justification'); + assert.match(qualified.path, /challenge-b{12}-[0-9a-f]{12}\.json$/); +}); + +test('Marked-shaped challenge selects the simpler qualified candidate within tolerance', async (context) => { + const fixture = await contributionFixture(context, { + includeComparison: true, + selectedSimpler: true, + }); + const service = await createOptimizationContributionService(fixture.root, fixture.dependencies); + const result = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + comparison_sequence: 2, + }); + assert.equal(result.challenge.patch_quality.status, 'simpler_candidate_selected'); + assert.equal(result.challenge.candidate.complexity.added_lines, 4); + assert.equal(result.challenge.comparison.complexity.added_lines, 12); + assert.equal(result.challenge.candidate.control_metrics.length, 1); +}); + +test('candidate challenge observes class state, public signatures, and dependency manifests', async (context) => { + const fixture = await contributionFixture(context); + fixture.dependencies.inspectRepositoryState = async () => ({ + revision: HEAD, + diff_digest: DIFF, + changed_files: ['src/Lexer.ts', 'package.json'], + }); + fixture.dependencies.readCandidateDiff = async () => + '+ private links = new Map();\n+ export function parse() {}\n'; + const service = await createOptimizationContributionService(fixture.root, fixture.dependencies); + const result = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + simpler_not_applicable_reason: + 'The changed public surface and dependency manifest were reviewed.', + }); + assert.deepEqual( + result.challenge.diff_observations.risk_signals.map((signal) => signal.kind), + ['class_member_state', 'public_signature', 'dependency_manifest'] + ); +}); + +test('candidate challenge rejects a simpler target candidate when its control exceeds tolerance', async (context) => { + const fixture = await contributionFixture(context, { + includeComparison: true, + selectedSimpler: true, + }); + fixture.evidence.get(1).verification.observed[0].points[0].current = 1.1; + const service = await createOptimizationContributionService(fixture.root, fixture.dependencies); + const result = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + comparison_sequence: 2, + }); + assert.equal(result.challenge.patch_quality.status, 'no_confidence'); + assert.match(result.challenge.patch_quality.reason, /tolerance/); +}); + +test('qualified comparison rejects a more complex candidate inside the same workload', async (context) => { + const fixture = await contributionFixture(context, { includeComparison: true }); + const service = await createOptimizationContributionService(fixture.root, fixture.dependencies); + const result = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + comparison_sequence: 2, + }); + assert.equal(result.challenge.patch_quality.status, 'no_confidence'); + assert.match(result.challenge.patch_quality.reason, /simpler/); +}); + +test('candidate challenge refuses a working-tree-only candidate that has no exact PR revision', async (context) => { + const fixture = await contributionFixture(context); + fixture.dependencies.inspectCandidateCommit = async () => ({ + clean: false, + changed_files: ['src/Lexer.ts'], + }); + const service = await createOptimizationContributionService(fixture.root, fixture.dependencies); + await assert.rejects( + service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + simpler_not_applicable_reason: 'No simpler candidate is applicable.', + }), + /must be committed before challenge/ + ); +}); + +test('contribution receipt keeps review, approval, T-Rex, and head gates independent', async (context) => { + const fixture = await contributionFixture(context); + let github = githubEvidence({ + checks: [{ name: 'test', status: 'completed', conclusion: 'action_required' }], + threads: [ + { + resolved: false, + outdated: false, + author: 'maintainer', + path: 'src/Lexer.ts', + line: 42, + body: 'Use existing state.', + }, + { + resolved: false, + outdated: true, + author: 'maintainer', + path: 'src/Lexer.ts', + body: 'Old comment.', + }, + { + resolved: true, + outdated: false, + author: 'maintainer', + path: 'src/Lexer.ts', + body: 'Resolved comment.', + }, + ], + }); + const dependencies = { ...fixture.dependencies, githubInspector: async () => github }; + const service = await createOptimizationContributionService(fixture.root, dependencies); + const challenged = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + simpler_not_applicable_reason: 'No additional state or fallback is introduced.', + }); + const trexPath = 'artifacts/trex-receipt.json'; + await mkdir(join(fixture.root, 'artifacts')); + await writeFile( + join(fixture.root, trexPath), + JSON.stringify({ + schema_version: 1, + run_id: 'trex-preview-fixture', + source: { head_sha: HEAD }, + preview: { status: 'verified' }, + verdict: 'passed_with_limits', + summary: 'Selected browser flow passed.', + limitations: ['Coverage remains bounded.'], + }) + ); + const first = await service.inspect({ + campaign_directory: fixture.campaignDirectory, + challenge_path: challenged.path, + pull_request_url: 'https://github.com/markedjs/marked/pull/4048', + trex_policy: 'required', + trex_receipt: trexPath, + }); + assertContributionReceipt(first.receipt); + assert.equal(first.receipt.gates.performance.status, 'confirmed'); + assert.equal(first.receipt.gates.trex.status, 'passed_with_limits'); + assert.equal(first.receipt.gates.reviews.status, 'action_required'); + assert.equal(first.receipt.gates.reviews.current_threads, 1); + assert.equal(first.receipt.gates.reviews.outdated_threads, 1); + assert.equal(first.receipt.gates.reviews.resolved_threads, 1); + assert.equal( + first.receipt.gates.reviews.observations.find( + (observation) => observation.kind === 'thread' && !observation.outdated + ).line, + 42 + ); + assert.equal(first.receipt.gates.approvals.status, 'not_observed'); + assert.equal(first.receipt.gates.checks.status, 'approval_required'); + assert.equal(first.receipt.status, 'review_action_required'); + assert.equal(first.publication.status, 'current'); + assert.equal(first.publication.source_receipt_digest, first.receipt.receipt_digest); + + github = { + ...githubEvidence(), + url: 'https://github.com/example/other/pull/1', + repository: 'example/other', + number: 1, + }; + await assert.rejects( + service.refresh({ + campaign_directory: fixture.campaignDirectory, + challenge_path: challenged.path, + pull_request_url: 'https://github.com/example/other/pull/1', + trex_policy: 'required', + trex_receipt: trexPath, + }), + /different pull request/ + ); + + github = githubEvidence({ + checks: [{ name: 'test', status: 'completed', conclusion: 'success' }], + threads: [ + { + resolved: false, + outdated: true, + author: 'maintainer', + path: 'src/Lexer.ts', + body: 'Old comment.', + }, + ], + }); + dependencies.now = () => new Date('2026-08-10T00:00:02.000Z'); + const refreshed = await service.refresh({ + campaign_directory: fixture.campaignDirectory, + challenge_path: challenged.path, + pull_request_url: 'https://github.com/markedjs/marked/pull/4048', + trex_policy: 'required', + trex_receipt: trexPath, + }); + assert.equal(refreshed.receipt.status, 'waiting_for_maintainer'); + assert.equal(refreshed.receipt.gates.reviews.status, 'clear'); + assert.equal(refreshed.receipt.previous_receipt_digest, first.receipt.receipt_digest); + const ledger = await readFile(join(fixture.root, refreshed.receipt_path), 'utf8'); + assert.equal(ledger.trim().split('\n').length, 2); +}); + +test('Marked-shaped lifecycle invalidates publication and learns from revised feedback', async (context) => { + const fixture = await contributionFixture(context, { includeComparison: true }); + fixture.records[1].repository.revision = OLD_HEAD; + fixture.records[2].repository.revision = HEAD; + fixture.records[2].hypothesis = 'Replace duplicated lexer state with direct membership.'; + let repositoryRevision = OLD_HEAD; + let diff = '+ private links = new Map();\n+ try { if (ready) work(); } finally { fallback(); }\n'; + let github = githubEvidence({ + head: OLD_HEAD, + threads: [ + { + resolved: false, + outdated: false, + author: 'maintainer', + path: 'src/Lexer.ts', + line: 42, + body: 'Use the existing links object directly.', + }, + ], + }); + const dependencies = { + ...fixture.dependencies, + inspectRepositoryState: async () => ({ + revision: repositoryRevision, + diff_digest: DIFF, + changed_files: ['src/Lexer.ts'], + }), + readCandidateDiff: async () => diff, + githubInspector: async () => github, + }; + const service = await createOptimizationContributionService(fixture.root, dependencies); + const complex = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + simpler_not_applicable_reason: + 'The first candidate preserves the old fallback during screening.', + }); + const reviewed = await service.inspect({ + campaign_directory: fixture.campaignDirectory, + challenge_path: complex.path, + pull_request_url: 'https://github.com/markedjs/marked/pull/4048', + trex_policy: 'not_applicable', + trex_not_applicable_reason: 'Parser library has no browser flow.', + }); + assert.equal(reviewed.receipt.status, 'review_action_required'); + assert.equal(reviewed.publication.status, 'current'); + + repositoryRevision = HEAD; + github = githubEvidence({ + head: HEAD, + checks: [{ name: 'test', status: 'completed', conclusion: 'action_required' }], + threads: [ + { + resolved: false, + outdated: true, + author: 'maintainer', + path: 'src/Lexer.ts', + line: null, + body: 'Use the existing links object directly.', + }, + ], + }); + dependencies.now = () => new Date('2026-08-10T00:00:02.000Z'); + const stale = await service.refresh({ + campaign_directory: fixture.campaignDirectory, + challenge_path: complex.path, + pull_request_url: 'https://github.com/markedjs/marked/pull/4048', + trex_policy: 'not_applicable', + trex_not_applicable_reason: 'Parser library has no browser flow.', + }); + assert.equal(stale.receipt.status, 'stale'); + assert.equal(stale.publication.status, 'stale'); + assert.equal(stale.publication.source_receipt_digest, reviewed.receipt.receipt_digest); + + diff = '+ return Object.hasOwn(tokens.links, label);\n'; + dependencies.now = () => new Date('2026-08-10T00:00:03.000Z'); + const simpler = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 2, + comparison_sequence: 1, + }); + const current = await service.inspect({ + campaign_directory: fixture.campaignDirectory, + challenge_path: simpler.path, + pull_request_url: 'https://github.com/markedjs/marked/pull/4048', + trex_policy: 'not_applicable', + trex_not_applicable_reason: 'Parser library has no browser flow.', + }); + assert.equal(current.receipt.status, 'checks_approval_required'); + assert.equal(current.receipt.feedback_learning.length, 1); + assert.deepEqual(current.receipt.feedback_learning[0].rejected_patterns, [ + 'class_member_state', + 'cleanup_path', + 'fallback_path', + 'new_branch', + ]); + assert.equal(current.receipt.feedback_learning[0].feedback[0].line, 42); + assert.match(current.receipt.feedback_learning[0].revised_hypothesis, /direct membership/); + assert.equal(current.publication.status, 'current'); + assert.equal(current.publication.source_receipt_digest, current.receipt.receipt_digest); + assert.equal(current.publication.feedback_learning.length, 1); +}); + +test('head drift and required missing T-Rex evidence fail closed', async (context) => { + const fixture = await contributionFixture(context); + const dependencies = { + ...fixture.dependencies, + githubInspector: async () => githubEvidence({ head: 'c'.repeat(40) }), + }; + const service = await createOptimizationContributionService(fixture.root, dependencies); + const challenged = await service.challenge({ + campaign_directory: fixture.campaignDirectory, + selected_sequence: 1, + simpler_not_applicable_reason: 'No simpler candidate is applicable.', + }); + const result = await service.inspect({ + campaign_directory: fixture.campaignDirectory, + challenge_path: challenged.path, + pull_request_url: 'https://github.com/markedjs/marked/pull/4048', + trex_policy: 'required', + }); + assert.equal(result.receipt.gates.freshness.status, 'stale'); + assert.equal(result.receipt.gates.trex.status, 'missing'); + assert.equal(result.receipt.status, 'stale'); +}); + +test('contribution inspection rejects symlink escape before GitHub access', async (context) => { + const fixture = await contributionFixture(context); + let githubCalls = 0; + const service = await createOptimizationContributionService(fixture.root, { + ...fixture.dependencies, + githubInspector: async () => { + githubCalls += 1; + return githubEvidence(); + }, + }); + const outside = join(fixture.root, 'outside.json'); + await writeFile(outside, '{}'); + const closeout = join(fixture.root, fixture.campaignDirectory, 'closeout'); + await mkdir(closeout); + await symlink(outside, join(closeout, 'escape.json')); + await assert.rejects( + service.inspect({ + campaign_directory: fixture.campaignDirectory, + challenge_path: `${fixture.campaignDirectory}/closeout/escape.json`, + pull_request_url: 'https://github.com/markedjs/marked/pull/4048', + trex_policy: 'optional', + }), + /escapes the repository/ + ); + assert.equal(githubCalls, 0); +}); + +test('GitHub normalization retains empty-body inline feedback and never weights gates', () => { + const normalized = normalizeGitHubEvidence( + githubEvidence({ + reviews: [{ author: 'maintainer', state: 'commented', body: '' }], + threads: [ + { resolved: false, outdated: false, author: 'maintainer', path: 'src/a.ts', body: 'Why?' }, + ], + }) + ); + assert.equal(normalized.reviews.status, 'action_required'); + assert.equal(normalized.reviews.observations[0].summary, ''); + assert.equal( + deriveContributionStatus( + { + freshness: { status: 'current' }, + correctness: { status: 'passed' }, + performance: { status: 'confirmed' }, + patch_quality: { status: 'retained_with_justification' }, + trex: { status: 'not_applicable' }, + reviews: { status: 'action_required' }, + checks: { status: 'passed' }, + merge_authority: { status: 'external_maintainer' }, + }, + normalized.identity + ), + 'review_action_required' + ); +}); + +test('GitHub normalization treats fork workflow authorization as approval-required, not failed', () => { + const normalized = normalizeGitHubEvidence( + githubEvidence({ + checks: [ + { + name: 'Vercel', + status: 'completed', + conclusion: 'failure', + details_url: 'https://vercel.com/git/authorize?team=MarkedJS', + }, + { name: 'security/snyk', status: 'completed', conclusion: 'success' }, + ], + }) + ); + assert.equal(normalized.checks.status, 'approval_required'); + assert.match(normalized.checks.reason, /approval/); +}); + +test('GitHub normalization keeps submitted change requests independent from inline threads', () => { + const normalized = normalizeGitHubEvidence( + githubEvidence({ + reviews: [{ author: 'maintainer', state: 'changes_requested', body: 'Please revise.' }], + threads: [], + }) + ); + assert.equal(normalized.reviews.status, 'clear'); + assert.equal(normalized.approvals.status, 'changes_requested'); +}); + +test('GitHub normalization does not call an empty check list passing', () => { + const normalized = normalizeGitHubEvidence(githubEvidence({ checks: [] })); + assert.equal(normalized.checks.status, 'not_observed'); +}); + +async function contributionFixture( + context, + { includeComparison = false, selectedSimpler = false } = {} +) { + const root = await mkdtemp(join(tmpdir(), 'codevetter-contribution-')); + context.after(() => rm(root, { recursive: true, force: true })); + const campaignDirectory = '.codevetter/optimization-campaigns/fixture'; + await mkdir(join(root, campaignDirectory), { recursive: true }); + const records = [ + initializedRecord(), + promotionRecord( + 1, + selectedSimpler ? { movement: 4, metric: 10.2 } : { movement: 12, metric: 10 } + ), + ]; + if (includeComparison) { + records.push( + promotionRecord( + 2, + selectedSimpler ? { movement: 12, metric: 10 } : { movement: 4, metric: 10.2 } + ) + ); + } + const evidence = new Map( + records + .filter((record) => record.kind === 'promotion') + .map((record) => [record.sequence, promotionEvidence(record.metric)]) + ); + const dependencies = { + now: () => new Date('2026-08-10T00:00:00.000Z'), + campaignService: { + inspect: async () => ({ + manifest: { + campaign_id: 'fixture', + repository_revision: BASE, + artifact_directory: campaignDirectory, + }, + records, + }), + evidence: async ({ record_sequence: sequence }) => ({ + record: records[sequence], + evidence: evidence.get(sequence), + }), + }, + inspectRepositoryState: async () => ({ + revision: HEAD, + diff_digest: DIFF, + changed_files: ['src/Lexer.ts'], + }), + inspectCandidateCommit: async () => ({ clean: true, changed_files: [] }), + readCandidateDiff: async () => + '+ const cache = new Map();\n+ try { if (ready) work(); } finally { fallback(); }\n', + githubInspector: async () => githubEvidence(), + }; + return { root, campaignDirectory, dependencies, records, evidence }; +} + +function initializedRecord() { + return { sequence: 0, kind: 'initialized', decision: { status: 'initialized' } }; +} + +function promotionRecord(sequence, { movement, metric }) { + return { + sequence, + kind: 'promotion', + record_digest: sequence === 1 ? RECORD : '9'.repeat(64), + repository: { + revision: HEAD, + diff_digest: DIFF, + changed_files: ['src/Lexer.ts'], + }, + complexity: { + files_changed: 1, + added_lines: movement, + deleted_lines: 1, + delta_added_lines: movement, + delta_deleted_lines: 1, + }, + correctness: [{ status: 'passed' }], + decision: { status: 'keep' }, + metric, + }; +} + +function promotionEvidence(metric) { + return { + verification: { + workload_identity: { digest: WORKLOAD }, + observed: [ + { + kind: 'scale_point_comparison', + points: [ + { input: 0, unit: 'ms/op', baseline: 1, current: metric / 10 }, + { input: 2_000, unit: 'ms/op', baseline: 100, current: metric }, + ], + }, + ], + }, + }; +} + +function githubEvidence({ + head = HEAD, + checks = [{ name: 'test', status: 'completed', conclusion: 'success' }], + reviews = [], + threads = [], +} = {}) { + return { + url: 'https://github.com/markedjs/marked/pull/4048', + repository: 'markedjs/marked', + number: 4048, + head_sha: head, + base_sha: BASE, + state: 'open', + is_draft: false, + mergeable: 'mergeable', + viewer_permission: 'read', + checks, + reviews, + threads, + }; +} diff --git a/scripts/runtime-failure-capsule/local-flow-runtime.test.mjs b/scripts/runtime-failure-capsule/local-flow-runtime.test.mjs index 869341c6..5773c6ca 100644 --- a/scripts/runtime-failure-capsule/local-flow-runtime.test.mjs +++ b/scripts/runtime-failure-capsule/local-flow-runtime.test.mjs @@ -191,6 +191,9 @@ test('runtime MCP exposes product capabilities and fails closed on unknown captu 'promote_optimization_candidate', 'inspect_optimization_campaign', 'get_optimization_campaign_status', + 'challenge_optimization_candidate', + 'inspect_optimization_contribution', + 'refresh_optimization_contribution', ] ); assert.equal(tools[0].annotations.readOnlyHint, true); @@ -201,7 +204,7 @@ test('runtime MCP exposes product capabilities and fails closed on unknown captu const handle = await createRuntimeMcpHandler(root); const listed = await handle({ jsonrpc: '2.0', id: 1, method: 'tools/list' }); - assert.equal(listed.result.tools.length, 13); + assert.equal(listed.result.tools.length, 16); const qualification = await handle({ jsonrpc: '2.0', id: 2, @@ -221,7 +224,17 @@ test('runtime MCP exposes product capabilities and fails closed on unknown captu const campaignService = { status: (input) => ({ campaign_id: 'fixture', input }), }; - const campaignHandle = await createRuntimeMcpHandler(root, { campaignService }); + let contributionInput = null; + const contributionService = { + challenge: (input) => { + contributionInput = input; + return { schema_version: 'optimization-candidate-challenge/v1' }; + }, + }; + const campaignHandle = await createRuntimeMcpHandler(root, { + campaignService, + contributionService, + }); const campaignStatus = await campaignHandle({ jsonrpc: '2.0', id: 3, @@ -247,6 +260,37 @@ test('runtime MCP exposes product capabilities and fails closed on unknown captu assert.equal(escaped.result.isError, true); assert.match(escaped.result.structuredContent.error.message, /Unknown tool argument/); + const challenged = await campaignHandle({ + jsonrpc: '2.0', + id: 7, + method: 'tools/call', + params: { + name: 'challenge_optimization_candidate', + arguments: { + campaign_directory: '.codevetter/optimization-campaigns/fixture', + selected_sequence: 4, + simpler_not_applicable_reason: 'No simpler candidate is applicable.', + }, + }, + }); + assert.equal(challenged.result.isError, false); + assert.equal(contributionInput.selected_sequence, 4); + const mutationAttempt = await campaignHandle({ + jsonrpc: '2.0', + id: 8, + method: 'tools/call', + params: { + name: 'challenge_optimization_candidate', + arguments: { + campaign_directory: '.codevetter/optimization-campaigns/fixture', + selected_sequence: 4, + command: 'gh pr comment', + }, + }, + }); + assert.equal(mutationAttempt.result.isError, true); + assert.match(mutationAttempt.result.structuredContent.error.message, /Unknown tool argument/); + let plannerInput = null; const plannerHandle = await createRuntimeMcpHandler(root, { campaignService, @@ -287,7 +331,7 @@ test('runtime MCP process speaks line-delimited JSON-RPC without network setup', { jsonrpc: '2.0', id: 2, method: 'tools/list', params: {} }, ]); assert.equal(responses[0].result.serverInfo.name, 'codevetter-local-runtime'); - assert.equal(responses[1].result.tools.length, 13); + assert.equal(responses[1].result.tools.length, 16); }); test('validates the required recursive flow contract', () => { diff --git a/scripts/runtime-failure-capsule/mcp.mjs b/scripts/runtime-failure-capsule/mcp.mjs index 6187a2c9..f4ffc4a5 100644 --- a/scripts/runtime-failure-capsule/mcp.mjs +++ b/scripts/runtime-failure-capsule/mcp.mjs @@ -3,6 +3,7 @@ import { createInterface } from 'node:readline'; import { fileURLToPath } from 'node:url'; import { createOptimizationCampaignService } from './campaign.mjs'; +import { createOptimizationContributionService } from './contribution.mjs'; import { createLocalFlowService } from './flow-service.mjs'; import { planFlowOptimizationCampaign } from './flow-campaign-planner.mjs'; import { qualifyRepository } from './qualification.mjs'; @@ -16,6 +17,9 @@ export async function createRuntimeMcpHandler(repositoryRoot, options = {}) { const flowService = options.flowService ?? (await createLocalFlowService(repositoryRoot)); const campaignService = options.campaignService ?? (await createOptimizationCampaignService(repositoryRoot)); + const contributionService = + options.contributionService ?? + (await createOptimizationContributionService(repositoryRoot, { campaignService })); const flowCampaignPlanner = options.flowCampaignPlanner ?? ((input) => planFlowOptimizationCampaign(input)); return async function handle(request) { @@ -37,6 +41,7 @@ export async function createRuntimeMcpHandler(repositoryRoot, options = {}) { const result = await callTool( flowService, campaignService, + contributionService, flowCampaignPlanner, repositoryRoot, options.incumbentRepositoryRoot, @@ -67,6 +72,7 @@ export async function createRuntimeMcpHandler(repositoryRoot, options = {}) { async function callTool( flowService, campaignService, + contributionService, flowCampaignPlanner, repositoryRoot, incumbentRepositoryRoot, @@ -142,20 +148,56 @@ async function callTool( closedArguments(args, ['campaign_directory']); return campaignService.status(args); } + if (name === 'challenge_optimization_candidate') { + closedArguments( + args, + ['campaign_directory'], + ['simpler_not_applicable_reason'], + ['selected_sequence'], + ['comparison_sequence'] + ); + return contributionService.challenge(args); + } + if (['inspect_optimization_contribution', 'refresh_optimization_contribution'].includes(name)) { + closedArguments( + args, + ['campaign_directory', 'challenge_path', 'pull_request_url', 'trex_policy'], + ['trex_receipt', 'trex_not_applicable_reason'] + ); + return contributionService[name.startsWith('inspect') ? 'inspect' : 'refresh'](args); + } throw new Error('Unknown local runtime tool'); } -function closedArguments(value, required, optional = []) { +function closedArguments( + value, + required, + optional = [], + requiredIntegers = [], + optionalIntegers = [] +) { if (!value || typeof value !== 'object' || Array.isArray(value)) { throw new Error('Tool arguments must be an object'); } - const allowed = new Set([...required, ...optional]); + const allowed = new Set([...required, ...optional, ...requiredIntegers, ...optionalIntegers]); const unknown = Object.keys(value).filter((key) => !allowed.has(key)); const missing = required.filter( (key) => typeof value[key] !== 'string' || value[key].trim() === '' ); + const missingIntegers = requiredIntegers.filter( + (key) => !Number.isInteger(value[key]) || value[key] < 0 + ); + const invalidOptionalIntegers = optionalIntegers.filter( + (key) => value[key] !== undefined && (!Number.isInteger(value[key]) || value[key] < 0) + ); if (unknown.length > 0) throw new Error(`Unknown tool argument: ${unknown.join(', ')}`); if (missing.length > 0) throw new Error(`Missing tool argument: ${missing.join(', ')}`); + if (missingIntegers.length > 0) { + throw new Error(`Missing integer tool argument: ${missingIntegers.join(', ')}`); + } + if (invalidOptionalIntegers.length > 0) { + throw new Error(`Invalid integer tool argument: ${invalidOptionalIntegers.join(', ')}`); + } } export function toolDefinitions() { @@ -316,15 +358,39 @@ export function toolDefinitions() { ['campaign_directory'], readAnnotations ), + { + name: 'challenge_optimization_candidate', + description: + 'Challenge one kept optimization against deterministic diff complexity before publication.', + annotations: executeAnnotations, + inputSchema: { + type: 'object', + additionalProperties: false, + required: ['campaign_directory', 'selected_sequence'], + properties: { + campaign_directory: campaignDirectorySchema(), + selected_sequence: { type: 'integer', minimum: 0, maximum: 300 }, + comparison_sequence: { type: 'integer', minimum: 0, maximum: 300 }, + simpler_not_applicable_reason: { type: 'string', minLength: 1, maxLength: 1000 }, + }, + }, + }, + contributionTool( + 'inspect_optimization_contribution', + 'Inspect one GitHub pull request read-only and emit a revision-bound contribution receipt.', + executeAnnotations + ), + contributionTool( + 'refresh_optimization_contribution', + 'Refresh existing read-only GitHub evidence once without polling or notifying maintainers.', + executeAnnotations + ), ]; } function campaignTool(name, description, required, annotations) { const properties = { - campaign_directory: { - type: 'string', - description: 'Repository-relative directory under .codevetter/optimization-campaigns/.', - }, + campaign_directory: campaignDirectorySchema(), }; if (required.includes('hypothesis')) { properties.hypothesis = { @@ -341,6 +407,38 @@ function campaignTool(name, description, required, annotations) { }; } +function campaignDirectorySchema() { + return { + type: 'string', + description: 'Repository-relative directory under .codevetter/optimization-campaigns/.', + }; +} + +function contributionTool(name, description, annotations) { + return { + name, + description, + annotations, + inputSchema: { + type: 'object', + additionalProperties: false, + required: ['campaign_directory', 'challenge_path', 'pull_request_url', 'trex_policy'], + properties: { + campaign_directory: campaignDirectorySchema(), + challenge_path: { type: 'string', maxLength: 500 }, + pull_request_url: { + type: 'string', + pattern: '^https://github\\.com/[^/]+/[^/]+/pull/[0-9]+/?$', + maxLength: 300, + }, + trex_policy: { type: 'string', enum: ['optional', 'required', 'not_applicable'] }, + trex_receipt: { type: 'string', maxLength: 500 }, + trex_not_applicable_reason: { type: 'string', minLength: 1, maxLength: 1000 }, + }, + }, + }; +} + function success(id, result) { return { jsonrpc: '2.0', id, result }; }