diff --git a/docs.json b/docs.json index 7307b8fdb..00960970b 100644 --- a/docs.json +++ b/docs.json @@ -65,7 +65,15 @@ "docs/kane-cli-variables-and-context", "docs/kane-cli-configuration", "docs/kane-cli-tms-integration", - "docs/kane-cli-testmd" + { + "group": "Test.md", + "pages": [ + "docs/kane-cli-testmd", + "docs/kane-cli-testmd-composition", + "docs/kane-cli-testmd-running" + ] + }, + "docs/kane-cli-testrun" ] }, { @@ -156,7 +164,13 @@ "pages": [ "docs/kane-cli-cli-reference", "docs/kane-cli-error-codes", - "docs/kane-cli-troubleshooting", + { + "group": "Troubleshooting", + "pages": [ + "docs/kane-cli-troubleshooting", + "docs/kane-cli-troubleshooting-sharp" + ] + }, "docs/kane-cli-changelog" ] } diff --git a/docs/kane-cli-assurance-automation.mdx b/docs/kane-cli-assurance-automation.mdx index 8b7fc5b13..65ae07002 100644 --- a/docs/kane-cli-assurance-automation.mdx +++ b/docs/kane-cli-assurance-automation.mdx @@ -35,6 +35,8 @@ Rule of thumb: `agent` when something can read the pause and answer (an AI agent The same matrix drives `maintain reconcile`, with two reconcile-specific rules: no headless mode ever archives anything — ARCHIVE decisions wait for an interactive session — and a `ci`-mode run that hits a decision needing a human **stores the plan and exits `2`** (the work isn't lost; walk the stored plan interactively or apply it in `agent` mode). +`context ingest` follows the matrix with one extra rule *(0.7.1)*: it lands the files and then runs the extraction under the given mode — except `--mode ci`, or piped stdin without any `--mode`, which **lands only** (exit `0`, with a stderr guidance line naming the next command). Two extraction dials also matter headless: `--trust hold` holds everything new for review instead of committing it (headless-only; `ci` refuses the flag entirely with exit `2`), and `--trust auto` is the default everywhere. + ## Exit codes Consistent across extract, design, and the maintain commands that embed them: @@ -43,43 +45,59 @@ Consistent across extract, design, and the maintain commands that embed them: |---|---| | `0` | Complete. | | `1` | Runtime failure. For extract and design, a `ci`-mode fail-close on a high-risk question also exits `1`; reconcile's `ci` fail-close stores the plan and exits `2` instead. | -| `2` | Usage / auth / refusal — bad flags, failed input validation, no store, bare non-TTY without `--mode`, missing `--yes` on a destructive command. Nothing was mutated. | -| `3` | **Paused and resumable** — the only meaning of 3. A session is saved; resume it within 24 hours. | +| `2` | Usage / auth / refusal — bad flags, failed input validation, no store, bare non-TTY without `--mode`, missing `--yes` on a destructive command. Nothing was mutated — with two durable exceptions: a merged ingest whose *extraction* refused keeps its landed sources (the run says they're safe), and reconcile's `ci` fail-close keeps its stored plan. | +| `3` | **Paused and resumable** — the only meaning of 3. A session is saved; resume it within 24 hours. Since 0.7.1 sessions are durable from the first turn, so a crash that left a checkpoint also exits `3` and names the exact resume command (a crash before anything durable — or a failed pause save — still exits `1`). | ## The NDJSON stream (`--mode agent`) With `--mode agent`, stdout speaks a versioned NDJSON vocabulary — envelope `{"type": "", "v": 1, "verb": "extract"|"design", ...}`, one object per line. The vocabulary is open: new event types may appear, so **tolerate unknown types**. +*(0.7.2)* The stream is **strict**: stdout carries only NDJSON — the first line is an event, `done` is the last — and stderr stays silent (crash traces excepted). No version banner, no receipts, no progress lines; everything user-relevant arrives as a typed event. On 0.7.1, prose diagnostics could ride stderr and a merged ingest printed receipt lines before the stream — a consumer that skips non-JSON prefix lines works on both releases. + | type | payload highlights | |---|---| -| `run_start` | `mode`, `trace` (the per-run log path); design adds `use_case` | +| `ingested` *(0.7.1)* | one per source landed by a merged `context ingest … --mode agent` run — `source_id`, `status` (`created`/`unchanged`/`versioned`), `cid`; arrives **before** the extraction's own events | +| `run_start` | `mode`, `trace` (the per-run log path); design adds `use_case`; *(0.7.2)* `session` — a support id, present when telemetry is on | | `corpus` | extract: the `sources[]` this run covers + already-extracted `skipped[]` | | `source_start` / `source_skipped` | `source_id`, `index`/`total`, `resumed` / `reason` | | `plan` | the `--plan` transcription payload | | `assumed_default` | a question auto-answered with its recommended default: `id`, `selected_index`, `risk` | | `agent_activity` | progress: `kind` (`tool` / `decision` / `progress` / `thinking_done`) + a display `label` | -| `usage` | per agent turn: `credits` + running `total_credits` | +| `agent_message` *(0.7.2)* | the agent's narrative `text` — the conversational lead-in before a question batch and the closing statement at the end of a run | +| `warning` *(0.7.2)* | an actionable non-fatal condition: `code` (`ZERO_USE_CASES`, `SAVE_FAILED`) + `message` | +| `lock_steal` *(0.7.2)* | a stale run lock was taken over: `key`, `stale_owner`, `by`, `ts` — observability only, no action needed | +| `usage` | per agent turn: `credits` + running `total_credits` (*(0.7.2)* rounded to two decimals) | | `validate_failed` | a proposal failed kane-side validation: `codes[]`, `repairing` (the agent self-repairs) | +| `degraded` *(0.7.1)* | duplicate detection fell back to a reduced mode this run (`reason`) — new items are held for review instead of auto-committed | +| `held` / `update_held` *(0.7.1)* | items were **held** for your review instead of committed (`source_id`, `count`, `reason` / `count`, `targets[]`) — the `--trust hold` and degraded-detection paths | | `commit` | what landed: counts + `minted[]` (`cid` + `logical_id`); extract adds `proposal_id` | | `receipt` | design: per-phase commit receipt — `commit_n`, `phase`, `committed[]`, `warnings[]`, `parity`, and a human-readable `next` hint | | `message_sent` | your `--message` was delivered: `sid`, `chars` | -| `session_paused` | `sid`, the verbatim `resume` command, `expires_at`, and **`pending_questions[]`** in full | +| `panel_resolved` *(0.7.1)* | a pending question was answered by a `--answer` flag: `id`, `by`, `via` | +| `ask_deferred` *(0.7.1)* | a pending question batch was set aside because `--with-source` landed a new source first: `source_id`, `cid`, `questions` (count) | +| `session_paused` | `sid`, the verbatim `resume` command, `expires_at`, and **`pending_questions[]`** in full. Two additional shapes *(0.7.1)*, distinguished by their fields: a crash-paused session carries `crashed: true` and **no** `pending_questions` (resume re-enters the conversation); a held-for-review pause carries only `sid`, `resume`, and `held` (a count) — no `expires_at` | | `session_complete` | `sid` | | `gate_refused` | a design gate refused the run (may be the first event) | -| `error` | `message` + a stable `code` where one exists (`NO_STORE`, `PREFLIGHT`, `SOURCE_MISSING`, `BLOB_MISSING`, `HIGH_RISK_CI`, `STALE_BASIS`) | -| `done` | **always the last event**: `status` (`complete`/`paused`/`error`/`refused`/`interrupted`/`aborted`) + `exit_code` | +| `phase_entry_override` *(0.7.1)* | a design `--phase` entry point was applied: `phase`, `missing[]` | +| `error` | `message` + a stable `code` where one exists (`NO_STORE`, `PREFLIGHT`, `SOURCE_MISSING`, `BLOB_MISSING`, `HIGH_RISK_CI`, `STALE_BASIS`, `EXTRACT_LOCKED`, `TRUST_USAGE`, `TRUST_UNDER_CI`, `HOLD_MULTI_SOURCE`, `UC_UNREVIEWED`, `UNKNOWN_PHASE`, `PHASE_ORDER`, `CITE_UNVERIFIED`, `WRONG_VERB`, `INGEST_UNAUTHORIZED_REF`, `STRUCTURED_FLAGS_USAGE` / `STRUCTURED_TARGET_UNKNOWN` (a misused `--answer`/structured verdict; the unknown-target refusal lists the addressable ids), `PAIR_MISMATCH` / `BINDING_MISMATCH` (see below)). *(0.7.2)* Three reconcile refusals carry a bracketed marker at the END of the message instead of a `code` — `[SOURCE_HELD]` (a head-move refused while a live review holds the source — finish the named session first), `[SESSIONS_UNREADABLE]` (the hold check could not run — unreadable session files fail closed; list/clean the sessions), and `[HELD_REVIEW]` (a headless `--apply` met a held review that needs a human); match the marker, not a `code` field. Many runtime failures are message-only | +| `done` | **always the last event**: `status` (`complete`/`paused`/`error`/`refused`/`interrupted`/`aborted`) + `exit_code`; may carry `next[]` | + +**The `done` guarantee:** every `--mode agent` invocation ends its stream with exactly one `done` event — including refusals and graceful interrupts. *(0.7.2)* The stream starts at the first line: a merged ingest's landing failures (a bad path, an unsupported or oversized file, a refused URL, a misused `--mode` or `--as`) also arrive as `error` + `done` — codes `MODE_USAGE`, `AS_SINGLE_SOURCE`, `UNSUPPORTED_URL`, `INGEST_FAILED` — and sources already landed stay safe, with the run saying so. (On 0.7.1 these landing failures ended with a prose error line and exit `1`/`2` *before any NDJSON began* — no stream, no `done`.) The one exception is operator force: a second Ctrl+C can hard-kill the process (exit `130`) without a `done`. Any other stream that ends without `done` should be treated as a crash. One more parsing note: the agent may also repair a draft mid-turn on its own — that surfaces only as `agent_activity` lines (labels like `validation failed`, `refining the draft`); treat activity labels as display text, never script against them. + +Two more parsing rules: -**The `done` guarantee:** every `--mode agent` invocation ends its stream with exactly one `done` event — including refusals and graceful interrupts. The one exception is operator force: a second Ctrl+C can hard-kill the process (exit `130`) without a `done`. Any other stream that ends without `done` should be treated as a crash. One more parsing note: the agent may also repair a draft mid-turn on its own — that surfaces only as `agent_activity` lines (labels like `validation failed`, `refining the draft`); treat activity labels as display text, never script against them. +- **Receipts and prefixes.** *(0.7.2)* Nothing precedes the stream — the landing receipts **are** the `ingested` events, and every stdout line parses as JSON. On 0.7.1, a merged ingest printed a few prose receipt lines per file before the NDJSON began, so strict per-line `JSON.parse` consumers had to skip non-JSON prefix lines. That skip is harmless on 0.7.2 — a version-tolerant consumer can keep it. +- **`next[]` carries follow-up commands** *(0.7.1)*. Pauses, gate refusals, and `done` can carry a `next` list of ready-to-run follow-ups. The common shape is objects (`{cmd, why, title}`); a few refusal sites emit plain strings — handle both, and treat every entry as a command to offer, not to auto-run. ### Reconcile's stream -`maintain reconcile --mode agent` speaks the same envelope with `verb: "reconcile"` and its own event set: +`maintain reconcile --mode agent` speaks the same envelope with `verb: "reconcile"` and its own event set. *(0.7.2)* The stream opens with a minimal `run_start` (it carries `session` only — no `trace`), and the re-extract child rides the **same stream**: its extract-vocabulary events (`source_start`, `agent_activity`, `plan`, `commit`, …) interleave between the `reconcile_*` events, all stamped `verb: "reconcile"` — one command, one stream. The engine itself is unchanged: ADD/MODIFY auto-apply, ARCHIVE pauses, `--apply` resumes. | type | payload highlights | |---|---| | `reconcile_plan` | the triage ahead: `source_id`, `plan_path`, `rows[]` (`kind`, `ref`, `why`), `archive[]` (proposed archivals with their evidence-decay reasons) | | `reconcile_row_start` | per row: `kind`, `ref`, plus the impact counts where they apply (`stale`, `direct`) | -| `reconcile_row_end` | the row's `outcome` (`applied` \| `failed` \| `skipped` \| `plan-only` \| `paused`) + `exit_code`, and an additive `detail` carrying a failure's reason and hint. A row's embedded design run is folded in here, so the stream stays single-writer with exactly one `done` | +| `reconcile_row_end` | the row's `outcome` (`applied` \| `failed` \| `skipped` \| `plan-only` \| `paused`) + `exit_code`, and an additive `detail` carrying a failure's reason and hint (e.g. a `HELD_CITES_STALE` refusal with its re-stage hint). A row's embedded design run is folded in here, so the stream stays single-writer with exactly one `done` | | `reconcile_paused` | `plan_path` + `pending[]` (`ref`, `why`) — resume with the same reconcile command (or `--apply`) | | `reconcile_summary` | the honest totals, always the same field set: `applied`, `skipped`, `deferred`, `plan_only`, `failed`, `paused`, `stale_created` | | `done` | always last — same guarantee as above | @@ -121,6 +139,16 @@ $ kane-cli context extract --resume ext-20260716T140742-prd-online-store --mode The agent maps your statement to its own pending questions. A statement that answers nothing pending is treated as steering ("also cover the coupon path"); if it leaves a high-risk ambiguity standing, the run pauses again with refreshed questions. +Two structured alternatives to `--message` *(0.7.1)*: + +- **Answer by id** — `--answer =