Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,15 @@
"docs/kane-cli-variables-and-context",
"docs/kane-cli-configuration",
"docs/kane-cli-tms-integration",
"docs/kane-cli-testmd"
{
"group": "Test.md",
"pages": [
"docs/kane-cli-testmd",
"docs/kane-cli-testmd-composition",
"docs/kane-cli-testmd-running"
]
},
"docs/kane-cli-testrun"
]
},
{
Expand Down Expand Up @@ -156,7 +164,13 @@
"pages": [
"docs/kane-cli-cli-reference",
"docs/kane-cli-error-codes",
"docs/kane-cli-troubleshooting",
{
"group": "Troubleshooting",
"pages": [
"docs/kane-cli-troubleshooting",
"docs/kane-cli-troubleshooting-sharp"
]
},
"docs/kane-cli-changelog"
]
}
Expand Down
67 changes: 55 additions & 12 deletions docs/kane-cli-assurance-automation.mdx

Large diffs are not rendered by default.

36 changes: 34 additions & 2 deletions docs/kane-cli-assurance-context.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,23 @@ kane-cli context list # see what you have
kane-cli context ingest <src...> [--as <id>]
```

Snapshots one or more files into `.context/` (the store is created on first use). Each source gets a stable id — by default the filename slug (`prd-online-store.md` → `prd-online-store`), or pass `--as <id>` to name it yourself.
Snapshots one or more files into `.context/` (the store is created on first use).

Snapshots one or more files into `.context/` (the store is created on first use) **and then extracts them** *(0.7.1)*:

- On a terminal, the run continues straight into the interactive extract chat.
- `--mode agent` extracts headless on the NDJSON stream (one `ingested` event per landing, before the extraction begins — see [Automation](/docs/kane-cli-assurance-automation/#the-ndjson-stream---mode-agent)); `--mode override` extracts headless too, auto-taking every default.
- `--mode ci` — or piped stdin without a `--mode` — **lands only**: the files snapshot, nothing extracts, exit `0`, with a guidance line on stderr telling you what to run next.
- `--plan`, `--force`, and `--trust` pass through to the extraction; they refuse under the land-only modes.

Receipts adapt to the surface: a terminal prints the human copy (`landed prd (prd.md) — new source`); `ci`/piped runs keep the byte-exact script contract below. *(0.7.2)* Under `--mode agent` nothing prints outside the stream — the landing receipt is the `ingested` event itself.

```
$ kane-cli context ingest ./prd-online-store.md --mode ci
created prd-online-store source sha256:0661… blob sha256:3db8…
```

Each source gets a stable id — by default the filename slug (`prd-online-store.md` → `prd-online-store`), or pass `--as <id>` to name it yourself.

Ingest is deterministic about identity:

Expand All @@ -44,6 +60,11 @@ $ kane-cli context ingest ./prd-online-store.md
created prd-online-store source sha256:0661… blob sha256:3db8…
```

Two lineage helpers:

- **`--as` records versions**: a colleague hands you `PRD-v2.md` of an existing source `prd` — `ingest PRD-v2.md --as prd` records it as a new *version* of `prd` (head moves, dependents go stale). `--as` names one identity, so it refuses more than one file.
- **Version suggestion (terminal only)**: ingesting `prd-v2.md` when a source `prd` already exists prompts "looks like a new version of `prd` — ingest as `prd`?". Accept and it versions `prd`; decline and `prd-v2` mints as its own source. Piped runs never prompt and never auto-link — pass `--as` explicitly in scripts.

Accepted media: text (`.txt`) and markdown (`.md`, `.markdown`) up to 2 MB — cited verbatim by line; PNG/JPEG/WebP images up to 5 MB — cited whole-image. Anything else is rejected with `UNSUPPORTED_MEDIA`; oversized files with `FILE_TOO_LARGE`.

When the new bytes are a **changed version of a source you already extracted from**, prefer [`kane-cli maintain reconcile`](/docs/kane-cli-assurance-maintain/) over a bare re-ingest — it records the same head move *and* triages what the change means for your suite, in one step.
Expand Down Expand Up @@ -126,6 +147,8 @@ kane-cli context extract --resume <sid> --message "Account required — the upda

```bash
kane-cli context review [--queue derived|skipped|archived|drift] [--verdicts <file>] [--json]
kane-cli context review --approve <refs...> | --skip <refs...> | --defer <refs...> # 0.7.1
kane-cli context review --verdicts <file> --allow-archive --because "<reason>" # 0.7.1
```

Walks existing nodes through the same review checklist, landing every verdict as one batched record:
Expand All @@ -145,6 +168,10 @@ In any queue: approve promotes, reject archives (a trusted node *can* be demoted

with `resolution` one of `approved | edited | rejected | skipped | supersede` (plus optional `reason`, `edit`, `supersede_target`). It is atomic: every ref must resolve and sit in a verdict queue, or nothing commits (exit `2`). With `--json`, each landed verdict echoes as one NDJSON row. There is deliberately no auto-approve mode for review — trust requires a human decision.

**Archives need explicit consent** *(0.7.1)*. A headless rejection no longer destroys anything: `--verdicts` holds rejected entries as non-destructive `pending_archive` facts (exit `0`, with a loud summary). Actually archiving them requires `--allow-archive` **and** `--because "<reason>"` — the reason goes on the record. Under `--mode ci`, archives are refused under any flag (exit `2`, atomic).

**Structured verdicts** *(0.7.1)* — the flag form for scripted single decisions: `--approve <refs...>` lands approvals; `--skip` and `--defer` record nothing and leave the items queued. The three are mutually exclusive with `--verdicts`.

<a id="inspect" />
## Inspecting the graph

Expand Down Expand Up @@ -198,7 +225,7 @@ kane-cli context name <ref> <slug> # name one node
kane-cli context name --backfill [--yes] # assign ids to every unnamed node
```

Assigns a stable kebab-case name. Names are never part of a node's identity — renaming never re-addresses — and names follow edits, so a name assigned to version 1 keeps resolving to the current version.
Assigns a stable kebab-case name. Names are never part of a node's identity — renaming never re-addresses — and names follow edits, so a name assigned to version 1 keeps resolving to the current version. The sequential-id namespace (`uc-3`, `ac-12`, …) is reserved for ids assigned at mint — `name` refuses it (exit `2`).

### `context revert`

Expand Down Expand Up @@ -234,6 +261,7 @@ Freshness is orthogonal: `fresh` / `stale` (the source snapshot moved) / `orphan
├── derived/ # regenerable read caches (delete any time; rebuild restores)
├── proposals/<ts>/ # proposal + review artifacts per extract run
├── sessions/<sid>/ # resumable paused sessions (expire after 24h)
├── locks/ # advisory run locks (transient)
├── logs/ # per-run trace files
├── design/ # design rationale sidecars + technique overrides
├── reconcile/plans/ # stored reconcile plans
Expand All @@ -242,6 +270,10 @@ Freshness is orthogonal: `fresh` / `stale` (the source snapshot moved) / `orphan

Two rules worth repeating from the [overview](/docs/kane-cli-assurance/#store): the store is **single-writer**, and it is **not git-mergeable** — gitignore it and share by re-ingesting sources.

### Tracing a run

Every extract and design run prints a `trace: <path>` line naming its log file — the first place to look when a run surprises you.

Every extract run also writes a per-run trace to `.context/logs/extract-<ts>.log` (the path is printed at the start of the run) — the first place to look when a run surprises you.

## For agents and CI
Expand Down
34 changes: 34 additions & 0 deletions docs/kane-cli-assurance-coverage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ completeness (live graph):
→ kane-cli design tests --use-case uc-buy-as-a-guest
```

The panel always shows **both** axes: the pack's proven depth, then the live-graph completeness worklist with its ready-to-paste commands (`--json` emits both as one document).

- The default pack is the newest in `<cwd>/.testmuai/evidence`; `--from` takes a pack directory, a sealed `.evidence` file, or an execution id.
- Depth is **risk-weighted and lenient**: a high-risk criterion weighs more, and a passed-but-stale criterion still counts as proven — staleness is surfaced (`· N stale`), never silently demoted. Per-use-case status is `covered` (every AC proved) · `blocked` (something couldn't run, nothing failed) · `partial` · `uncovered`.
- Coverage reflects **this run**: sealed packs cover only what the run touched. Project-wide coverage lives in the graph axis, unaffected by any single pack.
Expand All @@ -59,6 +61,38 @@ gaps — stage design (5)
- `--stage cover` (needs a pack) — a covered criterion whose **execution** disappointed: `failed` → re-design that slice; `blocked` or never-run → the test exists, run it.
- `--stage all` — both, one ranking. `--top <n>` trims the list.

## `cover gaps` — the coverage ribbon *(0.8.2)*

The default output is one continuous band-table over two axes, entirely graph-fed — no pack is opened or needed:

- **designed** — how much of the live requirement set has a live test verifying it (risk-weighted: high-risk criteria weigh more). Freshness never moves the number — stale criteria surface as `STALE` debt instead.
- **proven** — the store's **own execution facts** (every `testmd run` / `testrun run` records its verdicts at finalize), scored by the same formula a sealed pack uses — so the live number and a pack's number agree at the formula level. `--rollup lenient|strict` selects the formula (default `lenient`). With no recorded runs anywhere, the proven line says so in words — never a fake 0%.

The ribbon is deliberately high-level. A three-line header — the title with the rollup and how recently anything ran, then both axis bars, the proven line carrying the class census — then **one row per live use-case**, ordered by severity (failing → blocked → design debt → stale → pending-run → clean): id, title, designed and proven percent bars colored by value, and per-class debt counts (`FAILING · BLOCKED · STALE · TO DESIGN · TO RUN`). A count column renders only when some use-case carries that debt, and a narrow terminal drops the counts and keeps id, title, and bars. The last line is the drill-in hint. There are no AC rows, commands, or gap rosters at this level — detail lives in the dossier. With zero execution facts the all-clear reads `designed · nothing run yet`; the ribbon only ever says "proven" when every criterion's verifying test actually passed. Piped output keeps the same grammar; color and the width-responsive layout differ.

*(0.7.2)* A change you deferred during a [reconcile review](/docs/kane-cli-assurance-maintain/#reconcile) still surfaces here — in the ribbon's debt counts, and as a pending row in the dossier and the `--json` document with the reconcile command as its remedy. Deferring parks a decision, it never hides one.

### `cover gaps <uc-id>` — the dossier *(0.8.2)*

The dossier owns the detail for one use-case: its axis bars scoped to that use-case, then **every live acceptance criterion** in a full-text census table — `id · STATE · RISK · criterion`, wrapped, never truncated — proven and not-yet-run rows included. A failing row cites the test file when its id resolves to exactly one file (id-only otherwise), and a `next` actions block **leads with evidence, never a blind re-run**:

```
→ kane-cli evidence serve <pack> (see why it failed)
```

with the run/design remediation second, carrying its own warning — a failed test re-run in authoring mode may heal itself around the failure, so fix the app first.

### Flags

- `--stage design|cover|all` (default `all`) filters the `--json` document's pending rows — it never moves a number, a row, or the table order.
- `--top <n>` bounds pending rows per use-case in the `--json` document; the table always renders one row per use-case and is never truncated.
- `--rollup lenient|strict` selects the proven-axis formula (default `lenient`).
- `--json` emits the nested machine document: the designed axis, the proven axis (absent entirely when the store has no execution facts), and the per-use-case pending rows, each carrying its `ready_command`. With a `<uc-id>` the document **closes over that use-case** — one entry, project-wide leftovers emptied. Debt that belongs to no live use-case rides only the `--json` document, never the human views.

## Agents and CI *(0.7.1)*

Both `cover` and `cover gaps` take `--mode agent|ci`: the run speaks the same NDJSON envelope as the other assurance commands — the whole `--json` payload arrives as a single `coverage` (or `gaps`) event — with a `<uc-id>`, the closed-over document *(0.8.2)* — `done` is always last, and `done`'s `next[]` carries the worklist's own ready-to-paste commands. A refusal is an `error` event plus `done` with exit `2`. See [Automation](/docs/kane-cli-assurance-automation/).

## The join: how a pack knows your graph

Every per-test result in an evidence pack carries a `definition_id` — a hash of the resolved test definition, identical to the one design stamps on each test it emits. The pack↔graph join is this hash equality and nothing else: no ids to sync, no registry to maintain. A hand-edited test hashes differently and simply stops joining — honest, not broken (a redesign — [`design tests --force`](/docs/kane-cli-assurance-design/#re-runs) or [`maintain evolve`](/docs/kane-cli-assurance-maintain/#evolve) — re-stamps the link).
Expand Down
11 changes: 10 additions & 1 deletion docs/kane-cli-assurance-design.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,11 @@ kane-cli design explain t-add-first-item # replay WHY — zero f
| `--strength pairwise\|3-wise` | Manual covering-array strength override; absent = risk-judged (3-wise when money, auth, or data loss is involved) |
| `--mode <mode>` | Ask policy for headless runs: `agent` \| `ci` \| `override` — bare non-TTY exits `2`. See [Automation](/docs/kane-cli-assurance-automation/) |
| `--force` | Redesign a use-case that already has a live design (supersedes its scenario+test pairs; equivalent ACs are reused) |
| `--phase <name>` *(0.7.1)* | Enter the design at a specific phase (`grounding`, `acs`, `scenarios`, `wiring`, `tests`), re-seeded from the committed earlier phases. Missing predecessors prompt interactively; in agent mode the run exits `2` with the runnable commands in `next` |
| `--allow-unreviewed` *(0.7.1)* | Design against a use-case that is still unreviewed (`derived`) without approving it first — see the gate below |
| `--resume <sid>` | Resume a paused session ([sessions](/docs/kane-cli-assurance-context/#sessions)) |
| `--message "<text>"` | With `--resume`: answer the pending questions (or steer) in plain words |
| `--answer <q>=<v>` *(0.7.1)* | With `--resume --mode agent`: answer a specific pending question by id — `<question-id>=<option number or free text>`, repeatable |
| `--plan` | Transcription only — print each finalize payload, commit nothing |

## The session — five phases
Expand All @@ -46,6 +49,8 @@ Interactive runs are a chat. The engine works phase by phase and parks between p

Between phases you steer in plain words: `looks good` (approve) · rename or correct an item (edit) · `drop 3` (reject) · `show 2` / `hide` (drill in) — plus the local slash commands `/explain <ref>` (why an item exists — free, replayed from the record), `/done` (end the session), and `/pause` (save + exit `3`). The chat shell — the question panel, the composer grammar, ctrl+t, ctrl+c-to-pause — is exactly the one [extract uses](/docs/kane-cli-assurance-context/#the-interactive-chat).

**Questions that need a typed value.** When an answer is a concrete value (a URL, a fixture id), the question either offers an input-bearing option row — selecting it opens an inline editor so the answer carries the pick and your value together *(0.7.1)* — or tells you to type the value directly. A plain option row sends only its label, so plain options on such questions are the genuine alternatives (use a placeholder, reduce scope, skip). If a needed value doesn't arrive, the agent re-asks once and then proceeds on its stated fallback — announced in the narrative, never silently.

Headless modes run all phases without parking and emit one combined result; a high-risk question pauses an `agent`-mode run (resumable) and fails a `ci`-mode run closed. See [Automation](/docs/kane-cli-assurance-automation/).

## What you get
Expand Down Expand Up @@ -95,6 +100,8 @@ Until a test has been authored, `kane-cli testrun` preflight reports it as `miss
<a id="re-runs" />
## Re-runs and `--force`

**The unreviewed-target gate** *(0.7.1)*. Designing against a use-case that is still unreviewed (`derived`) stops for an explicit decision: interactively you get a disclosure and choose; in agent mode the run exits `2` with the runnable review command in `next`; in `ci` mode it refuses outright. `--allow-unreviewed` bypasses the gate deliberately — reviewing first ([`context review`](/docs/kane-cli-assurance-context/#review)) is the recommended path.

A use-case with a live design refuses a re-run, staleness-aware:

```
Expand All @@ -104,6 +111,8 @@ A use-case with a live design refuses a re-run, staleness-aware:

`--force` regenerates the scenario+test pairs (superseding the old ones); ACs are dedup-first — an equivalent AC re-emitted by the engine reuses the existing node instead of piling up copies. When the staleness comes from a source document you just changed, [`kane-cli maintain reconcile`](/docs/kane-cli-assurance-maintain/) surfaces the same re-design as part of its changed-source triage; for staleness from older changes, [`kane-cli maintain evolve`](/docs/kane-cli-assurance-maintain/#evolve) re-designs the use-case with the blast radius stated first.

**Citations are verified before they commit** *(0.7.1)*. Every citation a design run wants to record is checked against the pinned source text before anything lands; one that doesn't verify is sent back to the agent to repair — designed items never carry fabricated provenance.

## `design explain` — replay the why

```bash
Expand All @@ -127,7 +136,7 @@ The design engine ships with an embedded catalog of test-design techniques and s
|---|---|
| `0` | Design complete (or `--plan` transcription complete). |
| `1` | Runtime failure. |
| `2` | Usage / refusal — unknown use-case, already-designed without `--force`, bare non-TTY without `--mode`. |
| `2` | Usage / refusal — unknown use-case, already-designed without `--force`, an already-designed or unreviewed target refused in a headless mode (the stream's `next` carries the follow-up commands), a `--phase` whose predecessors are missing, bare non-TTY without `--mode`. |
| `3` | Session paused and resumable — see [sessions](/docs/kane-cli-assurance-context/#sessions). |

## Next steps
Expand Down
Loading