feat(mcp): close the container gap + HTTP/SSE transport + opt-in resources/prompts

CHANGELOG.md

@@ -16,6 +16,8 @@ On a release, the maintainer moves the accumulated `[Unreleased]` entries into a
 
 ## [Unreleased]
 
+- **feat(mcp): close the container gap + HTTP/SSE transport + resources/prompts (opt-in) + a STATE.md snapshot.** Four upgrades to the MCP subsystem so it works everywhere and reaches remote servers. **(1) Container gap closed** — the model's MCP path is now a **host-dispatched `mcp` tool** (registered in `loopTools()`, the execute registry, alongside read/write/git), backed by a process-wide `internal/mcp.Manager`. Because the call runs host-side (not inside the box), MCP now works on **every** sandbox tier — crucially the macOS **container** default, where the previous path (model runs `nilcore mcp-call` via the sandboxed `run` tool) failed because debian-slim has neither `nilcore` nor a server runtime. Recorded as a bounded I4 relaxation in ARCHITECTURE §Execution model: servers are operator-configured (never model-emitted), the model selects only server + tool/resource/prompt + JSON args (data, I7), output is fenced, calls are audited — the same host-side place `setupMCP` already spawned servers for discovery. **(2) HTTP/SSE transport** — the JSON-RPC client is refactored onto a `transport` seam (`internal/mcp/transport.go`): **stdio** (subprocess) *or* **Streamable HTTP** (remote `url`, JSON *or* SSE reply, `Mcp-Session-Id` echoed, operator `headers` resolved host-side per I3). Stdlib `net/http` only — no module (I6 holds; `go.mod` untouched). **(3) Persistent Manager** — one live, initialized connection per server, reused across calls (stdio spawned once, HTTP session kept), concurrency-safe, evict-and-reconnect-once on a dropped connection, and an `ErrToolFailed` sentinel so a tool-level failure is never silently retried (no repeated side effect). **(4) Resources + prompts** — opt-in via `NILCORE_MCP_RESOURCES=1`: when set, descriptors are also generated for resources/prompts and the `mcp` tool honors `{server,resource}` / `{server,prompt,args}`; off by default ⇒ tools-only, byte-identical. `nilcore mcp-call` is retained (operator CLI + namespace-sandbox shell path) and now also speaks HTTP. Added `STATE.md` (a dated architecture/state snapshot). An adversarial pre-merge review (4 lenses × verify) then hardened it — confirmed-real fixes: **retry-safety** (the Manager now retries ONLY a provably-undelivered send via the `errDeliveryFailed` sentinel — never a server-received call, so a `delete`/`payment` tool can't run twice on a JSON-RPC error, HTTP non-2xx, or dropped reply); **I7 error-path fencing** (the `mcp` tool `guard.Wrap`s its untrusted error text, symmetric with the success path); **subprocess lifetime** (servers spawn under a Manager-owned context, not the per-turn request ctx, so a reconnected stdio server survives the turn); **single-flight `get()`** (connect+initialize run outside the lock behind a reservation — no global head-of-line stall, no double-spawn); **HTTP interop** (advertise protocol `2025-06-18` and send `MCP-Protocol-Version` on post-initialize requests); plus SSE multi-line `data:` join, an HTTP-notify status check, and a clearer SSE decode error. Tests: stdio + HTTP(JSON) + HTTP(SSE) client flows, Manager reuse/close over a real stdio subprocess and HTTP, concurrent single-flight, delivery-vs-received error classification, protocol-version header, error fencing, resource/prompt fetch + descriptor generation, HTTP config parsing — `-race` clean. `make verify` + `tui-verify` + full-repo `golangci-lint` (0 issues) green. _Owns:_ `internal/mcp`, `cmd/nilcore/{mcp,skills,chat,tui,main}.go`, `STATE.md`, `README.md`, `docs/{ARCHITECTURE,REFERENCE}.md`. _(MCP upgrade)_
+
 - **chore(cleanup): full-repo audit remediation — robustness, completeness, dead-code, doc-drift, tests.** A multi-agent audit (8 dimensions × adversarial verify) over the whole tree found **0 invariant breaks**; the confirmed findings were remediated. **Robustness/fail-closed:** graapprove kill-switch now fails CLOSED on any non-not-exist stat error; `wakeFeeder` delivers-then-disarms (+ an in-flight set) so a shutdown leaves the wake armed (at-least-once) instead of disarming-and-losing it; `SelfImproveGate` deny-defaults a nil human instead of panicking; Telegram+Slack ask-taps `claim` (atomic lookup+delete) closing the double-resolve TOCTOU; autosrc dequeue cancel-watcher Broadcasts under the lock (no lost-wakeup); `clipSelfacc` is rune-safe. **Completeness:** `final_verify`/`verify` events now carry a structural `fail_class` (the LRN-T01 producer half — the distiller/lessons pipeline read it but nothing emitted it); `NILCORE_AUTOAPPROVE_PRESET` (documented in ARCHITECTURE §0 but read nowhere) now seeds `cfg.AutoApprove` via `graapprove.Preset` (gives the presets a consumer). **Dead-code:** removed `vcache.HashWorktree` (exact duplicate of `verify.ContentHashWorktree`, no caller) and `graapprove.WithSentinel` + the write-never sentinel field (collapsed `killSwitchEngaged`). **Consistency:** deduped the byte-identical gate `describe()` (exported `policy.GateAction.Describe`, graapprove calls it); `DefaultBacklogPriority` is now `-1` (below `PriorityFile=0`) so every reactive source preempts idle backlog work. **Doc-drift:** reconciled stale references to this session's deletions (trust.Router, mcp.Gate, blackboard, NILCORE_TRUST) across code comments + docs. **Concurrency (the 2 HIGH bugs) — channel-intake demux:** both channel backends had `Receive` (serve loop) AND a drive's gate `Ask` reading the SAME socket/poll concurrently — a real data race over Slack's WebSocket (`-race`-reproduced) and Telegram's offset read-call-write across `getUpdates` (every update double-delivered). Fixed by a SINGLE intake goroutine that owns the socket/poll and DEMUXes each event — a gate answer to the waiting `Ask` (per-gate buffered channel, registered before the answer can arrive, authorizer-checked) and an ask-tap/message to the task queue `Receive` drains; `Receive`/`Ask` now only wait on channels, never the socket. Behavior preserved (unauthorized clicks ignored+logged, taps/messages during a gate still delivered). Concurrent Receive+Ask tests added for both backends, `-race` clean. **Tests:** added coverage for the recently-changed paths (the self-acceptance boundary_outcome trust hinge, the orchestrator hook's box/gate threading + OnSuccess/KeepBranch skip on redden, eventlog OnAppend panic-recovery, wakeFeeder at-most-once/not-due, GateActionType String/Class for the new types, Admit marker-evasion, the graapprove `countAutoApprovalsToday` rate counter) and raised the session gate-test wait budget to de-flake the merge-gating suite. The four design-decision seams were then resolved per-item: **project.DeriveAcceptance + SeedCriteria + PromotionPermitted — WIRED** (buildStack derives the advisor-proposed, sandbox-vetted, ADD-ONLY acceptance bar and seeds it so JudgeProject gates on it from iteration 0 — budget-headroom-gated so it makes no model call before the budget rail; and asserts PromotionPermitted on a greenfield-bootstrapped repo so a vacuously-green verifier is refused); **OracleEscalateAfter / native.EscalateAfterFn — DELETED** (the dynamic escalate-after seam was unwired end-to-end and a no-op — removed the interface method + impl + helper + native field + the dead RoutePlan.EscalateAfter, keeping the wired static EscalateAfter and RaceN/Plan); **Orchestrator.Gate / free-text policy.Gate and policy.EgressWith — KEPT + documented** as retained public primitives (the live paths use the typed GateStructured / the egress-profile builder) so a future audit does not re-flag them. `make verify` + `tui-verify` + full-repo `golangci-lint` (0 issues) + `-race` green. _Owns:_ `internal/{graapprove,autosrc,agent,backend,verify/vcache,policy,trust,mcp,channel/slack,channel/telegram,eventlog,session}`, `cmd/nilcore/{selfacc,autonomy,autoapprove,main,verifier}.go`, `docs/*`. _(repo cleanup)_
 
 - **chore(seams): resolve the five deferred-activation seams — clean, wire, plan, deprecate, fledge.** Acting on the prior pass's recorded backlog, each deferred seam was given a definite disposition. **(1) trust.Router — DELETED.** Production-dead (the live strength-routing path is `agent.TrustOracle` / `trust.Selector` via `NILCORE_TRUST_DEFAULT`; Router's single-construction-time-instance shape could never route per-worktree). Removed the package file + `trustRouterFor`/`routeContext`/`trustEnabled` + their tests; reconciled the example-comments in `oracle.go`/`orchestrator.go`/`selector.go`. **(4) mcp.Discover + mcp.Client.Gate — DEPRECATED/REMOVED.** Superseded by the codegen-descriptor path (the model invokes MCP tools through generated descriptors, never the in-process `Client`); both were set/called only in tests. Removed both + tests; `CallTool`'s doc now notes auth lives at the descriptor boundary. **(2) autosrc adapters — WIRED.** The autonomy daemon now drains the unified bounded priority queue from the file + wake funnels alongside the backlog: a `-autonomy-signals <dir>` feeder drives `FileSource`, and a wake-registry feeder drives `WakeSource` — the latter closing a real gap (serve armed durable wakes via the `sleep` tool but NEVER fired them; due wakes now re-engage through the verified queue, disarmed at-most-once). Adapters stay pure mappers; feeders own host I/O; default-off + byte-identical when empty. (CronSource + webhook-push remain available; cron needs a job set — the `schedule` verb's domain.) **(3) graapprove Deploy — PLANNED.** `docs/ROADMAP-DEPLOY.md`: the gate CLASS exists (Environments allowlist + $/day + prod deny) but nothing emits/executes a deploy; a grounded DEP-1..6 plan (executor leaf, target registry, post-deploy verify, mandatory rollback, the I4 egress relaxation) + per-invariant safety analysis. Plan only. **(5) verify/selfacc — FLEDGED into a runtime CLOSED LOOP.** The package always referenced an "operator-controlled wiring layer" it lacked; rather than ship the weak static-file v1 (which merely relocated operator work), selfacc was deepened into the loop that gives it novel value: during a run, once the project's verifier (the floor — I2) passes, the agent's OWN acceptance checks — authored for THIS goal by one bounded model call — must ALSO pass before the work is judged done. Each agent check is gated via a NEW typed `policy.GateAction{BindSelfAuthored}` (attended human / headless deny / `graapprove` auto-approve on EARNED trust — so review AMORTIZES: a proven self-check class auto-approves later within the operator envelope, answering "the gate just relocates work"); operator pre-approved checks (`NILCORE_SELFACC_FILE`) skip the gate. Bound checks run via the leaf's seam (`selfacc.Register`+`Resolve`) INSIDE the worktree box (I4); ALL must pass or the verdict reddens. It can only ADD to the bar — the hook is consulted ONLY when the floor is green, so it can never green a red floor (I2); every step is audited (`selfacc_propose/gate/bind/skip` + a `boundary_outcome` per gated check for trust — I5); nil/byte-identical when `NILCORE_SELFACC` is unset. Implemented as an opt-in `agent.SelfAccept` orchestrator hook (+ `agent.Env.Box`), `graapprove` `bind-self-authored` class (standard/trusted presets, a higher bar than open-pr), `cmd` `selfAcceptHook` capturing the run's model provider, and `nilcore selfacc propose|check` review surface; `docs/SELFACC.md` rewritten for the closed loop. All gates green (`make verify` + `tui-verify` + `golangci-lint` + `-race`). _Owns:_ removed `internal/trust/router.go` (+`_test`), `cmd/nilcore/{trust.go,trust_test.go,autonomy.go,autonomy_test.go,selfacc.go,selfacc_test.go,verifier.go,main.go}`, `internal/mcp/{client.go,codegen.go,mcp_test.go}`, `internal/agent/{oracle.go,orchestrator.go,orchestrator_selfaccept_test.go}`, `internal/policy/gateaction.go`, `internal/graapprove/{envelope,presets}.go`, `internal/trust/selector.go`, `docs/{ROADMAP-DEPLOY,SELFACC}.md`. _(Phase 16 — deferred-seam resolution)_

README.md

@@ -243,7 +243,7 @@ All opt-in — the default binary stays dependency-light and the loop is **byte-
 | Plug-in | Turn it on with | What you get |
 |---|---|---|
 | **Skills** | A `SKILL.md` (frontmatter + instructions) in `~/.config/nilcore/skills/` (or `$NILCORE_SKILLS_DIR`) | Surfaces to the loop as a `skill_<name>` tool; unused skills cost **~zero context**. |
-| **MCP servers** | `{name, command}` entries in `mcp.json` | `nilcore` generates typed wrappers under `mcp/servers/`; the executor discovers them on demand and invokes via `nilcore mcp-call`. |
+| **MCP servers** | `{name, command}` (stdio) or `{name, url, headers}` (remote HTTP/SSE) entries in `mcp.json` | `nilcore` generates typed wrappers under `mcp/servers/`; the executor discovers them on demand and invokes the host-dispatched `mcp` tool — so MCP works on **every** sandbox tier, including the macOS container default. Resources + prompts are opt-in (`NILCORE_MCP_RESOURCES=1`). |
 | **LSP retrieval** | `NILCORE_LSP_COMMAND=gopls` (or any language server) | Compiler-grade **"precise"** retrieval. |
 | **Live index** | `NILCORE_LIVE_INDEX=1` | A worktree-aware, incrementally-updated `live` code-intelligence tool. |