Skip to content

host-platforms.md

Latest commit

 

History

History
165 lines (136 loc) · 10.1 KB

host-platforms.md

File metadata and controls

165 lines (136 loc) · 10.1 KB
Raw

Host platform support

CodeyBox 0.7 ships Linux only for host-enforced local VM sandboxes. The orchestrator also runs on macOS and Windows, but only in the remote-executor topology: VMs execute on a Linux executor host while the orchestrator runs locally. One Mac-native exception: the Tart plugin gives local VMs on Apple Silicon with VM isolation, whose egress is verified per sandbox (below) rather than enforced on the orchestrator host. This page records why, and what is supported where.

The egress question

Sandbox isolation is enforced on the host, by nftables, on per-profile Linux bridges (scripts/setup-host-networks.sh). A sandbox VM attaches to a bridge whose host-side rules drop everything not on that profile's allowlist. The drops happen in the host kernel, on bridges the guest cannot see — an agent with sudo inside the VM cannot switch them off.

That mechanism is Linux-only, and the assessed alternatives do not reach equivalence — with one host-owned exception below the table: a provider-host packet filter (today: Tart Softnet on the Mac host) can serve profiled work after a per-sandbox canary, but it is never ranked above orchestrator-host enforcement. VM isolation (own guest kernel) and egress enforcement are separate claims: macOS + Tart gives the first (DedicatedKernel via Virtualization.framework) while the second stays NotEnforced until Softnet mode, operator opt-in, and a passing canary promote one sandbox at a time.

Candidate Verdict Reason
macOS pf anchor per VM (for Multipass) Rejected Multipass on macOS uses Apple's hypervisor framework with its own NAT network; there is no per-VM bridge attachment point in the host kernel where an unbypassable per-profile drop path can be installed. pf rules also require disabling SIP-protected defaults or installer-owned anchors that OS upgrades can reset.
Tart Softnet per-VM filter (for tart) Verified per sandbox Softnet runs on the Mac host, outside the guest, with a per-VM vmnet network and an IPv4 block-all default plus the resolved allowlist. Static classification stays NotEnforced; profiled work is served only after the host-owned per-sandbox canary passes (opt-in + Softnet mode + canary), ranked strictly below orchestrator-host enforcement. See below.
Windows Filtering Platform (WFP) callout per VM Rejected Multipass on Windows runs on Hyper-V with a virtual switch owned by the provider. Per-VM allowlist enforcement would need a custom WFP callout driver plus signed-driver install — an unsigned script cannot install an unbypassable host-kernel drop path, and Hyper-V NAT networks give no stable per-VM L2 hook.
In-guest filtering (iptables/nftables inside the VM) Rejected The threat model assumes a root agent. Anything enforced inside the guest is flushable from inside the guest (iptables -F). Enforcement must live outside the attacker-controlled kernel.
Per-provider user-space proxy on the host Rejected A proxy only constrains traffic that goes through the proxy. A root agent in a bridged/NAT VM can route around it. It is allowlist-shaped logging, not isolation.
Remote-executor topology (orchestrator on macOS/Windows, VMs on a Linux executor) Supported Enforcement stays exactly where it is today: nftables on Linux bridges on the executor host. No new mechanism is needed and no isolation claim is weakened.

"Only the remote-executor topology is supported on non-Linux hosts" is the staged answer for host-enforced egress: it reuses the proven enforcement instead of inventing a weaker one per platform. Tart on macOS is the one local-VM exception, and only through the per-sandbox verified path below — never as static enforcement.

Verified provider-host filters (per-sandbox, never static)

One host-owned concession exists for filters that run outside the guest on the provider host but outside the orchestrator host's kernel — today, Tart Softnet on the Mac host. The static classification never changes for these kinds (HostPlatformSupport.GetEgressEnforcement still reports NotEnforced; a plugin can never promote itself). Instead, placement may hand a profiled sandbox to an opted-in kind only after the host's own canary passes for that sandbox:

  • Opt-in is host config, CodeyBox:EgressVerification:Kinds (for example ["tart"]). A plugin cannot set it.
  • The canary runs through the normal sandbox exec path right after create and before any work phase: a TCP connect to an allowlisted destination must succeed; connects to a configured canary destination, to a global IPv6 address, and to the Mac host's LAN address must all fail within the per-check bound. The result and timings are recorded as an event.
  • On failure the sandbox is disposed, a loud alert is emitted, the kind is demoted to NotEnforced for the configured cool-down, and the item is re-placed on an enforced provider — never run unverified.
  • Long-lived sandboxes can re-verify on a cadence (ReverifyInterval), and a provider filter-process signal (IEgressFilterHealth) fails the sandbox closed when the filter dies.
  • Workload-trust routing ranks the verified value strictly below orchestrator-host enforcement: it is a concession for where static enforcement cannot exist, never an upgrade.

The Mac-only operator procedure (Softnet install, canary endpoints, scripted checks) lives with the provider: plugins/sandbox/CodeyBox.TartSandboxPlugin/README.md. Anything needing a real Mac is written there as an explicit procedure, not claimed as verified — no CI or dev machine here is a Mac, and the in-tree tests cover the host logic against fakes only.

Supported matrix

Orchestrator host incus multipass (local) tart (plugin, macOS only) multipass-remote sprites bubblewrap process (dev-only)
Linux ✅ enforced on host ✅ enforced on host ❌ macOS only ✅ enforced on executor ✅ enforced on executor ⚠️ shared kernel, no egress ⚠️ no isolation, dev only
macOS ❌ ❌ ✅ VM isolation (DedicatedKernel); egress NotEnforced by default, EnforcedOnProviderHostVerified per sandbox when Softnet mode is on, the kind is opted in, and the canary passes ✅ enforced on executor ✅ enforced on executor ❌ ❌
Windows ❌ ❌ ❌ ✅ enforced on executor ✅ enforced on executor ❌ ❌

Legend: ✅ = supported with network isolation; ⚠️ = runs but must never be described as isolated (see concepts/security.md sharp edges); ❌ = rejected at startup with a message pointing here.

Running the orchestrator on a platform does not imply guest sandboxes for it: guests are always Linux VMs. The matrix is about where the orchestrator runs, not what the agent edits.

The matrix is executable, not just prose: CodeyBox.Core.HostPlatformSupport declares it, the API startup path (SelectSandboxProvider) and options validation reject unsupported combinations, and HostPlatformSupportTests asserts the documented matrix matches what the code registers.

Quickstart per host OS

macOS / Windows (remote-executor only)

  1. Provision a Linux executor host (any KVM-capable Linux box or VM) and run sudo scripts/setup-host-networks.sh there. Verify with nft list table inet codeybox on the executor.
  2. From the executor, confirm Multipass is reachable over SSH from your machine (ssh user@executor multipass list).
  3. Build the orchestrator locally:
    • macOS/Linux: ./build.sh
    • Windows (PowerShell 7+): ./build.ps1
  4. Configure CodeyBox:SandboxProvider to multipass-remote (or sprites) with the executor's SshTarget, and run the normal getting-started.md flow from step 4 onward. Queue a trivial work item; it executes in a VM on the executor, under the executor's allowlist.
  5. GitRootDirectory and StateDatabasePath default per OS (%PROGRAMDATA%\CodeyBox\... on Windows, /var/lib/codeybox/... elsewhere); override in config if desired.

Linux (full support)

Follow getting-started.md as written, including sudo scripts/setup-host-networks.sh on the orchestrator host itself.

Egress verification procedure

Isolation is claimed only where the enforcement mechanism has been exercised. To demonstrate the block on any host claiming isolation:

Linux orchestrator or Linux executor host:

sudo scripts/setup-host-networks.sh /etc/codeybox/networks.conf
nft list table inet codeybox   # allowlist rules present per bridge
# Launch a sandbox on the `isolated` profile, then from inside it:
curl -m 10 https://api.anthropic.com -o /dev/null -w '%{http_code}\n'  # expect timeout/drop
getent hosts api.anthropic.com  # DNS resolves, TCP does not establish

Expected result: DNS may resolve but no TCP connection outside the profile allowlist establishes; nft counters on the bridge chain increment on drop. Record the nft output with the deployment — that is the isolation evidence, not the config file alone.

macOS/Windows orchestrator: run the same procedure on the Linux executor host. No isolation is claimed for anything enforced (or unenforced) on the macOS/Windows machine itself.

macOS orchestrator with Tart Softnet: the per-sandbox verified path has its own Mac-only operator procedure — scripts/verify-tart-softnet.sh plus docs/extending/tart-sandbox-plugin.md. It establishes the two properties CI cannot prove (fail-closed when the Softnet process dies; no IPv6 egress). Until an operator has run it on real hardware, the verified path is documented as unverified.

Build packaging note: the ACP bridge

src/CodeyBox.Agents.Claude/Resources/acp-bridge is a statically-linked Linux ELF (built via scripts/publish-acp-bridge.sh, which needs musl, clang, lld, and a Multipass verification VM). It runs inside Linux guests, so a macOS/Windows orchestrator host never executes it — but only a Linux host can rebuild it. The prebuilt resource is checked in, so ./build.sh / ./build.ps1 on any host consume it without rebuilding; do not attempt to regenerate it from macOS/Windows.