feat(gateway): support process-level runtime image overrides

[elezar]

User Story

As an OpenShell operator or release qualification maintainer, I want to override the trusted sandbox and supervisor runtime image references when the gateway starts, so that I can run an unmodified package against exact candidate artifacts without generating driver-specific gateway configuration.

Problem Statement

OpenShell embeds default sandbox and supervisor image tags into the gateway binary at build time. At runtime, the only supported overrides live in driver-specific TOML tables such as [openshell.drivers.docker], [openshell.drivers.podman], and [openshell.drivers.kubernetes].

Package qualification therefore has to create a driver-specific gateway.toml even when the only desired change is selecting exact candidate runtime images. This makes qualification-only configuration look like required package configuration and requires the same artifact selection to be expressed separately for each driver.

This need surfaced while adding Debian-package qualification in #3461 as part of #3454.

Impact / Why This Matters

Today, release tooling must either:

• generate and select a driver-specific gateway configuration;
• rebuild the gateway with a test-specific embedded image tag; or
• retag candidate images to match the binary's compiled defaults.

Generating TOML conflates harness-owned artifact selection with operator-owned gateway configuration. Rebuilding means the test no longer exercises the exact release binary. Retagging is unreliable for mutable defaults such as dev and latest, which the local drivers intentionally refresh.

A process-level override would let package tests preserve the package-owned service and configuration lifecycle while pairing the unmodified gateway binary with exact, immutable sandbox and supervisor artifacts. It would also give operators a concise way to use registry mirrors or digest-pinned trusted runtime components without maintaining otherwise-empty driver tables.

Proposed Design

Allow the gateway process to accept complete runtime image references through environment variables:

OPENSHELL_SANDBOX_RUNTIME_IMAGE=registry.example.com/openshell/sandbox@sha256:...
OPENSHELL_SUPERVISOR_IMAGE=registry.example.com/openshell/supervisor@sha256:...

These values act as process-level defaults for built-in compute drivers that consume the trusted sandbox and supervisor OCI images. Explicit values in the selected driver's TOML table remain authoritative.

The observable precedence is:

driver-specific TOML > process environment > compiled release default

The variables accept complete tagged or digest-pinned OCI references. Package-managed services can receive them through their existing environment-file mechanism. Package preflight and actual startup must resolve and validate the same effective references. Sandbox requests cannot override these gateway-owned trusted runtime images.

Acceptance Criteria

• The gateway accepts process-level sandbox-runtime and supervisor image references at startup.
• Docker, Podman, and Kubernetes use the process-level references when their selected driver configuration does not set explicit image references.
• Explicit driver-specific TOML image references override the process environment.
• When the environment variables are unset, existing compiled defaults and behavior remain unchanged.
• Tagged and digest-pinned OCI references are supported.
• Package preflight validates the same effective references used by gateway startup.
• Debian and RPM user services can consume the overrides through their existing gateway.env mechanism.
• End-to-end coverage installs an unmodified package, supplies exact candidate runtime images through the environment, and runs without generating an operator gateway.toml.
• Documentation describes the variables, precedence, supported drivers, and trust implications.
• The effective runtime image references and their configuration source are visible in startup diagnostics without exposing credentials.

Alternatives Considered

Generate driver-specific TOML. This works today, but duplicates a cross-driver artifact-selection concern and makes qualification configuration appear operator-owned.

Compile a gateway specifically for qualification. This can embed the candidate tag, but the resulting binary is not the release artifact being qualified.

Retag local images to the compiled references. This can work for immutable release tags, but mutable dev and latest references are refreshed and may resolve to a different remote image.

Expose only a shared tag variable. A tag-only override cannot select another registry or an immutable digest, so complete image references are more generally useful.

Agent Investigation

The shared defaults are produced by default_sandbox_runtime_image() and default_supervisor_image() in crates/openshell-core/src/config.rs; their tag comes from the build-time OPENSHELL_IMAGE_TAG or IMAGE_TAG. Docker, Podman, and Kubernetes consume these shared defaults but expose runtime overrides only through their own driver configuration tables. The Debian and RPM package units already load ~/.config/openshell/gateway.env, providing a package-compatible delivery mechanism for process-level settings.

[elezar]

🏗️ build-plan

Implementation Plan

Issue type: feat
Complexity: Medium
Confidence: High — the three in-process OCI drivers converge at the gateway composition boundary, package services already import gateway.env, and the Debian qualification lane added by #3461 provides a black-box path.

Summary

Add gateway-process defaults for the trusted sandbox-runtime and supervisor OCI references, resolving each effective value with driver-specific TOML > process environment > compiled release default. Resolve and validate the same values in both config preflight and daemon construction, report each effective reference and source at startup, preserve the sandbox trust boundary, and convert Debian package qualification from a generated gateway.toml to the packaged service's gateway.env hook.

Scope

• crates/openshell-gateway/src/lib.rs: resolve and validate trusted runtime image overrides consistently for Docker, Podman, and Kubernetes in both preflight and startup; emit safe startup diagnostics with the effective source.
• crates/openshell-server/src/compute/driver_config.rs: expose enough selected-driver TOML metadata to distinguish an explicit field from environment/default values without changing the public TOML schema.
• Driver and gateway tests: cover defaults, environment values, TOML precedence, complete tags and digests, invalid values, and rejection of trusted-image fields in sandbox request configuration.
• tests/ansible/playbooks/openshell-deb.yaml: remove the qualification-only TOML and provide the candidate Docker runtime images through gateway.env so the installed package's preflight and daemon exercise the same inputs.
• Package asset checks: preserve Debian and RPM EnvironmentFile wiring across preflight and startup.
• Gateway, compute-driver, package/man, and architecture documentation: document variables, precedence, supported drivers/reference forms, diagnostics, package delivery, and trust implications.

Implementation Steps

1. Add a pure resolver for explicit driver fields, process overrides, and compiled fallbacks. Validate and preserve complete OCI tag/digest references and classify their source.
2. Route Docker, Podman, and Kubernetes factory validation/build through the resolver, ensuring preflight and startup construct the same effective typed configuration. Log effective values and sources only after safe validation.
3. Add regression coverage proving sandbox requests cannot select either trusted runtime image.
4. Replace the Debian qualification TOML with gateway.env values and retain the existing installed-package conformance flow; assert Debian/RPM package wiring loads the file for preflight and startup.
5. Update operator, package, and architecture documentation.
6. Run focused crate/package tests, the installed-Debian tmachine conformance lane, mise run pre-commit, and full local CI as feasible.

Test Plan

• Unit tests: Gateway resolver/factory precedence, tags/digests, invalid inputs, and source reporting; server selected-field detection; Docker/Podman/Kubernetes request-schema rejection.
• Integration tests: Package asset checks plus Ansible/static validation for the changed package qualification profile.
• E2E tests: nix run .#tmachine -- test ubuntu-docker-rootful deb conformance, which installs the unmodified Debian package, loads candidate image tarballs, starts the packaged user service through gateway.env, and exercises sandbox create/exec/delete without an operator gateway.toml.

Risks & Open Questions

• Reject malformed, empty, non-Unicode, or credential-bearing environment values before logging them. Registry authentication remains engine/Kubernetes-secret configuration, not part of the image reference.
• Preserve Docker's existing supervisor_bin escape hatch; avoid claiming a sandbox-runtime image is active when that host binary is selected.
• Apply overrides only to built-in Docker, Podman, and Kubernetes, never remote/custom drivers. Helm-rendered explicit Kubernetes fields remain authoritative.

Documentation Impact

Update docs/reference/gateway-config.mdx, docs/reference/sandbox-compute-drivers.mdx, relevant package/man documentation, and architecture/build.md. No gateway TOML schema key or Helm value is added.

LSM Compatibility

No SELinux/AppArmor impact is expected. This changes configuration resolution only; it does not alter process identity, /proc access, binary execution, or inter-process visibility.

---

Revision 1 — initial plan based on origin/main 65eb916

[elezar]

🏗️ build-from-issue-agent

Implementation Complete

PR: #3504

What was built

The gateway now resolves trusted sandbox-runtime and supervisor OCI images for Docker, Podman, and Kubernetes with driver TOML > process environment > compiled default precedence. Preflight and startup share the resolver, startup diagnostics report the effective source, sandbox requests remain unable to select trusted runtime images, and Debian qualification now uses the packaged gateway.env mechanism without generating operator TOML.

Tests

• Unit: 8 focused resolver/trust-boundary tests passed
• Integration: package asset checks passed
• E2E: attempted locally; tmachine stopped before package installation because the required staged Debian/image/test artifacts were absent. The existing Release Dev/Tag package-conformance lane supplies these inputs.

Docs updated

• Gateway configuration and compute-driver references
• Linux installation and RPM configuration guides
• Gateway man page, build architecture, and cluster-debugging skill

The issue will auto-close when the PR is merged.