feat(sdk): add strict Dockerfile sandbox launch

[Peng-YM]

Design RFC: #17

What changed

This PR adds a backend-neutral, strict Dockerfile direct-launch path to the
Python SDK. Sandbox(context=...) uses the existing public Sandbox,
Commands, and Filesystem facades: it parses one Dockerfile, creates a
sandbox from FROM, applies the explicit supported instructions, and exposes
the optional background startup command as sandbox.startup_command.

The branch has been rebased onto the backend-neutral upstream main
architecture. The implementation keeps backend-native conversions behind the
existing boundary rather than coupling Dockerfile launch to either backend.

The public surface includes DockerContext, LocalDockerContext,
parse_dockerfile, check_direct_launch, apply_dockerfile, typed parse/check
and apply results, and constructor options context, auto_start_cmd, and
build_run_timeout. The SDK README and maintained
sdk/python/examples/dockerfile_launch.py now document and exercise the
current contract.

Why

Small single-stage Dockerfiles are a common way to describe a development
environment. Direct launch supports a narrow, auditable subset without
requiring BuildKit, a Docker daemon, or a registry push. It is intentionally
not a Docker build replacement: every launch reruns supported RUN, COPY,
and ADD instructions and creates no cache or snapshot.

Supported contract

FROM is deliberately rootfs-only. The base image supplies the sandbox root
filesystem, while its OCI ENV, USER, WORKDIR, CMD, and ENTRYPOINT
configuration is not inherited. The Dockerfile must state required runtime
settings explicitly.

The supported subset is exactly one literal FROM with optional AS alias;
shell-form RUN; shell-form local COPY/ADD sources for files, directories,
., and wildcards; literal --chown; literal local tar extraction; literal
ENV; absolute WORKDIR; literal USER; EXPOSE metadata; and exec- or
shell-form CMD/ENTRYPOINT with normalized OCI-style argv combination.

Multi-stage input, COPY --from, ARG, remote ADD URLs, JSON COPY/ADD,
exec-form RUN, relative WORKDIR, build-time variable expansion, unsupported
flags, and unsupported instructions are rejected. Execution paths fail closed:
Sandbox(context=...) parses strictly, check_direct_launch() returns False
with reason codes, and apply_dockerfile() rejects parsed unsupported items.
Non-strict parsing is diagnostics only.

After instructions complete, the SDK polls sandbox readiness and dispatches the
resolved startup argv in the background. A non-None CommandHandle confirms
dispatch, not process longevity or application health. Callers can wait() or
kill() the handle and own their application-specific health check. Dockerfiles
without a startup command, or launches with auto_start_cmd=False, expose
None.

Security and correctness

The context implementation validates the complete manifest and materializes all
COPY/ADD inputs before any sandbox operation. It applies root
.dockerignore, filters reserved Dockerfile metadata, rejects local symbolic
links, and validates paths and collisions. Local tar extraction accepts only
regular files and directories with safe paths. --chown is literal, quoted,
and limited to outputs created by the current instruction. Remote ADD is
rejected by design, avoiding network retrieval and host-side SSRF.

For unsupported Dockerfiles or build-once reuse, callers externally pre-build
an image, use Sandbox(image=...), and explicitly launch the desired command.
The SDK does not auto-start image configuration on that path.

Testing

The current unit gate contains 196 tests and passes with Ruff and mypy. It
covers strict parser behavior, startup argv resolution and cleanup, context and
.dockerignore filtering, pre-upload manifest materialization, collisions,
path validation, local tar traversal, ownership validation, and remote ADD
rejection. The maintained example adds independent sections for filtered
COPY ., finite CMD and ENTRYPOINT+CMD startup handles, COPY --chown,
local tar ADD, and a no-sandbox fail-closed precheck.

Real-sandbox follow-up

The maintained five-section example passed against a real standalone runsc
deployment through the backend-neutral facade using openyuanrong-sdk:

• filtered COPY . honored .dockerignore; the ignored secret was absent;
• explicit RUN/ENV/WORKDIR/USER state reached a finite CMD, and its
  startup_command.wait() completed successfully;
• exec-form ENTRYPOINT + CMD merged and produced the expected marker;
• COPY --chown and local tar ADD produced the expected owner and file tree;
• remote ADD was rejected during pre-check without creating a sandbox.

Additional live regressions placed pre-existing root-owned files in both COPY
and tar destinations. Those files remained root:root; only files and
directories created by the current instruction became myuser:myuser.

The same code is backend-neutral and has unit coverage for both backend
sessions. The default openyuanrong-sandbox backend could not be runtime-tested
with the current public standalone image: it lacks the backend's
/api/sandbox/v1 route and returns HTTP 404 before sandbox creation.

Licensing

The parser dependency is dockerfile-parse (BSD-3-Clause); context matching
uses pathspec (MIT). The value-level parsing approach was informed by the E2B
Python SDK (MIT). No Docker engine, BuildKit, or registry component is added.

[tianyuzhou95]

Thanks for putting together the RFC and the reference implementation. The overall direction is useful, and the submitted unit tests, Ruff, and mypy checks pass locally. However, the current implementation has several correctness and security issues that need to be addressed before merge.

Blocking findings

1. SDK-side ADD URL creates an SSRF boundary violation. _dockerfile_runner.py lines 234-244 downloads remote URLs with urllib in the SDK process and then uploads the response into the sandbox. An agent-supplied Dockerfile can therefore read loopback, link-local, cloud metadata, or private-network endpoints reachable from the SDK host and retrieve the response from inside the sandbox. Restricting redirect schemes does not address the host-level SSRF. The fetch should happen inside the sandbox, or remote ADD must be an explicit trusted-input-only opt-in with an enforceable network policy.

2. The build context bypasses .dockerignore and does not implement consistent Docker context semantics. _dockerfile_runner.py lines 246-275 passes a local directory directly to copy_from_local, so COPY . ... can upload files such as credentials, .env, or .git even when they are excluded by .dockerignore. DockerContext.walk() is not used, and non-local contexts cannot correctly support directories or wildcards. Please build a filtered context manifest, apply .dockerignore, expand source patterns, and use the same behavior for local and remote contexts.

3. Valid Dockerfile syntax is accepted as directly launchable but executed incorrectly. _dockerfile.py lines 433-487 parses JSON-form COPY/ADD with shlex, so COPY ["a b", "/dest/"] becomes source "[a b," and destination "/dest/]". Exec-form RUN is also treated as a shell string, and flags such as --chmod and --link are silently discarded. In addition, _dockerfile_runner.py lines 129-139 resets relative WORKDIR values instead of resolving them against the previous directory and ignores ARG, even though build arguments affect FROM, later instructions, and RUN. Unsupported syntax must either be implemented or rejected by check_direct_launch; it must not return success and then run with different semantics.

4. The evaluator does not inherit the base image configuration. It initializes build state as empty environment, /, and root, and resolves CMD/ENTRYPOINT only from instructions in the current Dockerfile. A Dockerfile containing only FROM nginx, for example, is reported as launchable but does not start the inherited nginx command; inherited USER, WORKDIR, CMD, and ENTRYPOINT are likewise lost. This needs OCI image-config plumbing or a narrower explicitly documented contract. Without it, the feature cannot claim that the resulting sandbox behaves like the Dockerfile.

5. The documented warning policy is silent in the main API. sandbox.py lines 292-303 discards the DockerfileApplyResult, so warnings for ignored instructions never reach callers of Sandbox(context=...). Unknown instructions also remain warnings under strict=True. Please reject unsupported behavior by default, or emit/expose the warnings and make strict mode consistently reject every ignored instruction.

6. CMD/ENTRYPOINT resolution and readiness do not match the stated contract. _dockerfile.py lines 154-167 drops a shell-form CMD when paired with an exec-form ENTRYPOINT, whereas Docker appends /bin/sh -c .... The launcher also infers shell form from whether a single argument contains a space, which breaks valid one-element exec-form commands. Finally, the readiness check only confirms that the sandbox is alive before starting the background command, and the returned process handle is discarded, so an immediately failing application can still produce a successful constructor. Please preserve the parsed command form explicitly and either define an application-start check or describe this as sandbox readiness rather than application readiness.

I recommend revising the RFC around a strict, trusted Dockerfile subset and making unsupported constructs fail closed. The PR also needs to be rebased onto the current backend-neutral SDK architecture before these changes can be integrated.

[Peng-YM]

Thanks for the detailed review. I addressed each blocking finding in commit 4f2a35e, and rebased the branch onto the current upstream/main. Here is the point-by-point resolution.

1. SDK-side ADD URL / SSRF

Resolved by removing remote downloads from the execution path entirely.

• Any URL source in ADD is now classified as remote_add by check_direct_launch().
• Strict parsing rejects it before a backend session or sandbox is created.
• The runner no longer contains an urllib download path.
• Non-strict parsing remains diagnostics-only and cannot be passed to execution when unsupported items are present.

This is intentionally fail-closed rather than an opt-in host-side fetch.

2. .dockerignore and consistent context semantics

Resolved with a shared, filtered context manifest used by both local and custom DockerContext implementations.

• The manifest is built from DockerContext.walk() and reads file content only through DockerContext.open().
• Root .dockerignore rules are applied with Docker-style ordering, negation, directory, and wildcard behavior.
• COPY/ADD support literal files, directories, ., and wildcard expansion with deterministic ordering.
• Dockerfile and .dockerignore are reserved and are never copied by COPY ..
• Local and non-local contexts now use the same source-selection and destination logic.
• Invalid, duplicate, escaping, colliding, and ignored-only sources fail before sandbox mutation.
• LocalDockerContext.open() uses descriptor-relative, no-follow opens and rejects symlinks, preventing context escape and symlink races.

3. Accepted syntax executed with different semantics

Resolved by narrowing the executable subset and making unsupported syntax fail closed.

check_direct_launch() and strict parsing now reject, among other cases:

• JSON-form COPY/ADD
• exec-form RUN
• RUN flags such as --mount, --network, and unknown flags
• relative WORKDIR
• ARG
• FROM flags or variable expansion
• COPY/ADD --chmod, --link, --from, and unknown flags
• unsupported build-time variable expansion
• ignored or unknown instructions

Supported shell-form instructions retain their defined semantics. Unsupported constructs produce stable reasons such as unsupported_syntax, remote_add, or multi_stage; they are never converted into executable build instructions.

4. Base image configuration inheritance

Resolved by explicitly narrowing and documenting the contract: FROM supplies the root filesystem only.

The direct-launch path does not claim to inherit OCI image ENV, USER, WORKDIR, CMD, or ENTRYPOINT. A directly launched Dockerfile must declare any configuration it needs. check_direct_launch() reports this rootfs-only limitation in its successful result, and the README, RFC issue, and PR description all state it explicitly.

Full OCI image-config inheritance remains outside the scope of this change rather than being approximated incorrectly.

5. Warning policy and strict execution

Resolved by separating diagnostics from execution.

• Sandbox(context=...) always invokes parse_dockerfile(..., strict=True) before backend creation.
• Strict mode rejects every unsupported or ignored instruction.
• apply_dockerfile() independently rejects a parsed object containing unsupported items, so a non-strict diagnostic result cannot be executed accidentally.
• Non-strict parsing is only for check_direct_launch() and diagnostic inspection.
• The constructor stores the returned startup handle instead of discarding the relevant apply result.

There is therefore no silent-warning execution path in the main API.

6. CMD/ENTRYPOINT resolution and readiness

Resolved by preserving command form explicitly and implementing the merge matrix rather than inferring form from whitespace.

• Shell forms normalize to ('/bin/sh', '-c', command).
• Exec ENTRYPOINT + exec CMD concatenates argument vectors.
• Exec ENTRYPOINT + shell CMD appends /bin/sh -c <command>.
• Shell ENTRYPOINT ignores CMD, matching Docker behavior.
• One-element exec-form commands remain exec form.
• Launching uses shlex.join(argv) without whitespace heuristics.

The readiness contract is now precise: construction guarantees sandbox readiness and successful background command dispatch, not application health. The returned CommandHandle is exposed as Sandbox.startup_command, allowing callers to inspect or wait for the process. Dispatch failure raises DockerfileBuildError and triggers constructor rollback.

Backend-neutral rebase and cleanup

The branch is rebased onto the backend-neutral SDK architecture and now creates SandboxSpec, loads the selected backend, and operates through BackendSession, Commands, and Filesystem. Any parse/apply/dispatch failure terminates and closes the session, including detached sandboxes, while preserving the original exception.

Validation

• make sdk-check: 196 tests passed, Ruff passed, mypy passed for 24 source files.
• Live standalone coverage passed for .dockerignore + COPY ., shell RUN, ENV/WORKDIR/USER, exec ENTRYPOINT + CMD, COPY --chown, local-tar ADD, remote-ADD preflight rejection, and precise ownership behavior without recursive changes to pre-existing files.
• The current public standalone image does not expose /api/sandbox/v1 for the default openyuanrong-sandbox backend, so live validation used the optional openyuanrong-sdk backend through the same public SDK facade; this limitation is disclosed in the PR and RFC.

Could you please take another look when convenient?