droid-control is built around one constraint: the agent operating the tool is also the runtime. Architecture is therefore not just code organization. It is information architecture for a droid that must decide what to load, what to ignore, what to delegate, and what evidence proves the work.
Editable source: diagrams/architecture-routing.excalidraw
The plugin is designed to keep a droid focused while it operates real software:
- Low context load: load the Linux tuistory path without dragging in Windows KVM notes, macOS VM controls, browser automation, and renderer internals.
- Evidence-first workflows: every command starts by making commitments, then ends by verifying the artifact against those commitments.
- Parallel execution: independent capture environments and render jobs can run in workers. Shared desktop input stays serialized; session names do not isolate focus.
- Clear ownership: commands decide what must be produced; atoms decide how to execute their slice.
- Platform specificity: OS-specific mechanics live in platform subdocuments, not in global instructions.
The three user-facing commands are deliberately thin:
| Command | Contract |
|---|---|
/demo |
Turn a PR or feature description into a visible proof story and a video deliverable. |
/verify |
Test a claim as an investigator and report whether the evidence confirms or refutes it. |
/qa-test |
Drive a terminal, browser, Electron, or native desktop flow and report step-level PASS / FAIL / BLOCKED evidence. |
A command parses arguments into commitments: layout, comparison mode, evidence type, video/showcase requirements, keystroke overlays, and any user-specified constraints. Those commitments are not suggestions. The verify stage later checks them explicitly.
This is the first guardrail against agent drift. The droid does not start with "make something impressive." It starts with a checklist it must satisfy.
skills/droid-control/SKILL.md is an orchestrator, not a controller. It does not run a state machine or encode every workflow. It performs three independent routing lookups and tells the droid which atom skills to load.
| Route | Question | Examples |
|---|---|---|
| Target | What are we driving? | Droid CLI, other terminal TUI, web/Electron app, native desktop app, raw PTY bytes |
| Stage | What does the workflow need? | capture, compose, verify |
| Artifact | Does compose need polish tools? | showcase presets, effects, keystroke overlays |
The routes compose without a cross-product explosion. Adding a new target means writing one target atom and one routing row; capture, compose, and verify can work with it immediately if the handoff shape is respected.
Atoms are plain files under skills/droid-control/atoms/<atom>/ATOM.md, not skills. Only droid-control appears in a session's skills list; a droid reads an atom only after routing selects it, by the absolute path the orchestrator gives it through ${DROID_PLUGIN_ROOT}. Every installed droid-control therefore costs one skill entry of context, not eleven.
Each atom is a self-contained surface the droid reads at a specific point in the workflow:
| Atom type | Atoms | Responsibility |
|---|---|---|
| Driver atoms | terminal-use, true-input, browser-use, desktop-use |
How to drive a class of environment. terminal-use is the terminal entrypoint; it runs the tuistory backend and routes real-terminal proof to true-input. |
| Target atoms | droid-cli, pty-capture |
Target-specific shortcuts, launch rules, and byte-capture patterns. |
| Stage atoms | capture, compose, verify |
Lifecycle phases with explicit inputs and outputs. |
| Polish atom | showcase |
Visual presets and cinematic layer guidance. |
The important property is not just smaller files. It is temporal relevance: the droid reads the capture rules while capturing, the compose rules while composing, and the verification rules when it has something to check.
The workflow is a waterfall because each atom hands the next atom exactly what it needs:
command commitments
-> routed atom set
-> capture outputs clips, screenshots, byte dumps, metadata
-> compose outputs a rendered artifact and render metadata
-> verify checks technical quality and original commitments
-> report summarizes evidence and conclusion
No central engine enforces this order. The documents make the next step obvious enough that the droid follows the flow naturally. This keeps the system easy to extend: new behavior is usually a new atom or a new row, not a new orchestration framework.
The compose handoff has two halves:
- Mechanical fields: layout, labels, clip paths, speed, fidelity, preset, output path, effects tier.
- Creative intent: what the viewer should understand, which moments matter, and how to frame the proof.
Mechanical fields prevent hallucinated parameters. Creative intent prevents paint-by-numbers output. The effects tier is the pattern in miniature: the command commits none, utilitarian, or full; compose chooses specific zooms, spotlights, and overlays only after it has real recordings to inspect.
The parent droid keeps judgment. Workers get exact commands.
| Work | Owner | Reason |
|---|---|---|
| Short interactive desktop task | Parent | One controller owns observation, input, permission waits, and cleanup. |
| Interpret PR / claim / QA goal | Parent | Requires context and judgment. |
| Write the interaction script | Parent | Defines the proof story. |
| Capture baseline and candidate branches | Workers only for independent environments | A shared desktop must be captured serially. |
| Render showcase video | Worker droid | Mechanical once props and clips are fixed. |
| Verify commitments | Parent | Requires the original contract and evidence judgment. |
This boundary follows the stage handoffs. Capture workers need resolved tctl commands and worktree paths, not PR context. Render workers need a props JSON and clip paths, not a feature explanation.
Editable source: diagrams/capture-compose-verify.excalidraw. The diagram shows the terminal comparison flow, where each branch has an isolated tuistory environment and can run in its own worker; a shared desktop is captured serially by the parent (see Delegation boundaries).
Every workflow starts by creating a run scope:
RUN_ID="$(date +%s)-$$"
RUN_DIR="$(mktemp -d /tmp/droid-run-${RUN_ID}-XXXXXX)"The run scope is not cosmetic. tctl sessions share /tmp/tctl-sessions/, and many droids may be filming on the same machine. Session names must be prefixed with RUN_ID; recordings, props, screenshots, and rendered videos must live under RUN_DIR.
Terminal workflows use bin/tctl as the only launch/control boundary. It hides two very different execution paths behind the same command shape:
| Backend | What tctl manages |
Best for |
|---|---|---|
tuistory |
Virtual PTY sessions, deterministic waits/snapshots, asciinema recording at launch. | Fast TUI automation and most demo captures. |
true-input |
Headless Wayland compositor, real terminal emulator, native key injection, PTY log/screenshot/video capture. | Real terminal rendering or keyboard-encoding proof. |
tctl also enforces Droid CLI launch invariants. droid-dev sessions must provide --repo-root, which lets tctl set DROID_DEV_REPO_ROOT and record provenance for the captured branch and commit. --cwd is independent: it sets the child's working directory (recorded in provenance) and defaults to --repo-root when unset, so a session can run one worktree's code from a different project directory.
Browser/Electron and native-desktop workflows intentionally do not go through tctl. They have their own control boundaries: agent-browser's persistent CDP daemon for DOM snapshots, screenshots, and connected apps; cua-driver's daemon for accessibility trees and per-(pid, window_id) element caches on desktop GUIs.
The compose stage uses the fframes project in fframes/ as a single video engine: a Rust crate whose droid-showcase binary renders the Showcase composition from SVG frames. The droid writes a Showcase props JSON; scripts/render-showcase.sh builds the binary on first use (cargo) and runs it. The binary owns the mechanical rendering pipeline:
- Accept
.cast,.mp4, and.webmclips only; validate props and resolve defaults and fidelity (omitted: side-by-sideinspect, singlestandard). - Convert
.castrecordings throughaggandffmpegat 1x, keeping the recording's timeline. - Stage clips as
clip-<index>inside a work directory created for this render only. - Probe every clip with
ffprobe; the longest one sets the content length. - Print the resolved plan (
showcase plan: {...}on stderr), then render every frame on all cores and encode limited-rangeyuv420p/bt709H.264 throughffmpeg, or one PNG frame with--still. - Remove that render's work directory on exit, failure, or SIGINT/SIGTERM.
fframes/src/timing.rs owns the timeline: the composition applies speed once to every clip, the clips run for longest clip / speed, and the content sequence is padded by one crossfade on each side so the clips start after the title crossfade and the final frame is held through the outro crossfade. Total length is 4s title + longest clip / speed + 3.5s outro; a shorter clip holds its final frame. This keeps droids out of the common failure modes: two recording.mp4 inputs overwriting each other, concurrent renders deleting each other's clips, a mismatched clip duration, casts sped up twice, wrong agg theme, invalid pixel formats, and hand-written encode commands with missing flags.
fframes/src/showcase.rs is the only video entry point (Showcase, an fframes Video). It composes the layer modules by props:
| Layer | Module | Purpose | Controlled by |
|---|---|---|---|
| Background + particles | scenery.rs |
Preset-driven warmth or coolness | preset |
| Title card / Droid outro | title.rs, outro.rs |
Opening and closing cards (outro plays fanning rotor → crossfade → DROID wordmark) | title, subtitle, speedNote |
| Window chrome + layouts | content.rs |
Single or side-by-side windows, panel entrances, zooms, spotlights | layout, labels, windowTitle, objectFit, effects |
| Callouts | callout.rs, cast.rs |
Cards pointing at a region or at text found on a cast clip's screen (replayed through avt, the emulator agg renders with) | effects |
| Section sweeps and headers / keystrokes | overlays.rs |
Timed in-scene overlays | sections, keys |
| Code cards | overlays.rs, code.rs |
Timed syntax-highlighted code cards | codeAnnotations |
| Transition presentation | transition.rs |
Title→content and content→outro crossfade | transitionStyle (default motion-blur) |
| Noise + colour grade + watermark | scenery.rs |
Topmost polish pass | fidelity, preset |
The key property is that the main composition is data-driven: the droid never writes composition code. Adding a new overlay or transition style is a new layer function plus a props field, not a new composition. Fonts (Geist, Geist Mono) are embedded in the binary, so renders look the same on every machine.
Platform-specific mechanics live below the atom that needs them:
skills/droid-control/atoms/true-input/platforms/linux.md
skills/droid-control/atoms/true-input/platforms/windows.md
skills/droid-control/atoms/true-input/platforms/macos.md
skills/droid-control/atoms/pty-capture/platforms/linux.md
skills/droid-control/atoms/pty-capture/platforms/windows.md
skills/droid-control/atoms/pty-capture/platforms/macos.md
skills/droid-control/atoms/desktop-use/ATOM.md
A Linux droid reads Linux Wayland instructions. A Windows VM byte-capture task reads Windows KVM instructions. The system does not rely on the droid to skim irrelevant sections correctly.
Desktop-use keeps routine setup, target selection, recovery, and recording rules in one compact entrypoint. It includes a host setup table rather than copied platform manuals. No user-home skill dependency or runtime documentation download is required.
For ordinary desktop tasks, the driver verifies each action and reports directly. Capture/verify stages are loaded for formal evidence deliverables, and compose only when a produced artifact was requested. Explicit GUI-only or cua-only constraints take precedence over Electron's default browser route.
Use the same composition rules when adding capability:
| Change | Preferred shape |
|---|---|
| New user workflow | Add a command that parses arguments into commitments, then routes through existing atoms. |
| New target type | Add one target atom and one target-route row. |
| New capture backend | Add a driver atom or extend tctl only if it belongs behind the same terminal boundary. |
| New visual treatment | Add props support and a layer in fframes/src/, then document compose/showcase behavior. |
| New platform mechanics | Add a platforms/<os>.md file under the relevant atom. |
If a change makes every droid read more global instructions, it is probably fighting the architecture. Prefer a new scoped surface over a larger shared surface.
droid-control is a small composition system for agent attention:
intent contract + orthogonal routing + scoped atom surfaces + explicit handoffs
= real-app automation that stays focused, parallelizable, and verifiable