Skip to content

feat: remote control — Stream Deck support, global shortcuts, token-gated local API#150

Merged
TheOrcDev merged 3 commits into
mainfrom
feat/remote-control
Jul 18, 2026
Merged

feat: remote control — Stream Deck support, global shortcuts, token-gated local API#150
TheOrcDev merged 3 commits into
mainfrom
feat/remote-control

Conversation

@TheOrcDev

@TheOrcDev TheOrcDev commented Jul 16, 2026

Copy link
Copy Markdown
Owner

Closes #143.

Two integration tiers

Global shortcuts (works today, no plugin): Settings → Global shortcuts registers OS-wide accelerators for record/stream/mic — bind them to a Stream Deck Hotkey action or any macro tool. Off by default.

Remote protocol + official plugin: a token-gated local surface with a hard security contract, plus apps/streamdeck-plugin (Elgato SDK v2) with six actions: Record, Go Live, Mic, Scene, Takeover (BRB), Window — key titles render backend-confirmed state.

Security model (public repo — assume hostile port probes)

  • Third transport role remote: allowlist of exactly two methods (remote.describe, remote.intent); session.*, oauth.*, everything else → forbidden-method (pinned by tests)
  • Remote sockets' event stream locked to remote.state/remote.ack at connection setup; the filter-mutation commands are refused
  • Off by default; rotatable token in the secret store; regenerate/disable closes live clients via a generation watch
  • Same-machine pairing: 0600 discovery file next to the app database (Discord-IPC trust model), removed on disable/shutdown
  • The backend only relays: intents execute in the renderer through the same code paths as the buttons — no second session-start, no validation bypass; state projection is minimal and leak-safe

Verification

  • smoke:remote-control (new, in smoke:local-gates) drives the real app end-to-end: discovery contract (0600/port/token), allowlist, filter lock, micToggle + sceneApply round trips against confirmed state, 150ms debounce, regenerate-cuts-clients — PASS
  • 1314 backend tests, clippy -D warnings, fmt, 1113 desktop tests (incl. a renderer intent-dispatch integration test), typecheck/lint/format, plugin builds
  • Protocol doc for Companion/Loupedeck/third parties: docs/remote-control.md

Plan: vault "2026-07-16 - Videorc Remote Control Stream Deck Plan" (RC0–RC5). Physical-deck by-eye = owner/franky47 acceptance; Elgato Marketplace submission is a follow-up.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added Remote Control support for compatible clients, including pairing via a local discovery file and token regeneration.
    • Added renderer UI for Remote control (enable/disable, copy token, regenerate) and for configuring OS global shortcuts.
    • Introduced a Stream Deck plugin with six remote actions plus a property inspector with live option lists.
  • Security
    • Enforced token-authenticated remote sessions with strict allowlisting and locked event boundaries; added OS-global shortcut trigger flow.
  • Documentation
    • Added complete Remote Control specification and smoke-gate guidance.
  • Tests
    • Added remote-control smoke testing and renderer integration coverage for intent handling/acks.

…ated local API (issue #143)

RC0: OS-global shortcuts (record/stream/mic) via Electron globalShortcut —
Stream Deck's native Hotkey action works today; configured in Settings,
off by default.

RC1: third backend transport role 'remote' with a HARD allowlist
(remote.describe + remote.intent only; everything else 403s), rotatable
token in the secret store, 0600 same-machine discovery file (written on
enable/startup, removed on disable/shutdown), intent validation + 150ms
per-family debounce, and a generation watch that closes live remote
sockets the moment the token rotates or the surface is disabled. Remote
sockets' event filter is LOCKED to remote.state/remote.ack at connection
setup and cannot be widened.

RC2+RC3: the backend only relays — intents execute in the renderer through
the exact same handlers as the on-screen buttons (record/stream via the
session flow with all validation, mic via audio processing, scenes via the
layout transaction, takeovers via Screens, windows via main IPC), every
intent is acked, and a minimal leak-safe state projection (recording/live/
mic/scene/takeover/windows) streams to remote clients for key rendering.

RC4: official Stream Deck plugin (apps/streamdeck-plugin, Elgato SDK v2):
six actions with state-driven titles, discovery-file pairing, reconnect
backoff. Builds in the workspace.

RC5: Settings 'Remote control' section (enable/token/copy/regenerate/
connected count) + 'Global shortcuts' section; docs/remote-control.md
protocol doc for third-party integrations; smoke:remote-control in
local-gates — proves the discovery contract, allowlist, filter lock,
intent round trips against backend-CONFIRMED state, debounce, and
regenerate-cuts-clients against the real app.

Verification: 1314 backend tests, clippy -D warnings, fmt, 1113 desktop
tests, typecheck, lint, format, plugin typecheck+build,
smoke:remote-control PASS end-to-end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 96437021-7198-4133-aed3-152a04775e3f

📥 Commits

Reviewing files that changed from the base of the PR and between 07730e8 and 41397a7.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (7)
  • apps/desktop/package.json
  • apps/desktop/src/main/index.ts
  • apps/desktop/src/renderer/src/hooks/studio-provider.integration.test.ts
  • apps/desktop/src/renderer/src/hooks/use-studio.tsx
  • apps/desktop/src/shared/backend.ts
  • crates/videorc-backend/src/main.rs
  • package.json

📝 Walkthrough

Walkthrough

This change adds a token-gated remote-control protocol with backend intent handling, renderer integration, global shortcuts, Stream Deck support, settings UI, discovery-file pairing, and smoke coverage for security, state round trips, debounce, and client invalidation.

Changes

Remote control surface

Layer / File(s) Summary
Remote runtime and intent contracts
crates/videorc-backend/src/backend_authority.rs, crates/videorc-backend/src/remote_control.rs, crates/videorc-backend/src/state.rs
Adds remote roles, token authentication, typed intents, validation, debounce tickets, persisted enablement, discovery files, and runtime state.
Backend authentication and WebSocket transport
crates/videorc-backend/src/main.rs
Adds remote authentication, method and event restrictions, discovery lifecycle, client tracking, generation-based disconnects, and remote RPC dispatch.
Desktop IPC and global shortcuts
apps/desktop/src/shared/*, apps/desktop/src/main/index.ts, apps/desktop/src/preload/index.ts, apps/desktop/src/renderer/src/lib/*
Adds secure shortcut contracts, main-process registration, renderer synchronization, preload bindings, and shortcut settings types.
Renderer remote controls and settings
apps/desktop/src/renderer/src/hooks/use-studio.tsx, apps/desktop/src/renderer/src/components/tabs/settings-tab.tsx, apps/desktop/src/renderer/src/hooks/studio-provider.integration.test.ts
Executes remote intents, publishes projected state, exposes management APIs, handles shortcuts, and adds settings and integration coverage.
Stream Deck client and actions
apps/streamdeck-plugin/src/*, apps/streamdeck-plugin/bin/*, apps/streamdeck-plugin/manifest.json, apps/streamdeck-plugin/ui/*, apps/streamdeck-plugin/package.json, apps/streamdeck-plugin/tsconfig.json
Adds discovery-based WebSocket clients, reconnect behavior, plugin packaging, six actions, property-inspector support, and client tests.
Smoke gate and protocol documentation
scripts/smoke-remote-control-app.mjs, docs/remote-control.md, package.json, AGENTS.md
Documents the protocol and adds smoke coverage for pairing, security restrictions, intent acknowledgements, confirmed state, debounce, regeneration, and cleanup.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant StreamDeck
  participant BackendWebSocket
  participant Renderer
  participant SettingsUI
  StreamDeck->>BackendWebSocket: Connect with discovery token
  BackendWebSocket->>StreamDeck: remote.describe and remote.state
  StreamDeck->>BackendWebSocket: remote.intent
  BackendWebSocket->>Renderer: remote.intent event
  Renderer->>BackendWebSocket: remote.intent.ack and remote.surface.publish
  BackendWebSocket->>StreamDeck: remote.ack and remote.state
  SettingsUI->>BackendWebSocket: remote.control.enable or regenerate
  BackendWebSocket->>StreamDeck: Close clients after generation change
Loading

Possibly related PRs

  • TheOrcDev/videorc#88: Updates the shared layout preset values used by remote sceneApply validation.

Suggested reviewers: petercr

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 52.08% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main feature set: Stream Deck support, global shortcuts, and a token-gated remote API.
Linked Issues check ✅ Passed The changes cover the requested Stream Deck controls for recording, streaming, scenes, mic, static assets, and foreground/background actions.
Out of Scope Changes check ✅ Passed No clear unrelated changes are present; the added docs, scripts, tests, and dependency align with the remote control feature.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/remote-control

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

🧹 Nitpick comments (3)
apps/streamdeck-plugin/package.json (1)

8-11: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Consider adding unit coverage for readDiscovery/defaultDiscoveryPath.

These are small pure functions in videorc-client.ts well suited for the existing test script; no test files are included in this slice.

As per coding guidelines, **/*.{ts,tsx,rs}: "prefer small pure helpers for behavior that needs tests."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/streamdeck-plugin/package.json` around lines 8 - 11, Add unit tests
under the existing test script for the pure helpers readDiscovery and
defaultDiscoveryPath in videorc-client.ts, covering their expected path and
discovery-reading behavior. Use the project’s established test conventions and
ensure the tests are included by the current src/*.test.ts pattern.

Source: Coding guidelines

apps/streamdeck-plugin/bin/plugin.js (1)

1-224: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Compiled tsc output committed alongside sources. Both files under bin/ are generated by npm run build (tsc -p tsconfig.json, outDir: "bin") but are checked into git next to their src/*.ts counterparts. This risks drift if someone edits src/ without rebuilding, and duplicates the source-level issues already flagged on plugin.ts/videorc-client.ts.

  • apps/streamdeck-plugin/bin/plugin.js#L1-L224: either gitignore bin/ and generate it as part of packaging/release, or add a CI check that fails if bin/plugin.js is stale relative to src/plugin.ts.
  • apps/streamdeck-plugin/bin/videorc-client.js#L1-L149: same treatment — gitignore or add a staleness check relative to src/videorc-client.ts.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/streamdeck-plugin/bin/plugin.js` around lines 1 - 224, Stop committing
generated TypeScript output under apps/streamdeck-plugin/bin: remove both
apps/streamdeck-plugin/bin/plugin.js (lines 1-224) and
apps/streamdeck-plugin/bin/videorc-client.js (lines 1-149) from version control,
add bin/ to the appropriate gitignore, and ensure the packaging or release
workflow runs npm run build to generate them.
apps/streamdeck-plugin/src/videorc-client.ts (1)

39-39: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Fixed 2s reconnect delay, no exponential backoff.

RECONNECT_DELAY_MS is a flat 2s retry with no backoff/jitter. For a local same-machine socket this is low risk, but if the backend is down for a long time this will hammer connect() indefinitely.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/streamdeck-plugin/src/videorc-client.ts` at line 39, Update the
reconnect scheduling around RECONNECT_DELAY_MS to use exponential backoff,
increasing the delay after each failed connection attempt and resetting it after
a successful connection. Preserve a bounded maximum delay and apply jitter so
repeated retries do not hammer connect() at a fixed interval.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/desktop/src/renderer/src/hooks/use-studio.tsx`:
- Around line 9241-9252: Add rejection handling to the setGlobalShortcuts
promise in the effect containing the shortcut registration flow, reporting the
IPC error through reportError and preserving the existing conflict toast for
partial failures. Include reportError in that effect’s dependency array so the
callback remains correctly tracked.
- Around line 9195-9210: Update the useEffect responsible for
remote.surface.publish to retry failed initial publication with bounded backoff,
ensuring remote.describe is eventually populated even without later state
changes. Preserve the existing connection guard and best-effort behavior, but
replace the silent catch in the publication flow with retry scheduling and
explicit failure diagnostics after the retry limit.
- Around line 9178-9192: Update the state projection containing notes and
preview to include comments: commentsWindow.open, and add commentsWindow.open to
the associated memo dependency array. Add a regression assertion verifying the
published comments-window state reflects whether the configured comments window
is open.

In `@apps/streamdeck-plugin/manifest.json`:
- Around line 39-59: Add the appropriate PropertyInspectorPath to the manifest
entries for scene-apply, takeover-toggle, and window-front, pointing each to the
existing inspector UI used to configure its settings. Ensure the inspector
persists layoutPreset, assetId, and window values so these actions receive their
required settings instead of remaining unset.
- Around line 1-17: Update the top-level manifest object with the plugin UUID
required for packaging, then add settings/property-inspector support for the
scene-apply, takeover-toggle, and window-front actions so users can configure
and persist layoutPreset, assetId, and window values consumed by those actions.

In `@apps/streamdeck-plugin/src/plugin.ts`:
- Around line 36-47: Update onKeyDown to await the boolean result from
client.sendIntent(intent), and call ev.action.showAlert() when it returns false.
Preserve the existing disconnected-client and missing-intent checks, and ensure
rejected backend intents no longer complete silently.
- Around line 16-48: Register the shared client event listeners only once
instead of inside onWillAppear. Add a constructor or equivalent one-time setup
in VideorcAction that subscribes refresh to the state, connected, and
disconnected events, while leaving onWillAppear responsible only for updating
the newly visible action’s title.

In `@apps/streamdeck-plugin/src/videorc-client.ts`:
- Around line 137-146: Update the message parsing in the ws 'message' callback
to reject null and non-object JSON results before accessing message.event or
message.payload. Preserve the existing behavior of returning silently for
malformed or unsupported messages, while continuing to process valid object
messages.
- Around line 108-163: Update sendIntent() to retain the generated request id
and track or otherwise associate it with the dispatched intent, then update
connect()’s message handler to validate the parsed envelope before reading event
and surface matching ok: false responses as Stream Deck feedback using the
existing client feedback mechanism. Preserve current describe/state handling and
ensure JSON null or other non-object payloads are ignored safely.

In `@crates/videorc-backend/src/main.rs`:
- Around line 4344-4348: Replace the manual connected_clients
increment/decrement around the Remote WebSocket connection flow with a
Drop-based RAII guard, following the existing WebSocketQueueMetricsInner/Weak
tracking pattern. Ensure the guard increments on creation and decrements
automatically on every exit path, including failed initial backend.ready sends,
breaks, and panics; remove the redundant manual decrement near the function end.

In `@crates/videorc-backend/src/remote_control.rs`:
- Around line 207-225: The write_discovery function currently creates the
discovery file before applying restrictive permissions, leaving a temporary
exposure window. Replace std::fs::write with an OpenOptions-based handle
configured via Unix OpenOptionsExt::mode(0o600), write the serialized body
through that handle, and retain the existing Unix permission handling while
avoiding any additional Windows ACL changes.

In `@scripts/smoke-remote-control-app.mjs`:
- Around line 16-18: Update the smoke script’s fail/error handling and cleanup
flow: make fail throw instead of calling process.exit, set process.exitCode in
the top-level catch, and ensure the temporary user-data directory is removed
after stopApp() completes. Preserve cleanup through the outer finally for both
success and failure paths, including the referenced import and shutdown
sections.

---

Nitpick comments:
In `@apps/streamdeck-plugin/bin/plugin.js`:
- Around line 1-224: Stop committing generated TypeScript output under
apps/streamdeck-plugin/bin: remove both apps/streamdeck-plugin/bin/plugin.js
(lines 1-224) and apps/streamdeck-plugin/bin/videorc-client.js (lines 1-149)
from version control, add bin/ to the appropriate gitignore, and ensure the
packaging or release workflow runs npm run build to generate them.

In `@apps/streamdeck-plugin/package.json`:
- Around line 8-11: Add unit tests under the existing test script for the pure
helpers readDiscovery and defaultDiscoveryPath in videorc-client.ts, covering
their expected path and discovery-reading behavior. Use the project’s
established test conventions and ensure the tests are included by the current
src/*.test.ts pattern.

In `@apps/streamdeck-plugin/src/videorc-client.ts`:
- Line 39: Update the reconnect scheduling around RECONNECT_DELAY_MS to use
exponential backoff, increasing the delay after each failed connection attempt
and resetting it after a successful connection. Preserve a bounded maximum delay
and apply jitter so repeated retries do not hammer connect() at a fixed
interval.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fcfe6d64-a398-4691-acbf-1649fc92d8e8

📥 Commits

Reviewing files that changed from the base of the PR and between 430f355 and d60a983.

⛔ Files ignored due to path filters (8)
  • apps/streamdeck-plugin/imgs/mic.svg is excluded by !**/*.svg
  • apps/streamdeck-plugin/imgs/plugin.svg is excluded by !**/*.svg
  • apps/streamdeck-plugin/imgs/record.svg is excluded by !**/*.svg
  • apps/streamdeck-plugin/imgs/scene.svg is excluded by !**/*.svg
  • apps/streamdeck-plugin/imgs/stream.svg is excluded by !**/*.svg
  • apps/streamdeck-plugin/imgs/takeover.svg is excluded by !**/*.svg
  • apps/streamdeck-plugin/imgs/window.svg is excluded by !**/*.svg
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (25)
  • AGENTS.md
  • apps/desktop/src/main/index.ts
  • apps/desktop/src/preload/index.ts
  • apps/desktop/src/renderer/src/components/tabs/settings-tab.tsx
  • apps/desktop/src/renderer/src/hooks/studio-provider.integration.test.ts
  • apps/desktop/src/renderer/src/hooks/use-studio.tsx
  • apps/desktop/src/renderer/src/lib/capture.ts
  • apps/desktop/src/shared/backend.ts
  • apps/desktop/src/shared/electron-ipc-contract.test.ts
  • apps/desktop/src/shared/electron-ipc-contract.ts
  • apps/desktop/src/shared/renderer-security-policy.ts
  • apps/streamdeck-plugin/bin/plugin.js
  • apps/streamdeck-plugin/bin/videorc-client.js
  • apps/streamdeck-plugin/manifest.json
  • apps/streamdeck-plugin/package.json
  • apps/streamdeck-plugin/src/plugin.ts
  • apps/streamdeck-plugin/src/videorc-client.ts
  • apps/streamdeck-plugin/tsconfig.json
  • crates/videorc-backend/src/backend_authority.rs
  • crates/videorc-backend/src/main.rs
  • crates/videorc-backend/src/remote_control.rs
  • crates/videorc-backend/src/state.rs
  • docs/remote-control.md
  • package.json
  • scripts/smoke-remote-control-app.mjs

Comment thread apps/desktop/src/renderer/src/hooks/use-studio.tsx Outdated
Comment thread apps/desktop/src/renderer/src/hooks/use-studio.tsx Outdated
Comment on lines +9241 to +9252
void window.videorc?.setGlobalShortcuts?.(shortcuts).then((result) => {
const failed = Object.entries(result?.registered ?? {})
.filter(([, ok]) => !ok)
.map(([action]) => action)
if (failed.length > 0) {
toast.error('Some global shortcuts could not be registered', {
id: 'global-shortcuts-conflict',
description:
'Another app may already use that key combination. Pick a different one in Settings.'
})
}
})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Handle shortcut-registration rejection.

If the IPC request rejects, this creates an unhandled promise rejection and leaves users without registration diagnostics. Add .catch(...) reporting and include reportError in the effect dependencies.

As per coding guidelines, “Prefer explicit status and diagnostics over silent fallbacks.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/renderer/src/hooks/use-studio.tsx` around lines 9241 - 9252,
Add rejection handling to the setGlobalShortcuts promise in the effect
containing the shortcut registration flow, reporting the IPC error through
reportError and preserving the existing conflict toast for partial failures.
Include reportError in that effect’s dependency array so the callback remains
correctly tracked.

Source: Coding guidelines

Comment thread apps/streamdeck-plugin/manifest.json
Comment thread apps/streamdeck-plugin/manifest.json
Comment thread apps/streamdeck-plugin/src/videorc-client.ts Outdated
Comment thread apps/streamdeck-plugin/src/videorc-client.ts
Comment thread crates/videorc-backend/src/main.rs Outdated
Comment thread crates/videorc-backend/src/remote_control.rs
Comment thread scripts/smoke-remote-control-app.mjs Outdated

@franky47 franky47 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So many useEffects 😭

Q: does the takeover screens mute the mic too? Generally my BRB scene is only one splash screen in OBS, no cam no mic.

: intent.kind === 'micMute'
}
}))
return void (await ack(true))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: use Immer here

if (status) setRemoteStatus(status)
})
.finally(() => setRemotePending(false))
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: eww

…r (PR #150 review)

franky47's review + 12 CodeRabbit findings:

Renderer — the remote-control code now uses NO useEffect:
- Intent executor and global-shortcut handler are React 19.2 effect events
  (stable identity, latest closure), subscribed once in the existing
  client-setup effect; the intent-handler ref and its sync effect are gone.
- The state projection is pushed by a RemoteSurfacePublisher the render
  body hands the latest snapshot (same latest-value pattern as the hook's
  render-synced refs); it dedupes, debounces past the commit, republishes
  on reconnect, and retries failed publishes with bounded backoff — a
  failed FIRST publish no longer leaves remote.describe empty forever.
- Global-shortcut registration is a render-synced registrar that dedupes
  by value and reports IPC rejection instead of dropping it.
- The projection now includes the comments window; describe rides the same
  snapshot.
- Settings tab no longer polls: the backend pushes remote.control.status
  (enable/disable/regenerate + client connect/disconnect) into studio
  context; the tab renders it and fires actions.
- Nested captureConfig mic mutations use Immer.

Backend:
- Remote client counting is a Drop guard — the early return on a failed
  initial ready send leaked connected_clients forever (phantom deck in
  Settings). Connect/disconnect emit remote.control.status.
- Discovery file is born 0600 via OpenOptionsExt (no create-then-chmod
  umask window).

Stream Deck plugin:
- manifest gains the top-level UUID; scene/takeover/window actions get a
  dependency-free property inspector (ui/property-inspector.html) whose
  options populate live from remote.describe via sendToPlugin — these
  three actions were unconfigurable before.
- Shared-client listeners register once in the action constructor instead
  of stacking on every onWillAppear.
- sendIntent returns end-to-end truth (admission response + renderer's
  remote.ack, 5s timeout) and key presses showAlert on failure; message
  envelopes are guarded against non-object JSON.
- readDiscovery/defaultDiscoveryPath unit tests.

Smoke: fail() throws so the launched app always stops; exitCode instead of
process.exit; temp user-data dir removed in finally.

Gates: cargo 1314 green + clippy -D warnings, desktop 1113 green,
typecheck/lint/format, scripts 647 green, smoke:remote-control PASS
end-to-end against the dev app.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@TheOrcDev

Copy link
Copy Markdown
Owner Author

All review changes are in (07730e8).

@franky47 — useEffects: all gone. The remote-control code now adds zero useEffects:

  • The intent executor and global-shortcut handler are React 19.2 useEffectEvents — stable identity, latest closure — subscribed once in the already-existing client-setup effect. The handler-ref + ref-sync effect are gone.
  • The state projection is pushed by a small imperative RemoteSurfacePublisher (lib/remote-surface.ts): the render body hands it the latest snapshot (the same latest-value pattern the hook already uses for its render-synced refs), and it owns change-detection, post-commit debounce, republish-on-reconnect, and bounded retry. Same idea for global-shortcut registration (lib/global-shortcuts.ts).
  • The Settings tab no longer polls at all — the backend now pushes remote.control.status (on enable/disable/regenerate and on every deck connect/disconnect), so the tab just renders context state and fires actions. That also removed the block you eww'd, and the connected-clients count is now live.
  • Immer is in for the nested captureConfig mic mutations, as suggested.

Your question — does a takeover mute the mic? No — takeovers are visual-only today: the camera/screen composition is replaced by the splash, but the mic keeps its live state (deliberate default: "BRB while I answer the door" often still wants you audible until you choose otherwise, and muting is one key away). For your OBS-style BRB workflow I'd add an opt-in "Mute mic while a takeover is active" setting (auto-restore on hide) as a follow-up — happy to file/build that if you want it.

CodeRabbit findings, all 12 addressed:

  • comments window added to the projection; publish failures now retry with bounded backoff (a failed first publish can no longer leave remote.describe empty forever); shortcut IPC rejection is reported.
  • Plugin: top-level manifest UUID; the three setting-backed actions got a dependency-free property inspector whose dropdowns populate live from remote.describe (they were genuinely unconfigurable before — good catch); shared-client listeners register once in the constructor; sendIntent now returns end-to-end truth (admission response + the renderer's remote.ack, 5s timeout) and failed presses showAlert; message envelopes are guarded against non-object JSON; readDiscovery unit tests added.
  • Backend: remote client counting is a Drop guard (the early-return on a failed initial ready send leaked connected_clients → phantom deck in Settings); the discovery file is born 0600 via OpenOptionsExt instead of create-then-chmod.
  • Smoke: fail() throws so the launched app always stops, process.exitCode instead of process.exit, temp user-data dir removed in finally.

Gates: cargo 1314 green + clippy -D warnings, desktop 1113 green, TS gates, scripts 647 green, and smoke:remote-control passes end-to-end against the real dev app with all of the above.

🤖 Generated with Claude Code

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (4)
apps/streamdeck-plugin/bin/videorc-client.js (1)

71-79: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Synchronously tear down connection state and settle pending intents.

Setting this.ws = null immediately after close() prevents the asynchronous onGone handler from executing its body (since this.ws === ws evaluates to false). As a result, pending requests will hang until they time out, and the disconnected event is not emitted.

Execute the teardown directly in stop() to guarantee clean lifecycle termination.

🐛 Proposed fix
     stop() {
         this.stopped = true;
         if (this.reconnectTimer) {
             clearTimeout(this.reconnectTimer);
             this.reconnectTimer = null;
         }
-        this.ws?.close();
-        this.ws = null;
+        if (this.ws) {
+            this.ws.close();
+            this.ws = null;
+        }
+        this.settleAllPending(false);
+        if (this.connected) {
+            this.connected = false;
+            this.emit('disconnected');
+        }
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/streamdeck-plugin/bin/videorc-client.js` around lines 71 - 79, Update
stop() to synchronously perform the same teardown and pending-intent settlement
normally handled by the websocket onGone path before clearing this.ws. Ensure
pending requests are settled and the disconnected event is emitted, then close
the socket and clear connection state without allowing the asynchronous onGone
handler to skip cleanup because this.ws was nulled first.
apps/desktop/src/renderer/src/components/tabs/settings-tab.tsx (1)

377-379: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Make the remote-control copy platform-neutral.

“This Mac” and Cmd+… examples are wrong on Windows/Linux and can guide users toward unusable bindings. Use “this computer” and derive Cmd/Ctrl examples from runtimeInfo.platform.

Also applies to: 426-437

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/renderer/src/components/tabs/settings-tab.tsx` around lines
377 - 379, Update the Remote control copy in the settings tab to use “this
computer” instead of “this Mac,” and make all shortcut examples in the related
content platform-aware by deriving Cmd for macOS and Ctrl for Windows/Linux from
runtimeInfo.platform. Ensure the displayed bindings consistently use the derived
modifier.
apps/streamdeck-plugin/src/videorc-client.ts (2)

105-113: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Fully clean up admitted intents on timeout and stop.

Once a request moves to pendingAcks, its timeout callback only deletes pendingRequests, leaving an unbounded stale pendingAcks entry if no ACK arrives. Additionally, stop() nulls this.ws before onGone, so pending intents and connected state are not cleaned immediately.

Delete the matching ACK entry inside settle(), and have stop() call settleAllPending(false) and transition connection state explicitly. Add timeout and stop lifecycle tests.

Also applies to: 127-152, 209-215

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/streamdeck-plugin/src/videorc-client.ts` around lines 105 - 113, Update
settle() to remove the corresponding entry from pendingAcks whenever a request
is resolved or timed out, preventing stale admitted intents. In stop(), invoke
settleAllPending(false) and explicitly transition the connection to disconnected
before or while closing the socket, rather than relying on onGone after nulling
ws. Add lifecycle tests covering timeout cleanup and stop cleanup, including
pending intents and connected state.

58-72: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Validate discovery fields before constructing the WebSocket URL.
readDiscovery() only checks truthiness, so wrong-type values like { port: "bad", token: [] } can still reach connection setup. Require a 1..65535 integer port, a non-empty string token, and a string host; add wrong-type tests.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/streamdeck-plugin/src/videorc-client.ts` around lines 58 - 72, Update
readDiscovery to validate parsed.host, parsed.port, and parsed.token before
returning a Discovery: require host to be a string, token to be a non-empty
string, and port to be an integer in the 1–65535 range. Reject invalid values
before WebSocket URL construction, preserve valid defaults where applicable, and
add tests covering wrong-type fields.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/desktop/src/renderer/src/components/tabs/settings-tab.tsx`:
- Around line 119-132: Update the remote-control switch rendering and action
flow around remoteStatus and runRemoteAction so the switch is disabled whenever
remoteStatus is null, including while status is unavailable or reconnecting. Add
an explicit unavailable/reconnecting status message for that state, while
preserving the existing pending-state behavior and normal actions once the
backend has seeded a status.

In `@apps/desktop/src/renderer/src/lib/global-shortcuts.ts`:
- Around line 22-35: Update GlobalShortcuts.sync to compare against the
currently pending shortcuts as well as lastRegisteredBody, cancel any scheduled
registration when the requested configuration matches the registered state, and
replace the zero-delay timer with a debounce delay so rapid edits produce one
registration. Ensure the timer is cleared/reset when cancelling stale pending
work, while register continues applying the latest pending configuration.

In `@apps/desktop/src/renderer/src/lib/remote-surface.ts`:
- Around line 58-81: Update the RemoteSurface connection lifecycle around
attach, markConnected, and detach to maintain a generation counter, incrementing
it on each call. Capture the generation when publishing, then after each await
ignore completions from obsolete generations before scheduling retries or
updating lastPublishedBody, so stale connections cannot affect deduplication for
the current connection.

In `@apps/streamdeck-plugin/bin/videorc-client.js`:
- Around line 94-98: Update the settle function to remove the corresponding
requestId entry from pendingAcks in addition to pendingRequests, ensuring both
maps are cleaned up when an intent resolves or times out.

---

Outside diff comments:
In `@apps/desktop/src/renderer/src/components/tabs/settings-tab.tsx`:
- Around line 377-379: Update the Remote control copy in the settings tab to use
“this computer” instead of “this Mac,” and make all shortcut examples in the
related content platform-aware by deriving Cmd for macOS and Ctrl for
Windows/Linux from runtimeInfo.platform. Ensure the displayed bindings
consistently use the derived modifier.

In `@apps/streamdeck-plugin/bin/videorc-client.js`:
- Around line 71-79: Update stop() to synchronously perform the same teardown
and pending-intent settlement normally handled by the websocket onGone path
before clearing this.ws. Ensure pending requests are settled and the
disconnected event is emitted, then close the socket and clear connection state
without allowing the asynchronous onGone handler to skip cleanup because this.ws
was nulled first.

In `@apps/streamdeck-plugin/src/videorc-client.ts`:
- Around line 105-113: Update settle() to remove the corresponding entry from
pendingAcks whenever a request is resolved or timed out, preventing stale
admitted intents. In stop(), invoke settleAllPending(false) and explicitly
transition the connection to disconnected before or while closing the socket,
rather than relying on onGone after nulling ws. Add lifecycle tests covering
timeout cleanup and stop cleanup, including pending intents and connected state.
- Around line 58-72: Update readDiscovery to validate parsed.host, parsed.port,
and parsed.token before returning a Discovery: require host to be a string,
token to be a non-empty string, and port to be an integer in the 1–65535 range.
Reject invalid values before WebSocket URL construction, preserve valid defaults
where applicable, and add tests covering wrong-type fields.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9ee533fd-75d7-4712-b111-7991e6789c1e

📥 Commits

Reviewing files that changed from the base of the PR and between d60a983 and 07730e8.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (16)
  • apps/desktop/package.json
  • apps/desktop/src/renderer/src/components/tabs/settings-tab.tsx
  • apps/desktop/src/renderer/src/hooks/use-studio.tsx
  • apps/desktop/src/renderer/src/lib/global-shortcuts.ts
  • apps/desktop/src/renderer/src/lib/remote-surface.ts
  • apps/streamdeck-plugin/bin/plugin.js
  • apps/streamdeck-plugin/bin/videorc-client.js
  • apps/streamdeck-plugin/manifest.json
  • apps/streamdeck-plugin/src/plugin.ts
  • apps/streamdeck-plugin/src/videorc-client.test.ts
  • apps/streamdeck-plugin/src/videorc-client.ts
  • apps/streamdeck-plugin/ui/property-inspector.html
  • crates/videorc-backend/src/main.rs
  • crates/videorc-backend/src/remote_control.rs
  • docs/remote-control.md
  • scripts/smoke-remote-control-app.mjs
🚧 Files skipped from review as they are similar to previous changes (7)
  • docs/remote-control.md
  • apps/streamdeck-plugin/manifest.json
  • apps/streamdeck-plugin/bin/plugin.js
  • crates/videorc-backend/src/remote_control.rs
  • apps/streamdeck-plugin/src/plugin.ts
  • scripts/smoke-remote-control-app.mjs
  • crates/videorc-backend/src/main.rs

Comment on lines +119 to +132
// Remote-control status is pushed into the studio context by the backend
// (remote.control.status events) — this tab only renders it and fires
// actions; the switch settles when the backend confirms.
const remoteStatus = remoteControl.status
const [remotePending, setRemotePending] = useState(false)
const runRemoteAction = async (
action: () => Promise<RemoteControlStatus | null>
): Promise<void> => {
setRemotePending(true)
try {
await action()
} finally {
setRemotePending(false)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Disable the switch while remote status is unavailable.

When remoteStatus is null, remoteControlRequest() can return null, making the enabled switch silently do nothing. Disable it until status is seeded and show an unavailable/reconnecting message.

As per coding guidelines, “Prefer explicit status and diagnostics over silent fallbacks.”

Also applies to: 366-375

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/renderer/src/components/tabs/settings-tab.tsx` around lines
119 - 132, Update the remote-control switch rendering and action flow around
remoteStatus and runRemoteAction so the switch is disabled whenever remoteStatus
is null, including while status is unavailable or reconnecting. Add an explicit
unavailable/reconnecting status message for that state, while preserving the
existing pending-state behavior and normal actions once the backend has seeded a
status.

Source: Coding guidelines

Comment on lines +22 to +35
sync(shortcuts: GlobalShortcutsConfig): void {
const body = JSON.stringify(shortcuts)
if (body === this.lastRegisteredBody) {
return
}
this.pending = shortcuts
if (this.timer) {
return
}
this.timer = setTimeout(() => {
this.timer = null
this.register()
}, 0)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Cancel stale registrations and debounce actual edits.

After A → B → A, the second A returns early while pending B remains scheduled, so the OS can register a shortcut no longer shown in Settings. Also, setTimeout(..., 0) registers every separately typed character.

Proposed fix
   sync(shortcuts: GlobalShortcutsConfig): void {
     const body = JSON.stringify(shortcuts)
     if (body === this.lastRegisteredBody) {
+      this.pending = null
+      if (this.timer) {
+        clearTimeout(this.timer)
+        this.timer = null
+      }
       return
     }
     this.pending = shortcuts
     if (this.timer) {
-      return
+      clearTimeout(this.timer)
     }
     this.timer = setTimeout(() => {
       this.timer = null
       this.register()
-    }, 0)
+    }, 250)
   }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
sync(shortcuts: GlobalShortcutsConfig): void {
const body = JSON.stringify(shortcuts)
if (body === this.lastRegisteredBody) {
return
}
this.pending = shortcuts
if (this.timer) {
return
}
this.timer = setTimeout(() => {
this.timer = null
this.register()
}, 0)
}
sync(shortcuts: GlobalShortcutsConfig): void {
const body = JSON.stringify(shortcuts)
if (body === this.lastRegisteredBody) {
this.pending = null
if (this.timer) {
clearTimeout(this.timer)
this.timer = null
}
return
}
this.pending = shortcuts
if (this.timer) {
clearTimeout(this.timer)
}
this.timer = setTimeout(() => {
this.timer = null
this.register()
}, 250)
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/renderer/src/lib/global-shortcuts.ts` around lines 22 - 35,
Update GlobalShortcuts.sync to compare against the currently pending shortcuts
as well as lastRegisteredBody, cancel any scheduled registration when the
requested configuration matches the registered state, and replace the zero-delay
timer with a debounce delay so rapid edits produce one registration. Ensure the
timer is cleared/reset when cancelling stale pending work, while register
continues applying the latest pending configuration.

Comment on lines +58 to +81
attach(client: RemoteSurfaceClient): void {
this.client = client
this.connected = false
}

/** The socket is live: the backend's surface slate is blank, so the last
* published body no longer counts — republish immediately. */
markConnected(): void {
this.connected = true
this.lastPublishedBody = null
this.retryAttempt = 0
this.schedule(0)
}

detach(): void {
this.client = null
this.connected = false
this.lastPublishedBody = null
this.retryAttempt = 0
if (this.timer) {
clearTimeout(this.timer)
this.timer = null
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Ignore publication completions from obsolete connections.

An old in-flight publish can resolve after reconnect, restore lastPublishedBody, and make the new connection skip its initial publish. Track a generation incremented by attach, markConnected, and detach; verify it after await and before retrying or updating dedupe state.

Proposed fix
 export class RemoteSurfacePublisher {
+  private generation = 0

   attach(client: RemoteSurfaceClient): void {
+    this.generation += 1
     this.client = client
     this.connected = false
   }

   markConnected(): void {
+    this.generation += 1
     this.connected = true
     ...
   }

   detach(): void {
+    this.generation += 1
     ...
   }

   private async flush(): Promise<void> {
+    const generation = this.generation
     const { client, latest } = this
     ...
       await client.request('remote.surface.publish', latest)
+      if (generation !== this.generation || client !== this.client || !this.connected) return
       this.lastPublishedBody = body
     } catch (error) {
+      if (generation !== this.generation || client !== this.client) return
       ...

Also applies to: 107-130

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/renderer/src/lib/remote-surface.ts` around lines 58 - 81,
Update the RemoteSurface connection lifecycle around attach, markConnected, and
detach to maintain a generation counter, incrementing it on each call. Capture
the generation when publishing, then after each await ignore completions from
obsolete generations before scheduling retries or updating lastPublishedBody, so
stale connections cannot affect deduplication for the current connection.

Comment on lines +94 to +98
const settle = (ok) => {
clearTimeout(timer);
this.pendingRequests.delete(requestId);
resolve(ok);
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Prevent closure leak in pendingAcks on intent timeout.

When an admitted intent times out waiting for a remote.ack, the timer calls settle(false). Currently, settle removes the request from pendingRequests but leaves it in pendingAcks, permanently leaking the closure and inflating the map over time.

Find and remove the entry from pendingAcks inside settle to properly clean up timed-out intents.

♻️ Proposed fix
-            const settle = (ok) => {
-                clearTimeout(timer);
-                this.pendingRequests.delete(requestId);
-                resolve(ok);
-            };
+            const settle = (ok) => {
+                clearTimeout(timer);
+                this.pendingRequests.delete(requestId);
+                for (const [id, s] of this.pendingAcks) {
+                    if (s === settle) {
+                        this.pendingAcks.delete(id);
+                        break;
+                    }
+                }
+                resolve(ok);
+            };
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const settle = (ok) => {
clearTimeout(timer);
this.pendingRequests.delete(requestId);
resolve(ok);
};
const settle = (ok) => {
clearTimeout(timer);
this.pendingRequests.delete(requestId);
for (const [id, s] of this.pendingAcks) {
if (s === settle) {
this.pendingAcks.delete(id);
break;
}
}
resolve(ok);
};
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/streamdeck-plugin/bin/videorc-client.js` around lines 94 - 98, Update
the settle function to remove the corresponding requestId entry from pendingAcks
in addition to pendingRequests, ensuring both maps are cleaned up when an intent
resolves or times out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[feature] Remote control (stream deck support)

2 participants