Skip to content

About

Shared browser recording protocol, extension snapshot and conformance vectors for Cassini apps

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Cassini browser recording protocol

Cassini records online meetings with each participant on their own audio track. The only place where those tracks still exist separately is inside the browser: once the audio reaches the speakers it is one mix, and the operating system cannot tell who said what. So the recording has to start in the browser. Separation still depends on the sources the meeting exposes: a mixed receiver remains mixed, and a track is not a verified participant.

This repository is the code that does that, shared between Cassini for GNOME and Cassini for Mac (Windows later). Each app has its own native recorder, written in Python and Swift respectively. What they have in common is the browser side, and the agreement on how browser and recorder talk to each other. That common part lives here, so there is one copy of it instead of two drifting ones.

how it works

  • extension/ is a Chromium extension (Manifest V3). On a meeting page it hooks RTCPeerConnection early, finds the audio tracks the page sends and receives, and turns each one into timestamped PCM with an AudioWorklet. When the page hides its tracks it falls back to recording the whole tab through chrome.tabCapture.
  • The extension streams that audio to a small local program through Chrome native messaging (length-prefixed JSON on stdin/stdout, nothing on the network). Each app ships its own such program; both answer to the name io.codemyriad.cassini.
  • SPEC.md is the contract between the two: the messages, their types and limits, how timestamps and gaps are handled, and the on-disk result (a folder with one WAV per source and a session.json). schemas/ holds the same thing as JSON Schema.
  • conformance/ keeps the two implementations honest. vectors.json is a list of raw request sequences with the responses and files each one must produce; run.py pipes them into any host executable. A host that passes the vectors works with the extension. Error messages are deliberately not part of the contract, only error codes.
  • scenarios/ has the audio helpers the browser tests in both apps share (tone generation and spectral power checks), so a test that proves "the 880 Hz source did not bleed into the 440 Hz one" reads the same on both platforms. It also holds the shared test 2 driver (live-folder.mjs, with folder-clock.mjs, native-launcher.mjs and control-relay.mjs), which each app runs against its real writer; see TESTING.md.

A few things that are easy to get wrong, and that the spec and vectors pin down: a track is a source, not a person (Google Meet, for one, multiplexes speakers over three receiver slots); timestamps come from the audio render clock, not from when a message arrived; a recording that gets interrupted must still leave playable files behind.

how the apps use it

Each app vendors this repository at a pinned commit under browser-protocol/ and keeps its installed extension byte-identical to extension/. verify.py checks that:

python3 browser-protocol/verify.py --pinned BrowserExtension     # Mac
python3 browser-protocol/verify.py --pinned browser-extension    # GNOME

Install the test-only Python dependencies, then run the request/output vectors against a host:

python3 -m pip install -r requirements-test.txt
python3 conformance/run.py /absolute/path/to/native-host

The host must speak native-messaging stdio and answer start with the folder it created. Every case runs the host with a fresh private Library: the runner sets CASSINI_LIBRARY_DIR to a new temporary folder (and HOME/XDG directories beside it). Hosts MUST honour that test-only override when it is set, also when they normally resolve the Library from the system user record, where HOME cannot redirect them. Products never set it. A host that answers start with a folder outside that Library stops the whole run, and the stray folder is reported, not deleted. The runner removes only folders it caused to exist; on failure it keeps them and prints their paths. The unit tests for the extension's worklet and clock calibration are plain Node:

node --test extension/tests/*.test.cjs

Browser-level tests cover real Chromium, WebRTC, native messaging, and a required real Nextcloud Talk stack. Apps additionally test their platform-specific native recorders and devices. TESTING.md describes the layers and what each one does and does not prove.

browser companion popup

The extension popup reports the active tab’s meeting-source readiness separately from its app-control connection. Its Start recording / Stop recording button asks the connected Cassini app (popup-record-v1), which records through its own control command; the popup explains why the button is unavailable when the app is missing, too old, or the tab is not ready. A detected source is not a verified participant or proof of saved audio. A missing current-tab health row means no meeting is reported, not proof that page injection succeeded or failed.

Reload guidance targets only the current tab, warns that reloading interrupts the call, and respects the background worker’s refusal to reload active captures. Connection details can be copied without meeting titles, URLs or recording paths. The packaged-file self-check remains diagnostic and never gates page readiness.

Run node tests/popup.mjs for the popup’s isolated Chromium interaction checks; node --test extension/tests/popup-state.test.cjs covers its evidence-to-status mapping, and node --test extension/tests/popup-record.test.cjs the app ask. Browser fixtures initiate capture through the trusted direct popup RPC; the product popup never uses it. The popup does not expose app auto-record policy, confirmed saved audio, or an app launcher: those need explicit app-backed APIs instead of inferred or simulated state.

brand and shared copy (D-976)

The popup header is the cassini.talk Saturn (saturn-nav, 26 px) beside the lowercase wordmark (cassini-ux L-04, L-15). Every colour is a cassini-ux brand role: extension/brand-tokens.css is generated from the vendored ux/brand/tokens.json by python3 tools/brand-tokens.py --write, light on :root and dark under prefers-color-scheme: dark; the popup holds no colour literal. Start and Stop use the Record red (L-20), secondary buttons ink at 10% (L-54). The subtitle and the privacy line are the cassini-ux catalog's settings.tab.browserCompanion and companion.privacy, verbatim.

ux/UPSTREAM.json pins the vendored cassini-ux files (tokens, saturn-nav.svg, strings/strings.json) by commit and SHA-256. Never edit them here: change cassini-ux, copy the files at the new commit, update the pin, regenerate the CSS, and run python3 -m unittest tests.test_brand_alignment.

changing things

Changes to the protocol, the schemas, the vectors or the extension land here first, with a vector for any new rule. Then each app bumps its pinned commit. A host change that makes a vector fail is a protocol change, not a bug fix, and gets discussed as one. The wire version stays at 1 as long as additions are optional and old hosts keep working; SPEC.md has the changelog.

Most of this code was written by coding agents (one per app, plus a reviewer), steered and read by a human but not proof-read line by line. The vectors and the browser tests are the evidence that it works; a gap in them is a bug worth reporting.

SPEC v1.1 and native sessions

SPEC v1.1 amends wire/session version 1 additively. Both native recorders use the same one-WAV-per-source folder: source: "macos" and source: "gnome", with microphone and system/application audio kept separate. conformance/native-session/ contains complete synthetic WAV files and both native manifest variants, checked by python3 conformance/check-native-fixture.py.

Chromium is the current target, with Chrome 116 as the minimum. Product priorities are Teams web, Zoom web, then Meet. products/ contains account-based manual checklists and a result template. tests/product-topologies.mjs tests controlled mixed-receiver, PC-free and switching-slot models; their results do not establish live product/version compatibility. Playwright is pinned to 1.61.1. npm ci, npx playwright install chromium, and npm test run the unit/topology checks. Firefox and Flatpak remain future work.

updating a vendored copy

After staging upstream edits, python3 refresh-hashes.py refreshes SHA256SUMS. Validation, commit and push happen before either app updates. From this clone:

python3 vendor.py FULL_COMMIT_SHA /path/to/app/browser-protocol /path/to/app/BrowserExtension

GNOME's installed path is browser-extension. The vendor tool records PINNED_COMMIT and UPSTREAM_COMMIT, the raw upstream Git commit object. verify.py --pinned reconstructs the commit SHA and full Git tree from the vendored files, including SHA256SUMS. No private-repo network credentials are needed in app CI. A changed pin or a locally edited file plus an updated checksum fails this check. The proof files are app metadata, not upstream sources.

The Mac repository owns the approved DrMeyers native runner for manual/nightly jobs only. Runner registration and synthetic fixtures do not grant macOS audio consent or replace hardware validation. Live product testing is manual and requires test accounts.

The runner validates complete response/session schemas, derives track metadata and exact PCM (including gap silence) from accepted requests, checks WAV fmt/sample bytes and returned paths, and requires error termination without waiting for EOF. Corrupt-host negative tests are a mandatory CI gate. Vector counts alone from older snapshots are framing smoke evidence.

The shared npm test also loads the extension and kills its worker after five seconds of saved generated audio, in Separate tracks and Whole tab modes. It checks retained tone, interruption/path, cleared badge and closed offscreen capture. CI uses an isolated PulseAudio null sink and a small checked-in browser-test writer (not either production recorder). Either app runs the same adapter against its real host with CASSINI_NATIVE_HOST=/absolute/executable CASSINI_TEST_EXTENSION=/absolute/extension node tests/worker-recovery.mjs. CASSINI_WORKER_TEST_MODES=tracks runs the device-independent portion; Whole tab needs an output device and stays on DrMeyers rather than hosted macOS. The Mac loaded-extension and tab adapters also call scenarios/worker-recovery.mjs; GNOME adopts this adapter with its pin.

App/browser management is a separate persistent host, io.codemyriad.cassini.control; see SPEC-CONTROL.md. Its desktop-free VM corpus is node conformance/control/run-extension.cjs; a real control-host test launcher can run python3 conformance/control/run.py /absolute/host-launcher against isolated test IPC. The audio protocol/corpus remain unchanged. App-side discovery uses counts without recording; GNOME owns page discovery, while background/relay and management transport are shared.

Licence

Copyright © 2026 Codemyriad OÜ and contributors.

This repository (the protocol, the browser extension and the conformance files) is free software under the GNU General Public License, version 3 or later (GPL-3.0-or-later); see LICENSE. Cassini for GNOME and Cassini for Mac, which vendor these files, use the same licence.

Generic interactive-media evidence (companion 0.2.26)

Verified automatic admission uses live received RTC progress plus a live page-captured microphone actually sent on the same peer connection, with current document/call binding. This describes interactive media, not proven answered human intent: echo, microphone tests and early media can qualify. Passive viewing, media playback, preview-only capture, synthetic senders and unrelated/cross-frame peers do not. Listen-only, stale or unbound observations stay unknown; manual recording remains. D80.5 binds the cohort to one current browser document/frame and relay port: fresh page-captured sending and remote received progress may be on different PCs. The real split-PC Talk flow must publish acknowledged active evidence in CI. A microphone the page processes through WebAudio (Microsoft Teams) or copies with MediaStream.clone() still counts as page-captured (companion 0.2.28). A microphone preview, synthetic sender, passive viewer, or combination across documents/frames cannot qualify; unrelated split PCs within one document/frame are intentional interactive-media positives, like echo.

Apps must acknowledge both call-signals-v1 and verified-auto-admission-v1 and adopt SPEC-CONTROL.md's freshness/identity rules. Heartbeat is transport health, not media proof. Copy diagnostics shares the popup's content-free evidence/capability/transport model and excludes URLs, titles, identifiers, paths and media. Protocol publication does not make old apps adopt the rule; GNOME/Mac adoption is a separate change.

About

Shared browser recording protocol, extension snapshot and conformance vectors for Cassini apps

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages