You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8ed20ee
Browse filesBrowse the repository at this point in the historyBrowse files
feat(npm): make @deepgram/cli the real Deepgram CLI (#146)
## Why
`@deepgram/cli` on npm is the obvious name for "the Deepgram CLI", and
it's broken:
- `npm i @deepgram/cli` fails with `notarget`. The only versions are two
old betas (`0.3.0-beta.0`, `0.4.0-beta.0`) from
`lukeocodes/deepgram-cli`.
- `npm i @deepgram/cli@0.4.0-beta.0` installs, then crashes on start:
`MODULE_NOT_FOUND …/src/hooks/init/log`.
Coding agents try npm first. In our build-journey evals, agents asked to
"use the Deepgram CLI" went to `@deepgram/cli`, hit both failures, and
gave up on the CLI.
This PR makes `@deepgram/cli` the real CLI.
## What it does
**New `npm/` package.** `@deepgram/cli` has no dependencies and no
install scripts. It provides `dg` and `deepgram`, and runs the Python
`deepctl` at the **same version as the npm package**:
1. A cached private venv (`~/.cache/deepgram-cli/venvs/<V>`) if one is
already built. This is fast and works offline.
2. Otherwise, with uv present, a venv built with `uv pip install
--require-hashes --no-deps --only-binary :all:` against the hash-pinned
`requirements-<V>.txt` the package ships. No local Python is needed.
3. Otherwise, a Python 3.10+ with `venv` builds the same hash-checked
venv the same way, in place under a lock. All bootstrap output goes to
stderr, so `dg -o json` and `dg mcp` stdout stay clean on the first run.
4. Otherwise, a clear error with the three fixes: install uv, install
`python3-venv`, or use `install.sh`/brew.
`DEEPCTL_NPM_RUNNER=uvx|venv|path` forces a strategy. `path` uses a
`deepctl` already on PATH, and only when asked.
**Python side.** `InstallMethod.NPM` is detected through
`DEEPCTL_LAUNCHER`, which `deepctl_core._launcher` reads once and
removes from the environment so it doesn't leak to child processes. With
it:
- `dg update` and the update banner tell npm and npx users the npm
command.
- Plugins go into the isolated plugin venv, so they survive npm version
bumps.
- The skills prompt says `dg`.
**Release.**
- release-please bumps `npm/package.json` along with the root package
(`extra-files`).
- A new `publish-npm` job runs after `verify-published`, so npm never
points at a version PyPI can't install.
- It uses npm trusted publishing (OIDC, environment `npm-publish`, npm
pinned to 11.21.0) and sets provenance.
- It's safe to re-run: it skips a version that's already published, and
it fails closed if the dist-tag lookup errors.
- If the version is lower than the current `latest`, it publishes under
`v<maj>.<min>-maint` instead of moving `latest`.
- Recovery is "Re-run failed jobs".
**CI.** Three new jobs, added to `all-checks`:
- `launcher` (Linux, macOS, Windows): Node unit tests, `cmp LICENSE
npm/LICENSE`, and a check that stdin stays blocking for deepctl.
- `launcher-node` (Linux): Node unit tests on every version
`package.json`'s `engines` admits (20.0.0, 20, 22, 24).
- `launcher-hashed` (Windows, macOS): the release path end to end
against a local package index — builds this commit's packages, writes
the hash-locked `requirements-<V>.txt`, and runs the launcher through
both uv and the pip fallback.
- A `release-bump-gate` status (`scripts/check_release_bumps.py`) fails
the build if a package's shipped code changed since its last release tag
without a version bump — the drift that would otherwise let a sibling
package ship against a `deepctl-core` PyPI hasn't released yet. It runs
again before publish with the same check.
**Docs.** npm/npx install lines in `README.md` and
`web/src/data/llms-full.txt`. The latter also fixes the
`deepgram/tap/deepctl` → `deepgram/tap/deepgram` typo.
The first npm version will be **0.4.1**, from the next release PR.
`0.4.1` is newer than `0.4.0-beta.0`, so `latest` moves and plain `npm i
@deepgram/cli` works.
## Hardening
This went through five rounds of independent review, each followed by a
fix round. Every finding was reproduced in Docker before it was fixed,
and each fix was checked against its repro. Highlights:
**Cache security.**
- Every cache ancestor must be owned by you or root.
- Group/other-writable ancestors must be sticky.
- Group-write is never trusted, even for a user-private group (`gid ==
uid`); a shared-group cache falls back to a private temp dir.
- The `/tmp` fallback is `0700` and owner-checked.
These checks block another user from planting or swapping a cached venv.
Tested by pre-creating the cache, with a symlink, and with a
150-iteration swap loop.
**Locking.**
- `mkdir` lock with owner, pid and host.
- A lock is stale when its pid is dead, it's older than 60 min, or it
was taken by our own pid after a pid reuse.
- Stale locks are renamed aside and their owner verified before
deletion.
- Release is ownership-checked.
**Signals and stdio.**
- SIGTERM and SIGHUP are forwarded.
- With a terminal attached, SIGINT and SIGQUIT come from the process
group, so each Ctrl-C reaches deepctl exactly once and cancel still
exits 2.
- With no terminal (IDEs, agent hosts), a SIGINT sent only to the
launcher is forwarded after 1s.
- The launcher never constructs a Node std stream. That stream would put
the inherited stdin into non-blocking mode and break `dg mcp` over
stdio, `ffmpeg | dg listen`, and similar pipes.
- Launcher messages to stderr are best effort, so a full or closed
stderr can't break a run.
**Self-repair and pruning.**
- A venv whose base Python was removed gets rebuilt.
- Old version venvs are pruned only after 30 days unused, and never
while locked or in use.
## Testing
**Docker** (linux/arm64; the uvx, venv, signal and stdin cases also on
amd64): 43/43 cases pass. They cover:
- no Python;
- uv only;
- Python 3.12 venv: cold, warm, and offline warm and cold;
- Debian with and without `python3-venv` (PEP 668);
- Python 3.9 rejected;
- Alpine/musl;
- two concurrent first runs;
- SIGTERM, SIGINT and SIGHUP, with and without a TTY;
- unwritable HOME and an arbitrary uid;
- OpenShift-style uid with gid 0;
- shared primary group;
- the npx and npm labels;
- `npx ./tgz`;
- clean JSON stdout on a cold cache;
- an MCP `initialize` sent 1s late.
**Unit and lint:**
- Node: 111/111 on Node 20, 22 and 24 (the `engines` floor).
- Python: 2280 passed, 12 skipped in Docker as root; 2283 passed, 9
skipped as a non-root user (three permission tests skip under root).
- ruff, mypy, `check_dependency_floors.py` and actionlint are clean.
- The `test_release_bumps.py` tests that read this repository's git
history now skip when git or a git checkout isn't available, instead of
failing with `FileNotFoundError`. CI has both, so they still run there.
- The `pip install` steps in `release.yml` and the `release-bump-gate`
job are pinned to the versions in `uv.lock` (`build==1.5.0`,
`twine==7.0.0`, `packaging==26.3`).
**Not yet run:** real macOS and Windows runs, which the new `launcher`
job does on this PR, and the `publish-npm` job, which runs on the first
release now that the owner setup below is in place.
## Before the first release, owner steps (done 2026-10-23)
- [x] Trusted publisher on `@deepgram/cli` (set as `deepgramai`): GitHub
Actions, repo `deepgram/cli`, workflow `release.yml`, environment
`npm-publish`, permission **publish**. Note for anyone redoing this: npm
now separates `publish` from `stage publish`; the first attempt from the
web UI granted only `stage publish`, which a plain `npm publish` is
rejected under, so it was revoked and re-added with `npm trust github
@deepgram/cli --repo deepgram/cli --file release.yml --env npm-publish
--allow-publish`. Verify with `npm trust list @deepgram/cli`.
- [x] GitHub environment **`npm-publish`** created on this repo,
deployment branches limited to `main`, no required reviewer.
- [x] Both 2023 betas carry a deprecation message pointing at the new
install: `Unsupported 2023 beta. Install the Deepgram CLI with: npm i -g
@deepgram/cli@latest (0.4.1 or newer)`. (npm Support had deprecated them
on 2026-09-11 with its generic "no longer supported" text; the flag is
per version and does not block publishing.)
- [ ] After 0.4.1 publishes: confirm `npm view @deepgram/cli
dist-tags.latest` is `0.4.1`, then turn on "Require 2FA and disallow
tokens" for the package.
Until the first publish, `npx @deepgram/cli` still runs the old 2023
beta (now with the pointer message), so the README line goes live
slightly ahead of the package.
## Floors
`deepctl-cmd-update` and `deepctl-cmd-skills` already declare
`deepctl-core>=0.3.0` on this branch, and `deepctl-cmd-plugin` already
declares `deepctl-cmd-update>=0.2.7` for the `InstallMethod.NPM` import
it's always made without declaring. No hand edits are needed on the
release PR: the release sync job (`check_dependency_floors.py --fix`)
keeps these floors current automatically, and `release-bump-gate`
(above) catches it if a future change needs a bump this job doesn't
cover.
A normal `pip install -U deepctl` is already safe, because the root
floors pin everything.
## Known limits and follow-ups (not in this PR)
**Accepted limits:**
- A `pip install deepctl` into the same prefix overwrites npm's `dg`,
and brew and npm both want `dg`. The npm README says to use one install
method.
- Group-writable caches on shared-group systems (macOS `staff`, SUSE
`users`) fall back to a private temp dir. Any user-private-group member
could interfere, which is a limit of that convention.
- `nohup dg …` doesn't survive SIGHUP the way `nohup deepctl …` does,
because of how Node handles it. Use `setsid` or deepctl directly for
detached runs.
- With a TTY attached, a SIGINT sent only to the launcher's pid (not the
process group) isn't forwarded.
- npm and npx don't work on Intel Macs or Windows on ARM64. The launcher
installs with `--only-binary :all:`, and `cryptography` 50.0.0 has no
wheel for either platform. `README.md`, `npm/README.md` and
`llms-full.txt` say so. On an Intel Mac they point to Homebrew, because
the tap formula brings the Rust toolchain that a source build of
`cryptography` needs.
- The launcher sets `DEEPCTL_LAUNCHER=npm` for both a global install and
a project dependency, because it has no reliable, portable way to tell
them apart. `dg update` shows both the global and the project-local
command (`npm install @deepgram/cli@latest`); the update notice shows
only the global one. `npm/README.md` covers both and has a new Uninstall
section.
**Notes for reviewers:**
- `deepctl_core._launcher.capture_launcher()` still checks
`deepctl.main.LAUNCHER_CAPTURED` first. It can look like dead code on
this branch, but it's kept on purpose. In 0.5.0 (#165), `deepctl.main`
becomes stdlib-only and removes `DEEPCTL_LAUNCHER` before `deepctl_core`
is imported, so that flag is the only way npm installs are still
detected. The docstrings now explain this.
- `publish-on-release.yml` (line 46) also runs an unpinned `pip install
build twine`. That file isn't changed by this PR, so it's left for a
follow-up.
**Existing bugs found along the way:**
- `dg update` tells Homebrew users to run `brew upgrade deepctl`, but
the formula is `deepgram`.
- deepctl 0.4.0 crashes when HOME is unreadable (`PermissionError` on
`~/.deepctl/plugins/venv`).
- The `actions/checkout` pins are commented `# v4` but resolve to
v7.0.1.
**Phase 2:** PyInstaller per-platform binaries through
`optionalDependencies`, so npm installs need no Python or uv at all.
The history includes a manual re-publish path that was added and later
removed, so **squash on merge**.
0 commit comments