diff --git a/docs/extensions/pre-commit.mdx b/docs/extensions/pre-commit.mdx index 0b5167006f..c470e87ed0 100644 --- a/docs/extensions/pre-commit.mdx +++ b/docs/extensions/pre-commit.mdx @@ -18,7 +18,7 @@ Add the following to your `.pre-commit-config.yaml` file: ```yaml repos: - repo: https://github.com/semgrep/pre-commit - rev: 'v1.169.0' + rev: 'v1.170.0' hooks: - id: semgrep entry: semgrep @@ -51,7 +51,7 @@ Add the following to your `.pre-commit-config.yaml` file: ```yaml repos: - repo: https://github.com/semgrep/pre-commit - rev: 'v1.169.0' + rev: 'v1.170.0' hooks: - id: semgrep-ci ``` diff --git a/docs/semgrep-supply-chain/malware-firewall.mdx b/docs/semgrep-supply-chain/malware-firewall.mdx new file mode 100644 index 0000000000..023687960a --- /dev/null +++ b/docs/semgrep-supply-chain/malware-firewall.mdx @@ -0,0 +1,176 @@ +--- +title: "Semgrep Malware Firewall" +description: "The Semgrep Malware Firewall prevents malicious packages from being installed on developer machines by checking each requested package against Semgrep's continuously updated malware database." +hidden: true +--- + + +Semgrep Malware Firewall is in **private beta**. + + +## Overview + +The Semgrep Malware Firewall prevents malicious packages from being installed on your machine. It sits in front of your package installs, checks each requested package against Semgrep's continuously updated database of known malware, and blocks anything that matches. + +## How it works + +The firewall has the following parts: + +1. A lightweight **client proxy** that runs on every machine. + - You set it up once with `mfw install`. It sets the standard proxy environment variables `HTTP_PROXY` and `HTTPS_PROXY` so package managers respect it automatically—no per-tool configuration is needed. For example, you don't need to set up `tool.uv.index` manually for `uv`. + - It watches traffic to known package registry URLs such as `registry.npmjs.org`, identifies package-download requests specifically—including tarballs, wheels, and similar artifacts—and gates those artifact requests on a response from the Semgrep backend before the download completes. + - All other traffic is proxied through transparently. +2. The Semgrep-operated **malware firewall backend**, which receives requests from the local proxy containing an ecosystem, package name, and version, and replies with a verdict. +3. The **malware database** that is continuously updated from a variety of threat feeds and maintained by Semgrep's security research team. + +The request cycle: + +1. On install, the firewall runs `mfw login` to sign you in. Each verdict request is then authenticated with a short-lived bearer token. +2. When you run a normal package install, the local proxy intercepts every package the installer tries to fetch. +3. The proxy asks the Semgrep backend whether the specific version of that package is malicious. +4. The backend matches the version to the malware database and returns a verdict. +5. If the package is safe, the install proceeds normally. If it matches known malware, the proxy blocks the download and reports why. + +### Advisory and database sync objectives + +- For active, high-severity supply chain incidents, the Semgrep's security research team works to issue and deploy an advisory as soon as a threat is identified rather than waiting for it to be published to sources like the [Open Source Vulnerabilities](https://osv.dev) (OSV) database. +- For all other findings, the malware database syncs from OSV every 2 hours. + +### Cooldowns + +The firewall doesn't currently support configuring a cooldown period before a newly published package version can be installed. You can configure cooldowns at the package-manager level. See [cooldowns.dev](https://cooldowns.dev/) for a guide across ecosystems. + +## Reporting + +Semgrep AppSec Platform shows basic firewall reporting, including scanned and blocked dependencies, in the **Malware firewall** section of the **Dashboard**. + +## Prerequisites + +- You must have an active Semgrep account +- You must have [Semgrep Guardian](/guardian) enabled +- You must have access to a macOS or Linux terminal or shell + +## Install and verify + +You can install the firewall with the Semgrep Guardian skill or manually with the install script. If you already use [Semgrep Guardian](/guardian), the `/install-mfw` skill is the quickest path — it installs and configures everything for you. + + + + Semgrep Guardian ships with an `/install-mfw` skill that installs and sets up the firewall for you. + + + + In your AI coding agent with the [Semgrep Guardian](/guardian) plugin installed (such as Claude Code), run: + + ```bash + /install-mfw + ``` + + + Follow the prompts. The skill downloads the `mfw` client, adds a local certificate authority to your trust store, sets up shell integration and a background daemon, and walks you through signing in. + + + Restart your Terminal / shell so the proxy environment variables take effect. + + + + + + + Open your terminal. + + + Download and install the firewall: + + ```bash + curl -fsSL https://semgrep.dev/dist/mfw/install.sh | sh + ``` + + + When prompted, confirm with `y`. + + + Restart your terminal or shell so the proxy environment variables take effect. + + + + + +Verify the install: + +```bash +mfw doctor +``` + +You should see: + +``` +Semgrep mfw is protecting this machine ✅ +``` + +## Test the firewall + +Use a known-safe demo package to confirm that the firewall blocks malicious installs without touching real malware. + + + + Open or create a test directory and initialize a project: + + ```bash + uv init + ``` + + + Attempt to install the demo malware package: + + ```bash + uv add --no-cache semgrep-malware-demo-aws-cred-read + ``` + + + **Expected result:** the install is blocked. The firewall should intercept and reject the package. + + + To test with a second package manager, repeat with pip: + + ```bash + pip install --no-cache semgrep-malware-demo-aws-cred-read + ``` + + + +Both these packages should be blocked by the Semgrep Malware Firewall. If either install succeeds, the firewall isn't intercepting traffic correctly. Review the `mfw doctor` output, and see [Troubleshoot `mfw doctor` failures](#troubleshoot-mfw-doctor-failures). + +## Uninstall the firewall + +To remove the firewall and its proxy configuration from a machine: + +```bash +mfw uninstall +``` + +This clears the `HTTP_PROXY` and `HTTPS_PROXY` environment variables that `mfw install` configured and removes the local proxy process. Restart your terminal or shell afterward to confirm that the variables are cleared. + +To verify removal: + +```bash +mfw doctor +``` + +It should no longer report `Semgrep mfw is protecting this machine ✅`. + +## Troubleshoot `mfw doctor` failures + +| Symptom | Likely cause | Suggested fix | +| :--- | :--- | :--- | +| `mfw doctor` reports that the firewall isn't protecting the machine right after install | The shell wasn't restarted, so `HTTP_PROXY` and `HTTPS_PROXY` aren't set in the current session | Restart your terminal or shell, then re-run `mfw doctor` | +| `mfw doctor` reports an authentication error | You haven't run `mfw login`, or the short-lived bearer token has expired | Run `mfw login` again | +| Package installs are slow or time out | The local proxy can't reach the Semgrep verdict backend because of network or firewall rules that block egress | Confirm that outbound access to Semgrep's backend is allowed; review corporate proxy or VPN settings | +| A known-safe package is unexpectedly blocked | The backend returned a stale or incorrect verdict, or a false-positive match in the malware database | Contact [Semgrep support](/support) with the ecosystem, package name, and version so Security Research can review | +| `mfw doctor` isn't found, or the command isn't recognized | The install script didn't complete, or the shell `PATH` wasn't updated | Re-run the install command from [Install and verify](#install-and-verify), then restart your shell | + +If you still have issues, contact [Semgrep support](/support) with the full `mfw doctor` output. + +## Related products + +The Semgrep Malware Firewall focuses specifically on blocking malicious dependency installs during supply chain attacks. It complements, but is distinct from, [**Semgrep Guardian**](/guardian), which scans code generated by AI coding agents such as Claude Code and Cursor for security issues before it ships. Guardian catches insecure code that an agent writes. The Malware Firewall catches malicious packages that an agent or a human tries to install. diff --git a/docs/snippets/metrics.mdx b/docs/snippets/metrics.mdx index b650a637dc..fcd14198f6 100644 --- a/docs/snippets/metrics.mdx +++ b/docs/snippets/metrics.mdx @@ -27,6 +27,7 @@ These principles inform our decisions around data collection: $ semgrep --config=myrule.yaml # → no metrics (loading rules from local file) $ semgrep --config=p/python # → metrics enabled (fetching Registry) $ semgrep login && semgrep ci # → metrics enabled (logged in to semgrep.dev) +$ semgrep install-semgrep-pro # → metrics enabled (requires being logged in) ``` Semgrep does **not** enable metrics when running with only local configuration files or command-line search patterns. @@ -85,6 +86,7 @@ Environmental data provide contextual data about Semgrep CLI’s runtime environ - An [anonymous user ID](#anonymous-user-id) that identifies the machine - IP address that triggers a run - Value of the CI environment variable, if set +- How the Semgrep CLI was installed (`pip`, `homebrew`, `docker`, or `unknown`), detected heuristically from the install path and environment variables - Pseudoanonymized hash of the scanned project’s name - Pseudoanonymized hash of the rule definitions run - Pseduoanonymized hash of the config option. @@ -138,6 +140,13 @@ Additional data is reported when used in conjunction with an IDE integration, su Note: For all officially supported Semgrep IDE integrations, these metrics can be disabled via settings in the IDE. By default these settings follow any global telemetry/metrics settings the user may have already set for the IDE itself. +### Pro Engine installation + +When running `semgrep install-semgrep-pro` (which requires being logged in), additional data is reported to help us understand how many Pro Engine installations happen and how many succeed; e.g. + +- Whether the installation completed successfully +- Which installation step failed, as a fixed identifier such as `download-403` or `version-check-failed` (never a free-form error message) + ### Pseudoanonymization Certain identifying data (e.g. project URLs) are pseudoanonymized before being sent to the Semgrep, Inc backend. @@ -177,6 +186,7 @@ Semgrep, Inc will: | | Deployment ID | The ID organization associated with the logged in account | Understand popularity of logged in features by organization | 1234 | Number | | | Integration name | If Semgrep is being called by another tool, optional name of that integration | Reproduce and debug issues specific integrations | `gitlab` | String | | | CI | Notes if Semgrep is running in CI and the name of the provider | Reproduce and debug issues with specific CI providers | GitLabCI v0.13.12 | String | +| | Install method | How the Semgrep CLI was installed, detected heuristically | Understand which distribution channels are used; debug channel-specific issues | `pip` | String | | | Client IP | IP address that triggered a run | Understand broad ruleset usage | 0.0.0.0 | String | | | | | | | | | Performance | | | | | | @@ -222,6 +232,10 @@ Semgrep, Inc will: | |Autofix count|How many autofixes have been triggered through the integration|Understand the value that the integration provides to the user in helping remediate code issue|10|Number| | |Ignore count|How many findings have been ignored by the user through the integration|Understand the quality and noisiness of rules|5|Number| +|Pro Engine install|||||| +| |Success|Whether `semgrep install-semgrep-pro` completed successfully|Understand how many Pro Engine installs happen and how many succeed|`true`|Boolean| +| |Error|Fixed identifier of the install step that failed|Debug common installation failures|`download-403`|String| + ### Anonymous user ID > `anonymous_user_id: "5f52484c-3f82-4779-9353-b29bbd3193b6"`