Skip to content

fork: preserve canonical URLs in uv.lock across re-locks (v0.12.16) - #4

Open
harupy wants to merge 3 commits into
mainfrom
fork/preserve-lockfile-urls-v0.12.16
Open

harupy wants to merge 3 commits into
mainfrom
fork/preserve-lockfile-urls-v0.12.16

Conversation

@harupy

@harupy harupy commented Sep 20, 2026

Copy link
Copy Markdown
Member

Summary

Keep uv.lock free of proxy-specific URLs, so the same lockfile works across environments that reach PyPI through different mirrors. Rebased onto upstream 0.12.16; supersedes #3 (which targeted 0.12.0).

Context: astral-sh#6349. When UV_DEFAULT_INDEX points at an internal mirror, upstream uv lock bakes that mirror's host into both the source.registry field and every artifact URL, creating noisy diffs and breaking portability.

Two independent mappings, because the canonical hosts differ — the index lives at pypi.org/simple while artifacts live at files.pythonhosted.org/packages.

UV_INDEX_PROXIES — registry URLs

export UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simple

After resolution, Lock::rewrite_proxy_urls replaces every matching proxy registry URL with its canonical counterpart in source.registry. The canonical URLs are also injected into the satisfies() check so subsequent uv lock runs recognize the lockfile as up-to-date instead of re-resolving, and are mapped back to the proxy at install time so fetches still hit a reachable mirror.

UV_ARTIFACT_PROXIES — wheel and sdist URLs

export UV_ARTIFACT_PROXIES=https://files.pythonhosted.org/packages:https://pypi-proxy.example.com/packages

A proxy index advertises artifact URLs on its own host, so every wheel and sdist URL in uv.lock pointed at the proxy and every download went through it.

The rewrite happens in FileLocation::new, the single point where both File::try_from_pypi and File::try_from_pyx turn index-response URLs into locations. That covers resolution-time downloads and the URLs recorded in uv.lock at once, so — unlike the registry mapping — no reverse mapping is applied: artifacts are fetched directly from the canonical host.

Two differences from the registry mapping worth knowing:

  • These are prefix mappings, since every artifact has a distinct path below the base. Matching respects path boundaries, so a /packages base does not match /packages-internal/....
  • Only absolute artifact URLs are rewritten. An index advertising relative URLs resolves them against the index base at to_url() time and would still point at the proxy.

Deployment note: because artifacts are no longer fetched through the proxy, every environment running uv lock/uv sync must be able to reach files.pythonhosted.org directly. A runner that cannot reach it will fail rather than fall back to the proxy. This has not been verified on CI runners.

All fork changes are tagged with // fork: comments so rebasers can find them quickly.

Upstream drift since 0.12.0

Three changes in upstream required adapting this fork:

  • The lock module moved from uv-resolver to a new uv-lock crate. url_preservation.rs moves with it to crates/uv-lock/src/lock/.
  • Lock::by_id now maps to a PackageIndex newtype rather than a bare usize, so the by_id rebuild after rewriting PackageIds wraps the index.
  • Lock::find_by_id was removed. The test that checks by_id still resolves after a rewrite now goes through Lock::package with the by_id index, which is what the removed helper did internally.

Diff surface

File Change
.github/workflows/fork-release.yml +164
crates/uv-lock/src/lock/url_preservation.rs +342 (new, 6 unit tests)
crates/uv-distribution-types/src/artifact_proxies.rs +160 (new, 6 unit tests)
crates/uv-distribution-types/src/index_url.rs +30
crates/uv/src/commands/project/lock.rs +26/−8
crates/uv-lock/src/lock/mod.rs +23/−8
crates/uv-distribution-types/src/file.rs +6/−1
crates/uv-distribution-types/src/lib.rs +2

Test plan

  • cargo clippy -p uv-lock -p uv-distribution-types --all-targets — clean, no warnings.
  • cargo clippy -p uv --all-targets — clean, no warnings.
  • cargo test -p uv-lock --lib url_preservation — 6 passed.
  • cargo test -p uv-distribution-types --lib — 70 passed (includes 6 artifact_proxies tests).
  • cargo fmt --check — clean.

End-to-end run against a containerized index pair

Two toy PEP 503 indexes run in a container, each serving the same wheel but advertising artifact URLs on its own host: a "canonical" host on :8101 and a "proxy" host on :8102. Resolution is pointed at the proxy only:

export UV_DEFAULT_INDEX=http://localhost:8102/simple
export UV_INDEX_PROXIES=http://localhost:8101/simple:http://localhost:8102/simple
export UV_ARTIFACT_PROXIES=http://localhost:8101/packages:http://localhost:8102/packages
  • Registry URL is canonicalized. The lockfile records source = { registry = "http://localhost:8101/simple" } — the canonical host, not the proxy that actually resolved it.
  • Artifact URL is canonicalized. url = "http://localhost:8101/packages/toy_pkg-1.0.0-py3-none-any.whl", with the hash of the proxy-served bytes.
  • Traffic splits as designed. Server logs show the index query hitting the proxy ([proxy] GET /simple/toy-pkg/) while the artifact HEAD/GET hit the canonical host — confirming artifacts bypass the proxy.
  • Second uv lock does not re-resolve. Reports Existing uv.lock satisfies workspace requirements, and the lockfile is byte-identical afterwards. This closes the item left unverified in fork: preserve canonical URLs in uv.lock across re-locks (v0.12.0) #3.
  • uv sync installs from the canonicalized lockfile, fetching the wheel from the canonical host.
  • Re-verified with the published Linux release binary. The same scenario, run entirely inside a linux/amd64 container against uv-x86_64-unknown-linux-gnu.tar.gz from the pr-4 release (uv 0.12.16 (182ecec52)), reproduces every result above: canonical registry and artifact URLs, index traffic to the proxy, artifact traffic to the canonical host, and a second uv lock that leaves the lockfile untouched.

Known gap

There is no integration test under crates/uv/tests/it/ for either rewrite — coverage is the unit tests plus the end-to-end run above. AGENTS.md prefers integration tests, and the dev-dependencies needed to build one (wiremock, hyper, as already used in tests/it/network.rs) are available; worth adding if this fork lives longer than expected.

🤖 Generated with Claude Code

harupy and others added 3 commits September 20, 2026 18:26
Rewrite proxy registry URLs in `uv.lock` to their canonical counterparts
via the `UV_INDEX_PROXIES` environment variable, so that lockfiles stay
stable regardless of which mirror resolved the packages.

Context: astral-sh#6349. When `UV_DEFAULT_INDEX` points at an internal
mirror, upstream `uv lock` rewrites every `source.registry` URL in
`uv.lock` to that mirror, creating noisy diffs and breaking portability
across environments that use different mirrors.

Set `UV_INDEX_PROXIES` with `canonical:proxy` mappings:

    UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simple

After resolution, `Lock::rewrite_proxy_urls` replaces every matching proxy
registry URL with its canonical counterpart. The canonical URLs are also
injected into the `satisfies()` check so subsequent `uv lock` runs
recognize the lockfile as up-to-date instead of re-resolving, and are
mapped back to the proxy at install time so fetches hit a reachable
mirror.

All fork changes are tagged with `// fork:` comments so rebasers can find
them quickly.

Rebased from v0.11.14 onto 0.12.0. Notable adaptations:

- Upstream deleted the `IndexUrls` type, so the commit patching
  `IndexUrls::default_index()` was dropped; the resolver now routes
  through `IndexLocations::default_index()`, which remains patched.
- Upstream extracted index recording into `Lock::record_index` and moved
  the explicit-index pass into `satisfies_requires_dist`, so this fork
  only injects canonical URLs into the `remotes` set.
- `UrlString::new` is now private, so mappings are built through the
  public API (`DisplaySafeUrl::parse` -> `UrlString::from`), which also
  rejects malformed `UV_INDEX_PROXIES` entries.
- `std::env::set_var` is `unsafe` under edition 2024, so the env-reading
  entry points are split into thin wrappers over pure functions taking
  `&[ProxyMapping]`, which the tests call directly. This avoids `unsafe`
  and makes the tests parallel-safe.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A proxy index advertises artifact URLs on its own host, so every wheel
and sdist URL in `uv.lock` pointed at the proxy, and every download went
through it. That makes the lockfile non-portable in the same way the
`source.registry` URLs were.

Add `UV_ARTIFACT_PROXIES`, mapping canonical artifact base URLs to proxy
base URLs:

    UV_ARTIFACT_PROXIES=https://files.pythonhosted.org/packages:https://pypi-proxy.example.com/packages

The rewrite happens in `FileLocation::new`, the single point where both
`File::try_from_pypi` and `File::try_from_pyx` turn index-response URLs
into locations. That covers resolution-time downloads and the URLs
recorded in `uv.lock` at once, so no reverse mapping is needed at install
time.

Unlike the `UV_INDEX_PROXIES` mappings, these are prefix mappings, since
every artifact has a distinct path below the base. Matching respects path
boundaries, so a `/packages` base does not match `/packages-internal`.

Only absolute artifact URLs are rewritten. An index advertising relative
URLs resolves them against the index base at `to_url()` time and would
still point at the proxy.

Verified end-to-end against the internal mirror: `uv lock` records
`files.pythonhosted.org` URLs while `source.registry` stays on the proxy,
the recorded hashes match the proxy-served bytes, and the `.metadata`
sidecar is fetched from `files.pythonhosted.org`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `xattr -d com.apple.quarantine` line is self-explanatory; the comment
just added noise to the copy-pasteable block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant