Conversation
Rewrite proxy registry URLs in `uv.lock` to their canonical counterparts via the `UV_INDEX_PROXIES` environment variable, so that lockfiles stay stable regardless of which mirror resolved the packages. Context: astral-sh#6349. When `UV_DEFAULT_INDEX` points at an internal mirror, upstream `uv lock` rewrites every `source.registry` URL in `uv.lock` to that mirror, creating noisy diffs and breaking portability across environments that use different mirrors. Set `UV_INDEX_PROXIES` with `canonical:proxy` mappings: UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simple After resolution, `Lock::rewrite_proxy_urls` replaces every matching proxy registry URL with its canonical counterpart. The canonical URLs are also injected into the `satisfies()` check so subsequent `uv lock` runs recognize the lockfile as up-to-date instead of re-resolving, and are mapped back to the proxy at install time so fetches hit a reachable mirror. All fork changes are tagged with `// fork:` comments so rebasers can find them quickly. Rebased from v0.11.14 onto 0.12.0. Notable adaptations: - Upstream deleted the `IndexUrls` type, so the commit patching `IndexUrls::default_index()` was dropped; the resolver now routes through `IndexLocations::default_index()`, which remains patched. - Upstream extracted index recording into `Lock::record_index` and moved the explicit-index pass into `satisfies_requires_dist`, so this fork only injects canonical URLs into the `remotes` set. - `UrlString::new` is now private, so mappings are built through the public API (`DisplaySafeUrl::parse` -> `UrlString::from`), which also rejects malformed `UV_INDEX_PROXIES` entries. - `std::env::set_var` is `unsafe` under edition 2024, so the env-reading entry points are split into thin wrappers over pure functions taking `&[ProxyMapping]`, which the tests call directly. This avoids `unsafe` and makes the tests parallel-safe. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A proxy index advertises artifact URLs on its own host, so every wheel
and sdist URL in `uv.lock` pointed at the proxy, and every download went
through it. That makes the lockfile non-portable in the same way the
`source.registry` URLs were.
Add `UV_ARTIFACT_PROXIES`, mapping canonical artifact base URLs to proxy
base URLs:
UV_ARTIFACT_PROXIES=https://files.pythonhosted.org/packages:https://pypi-proxy.example.com/packages
The rewrite happens in `FileLocation::new`, the single point where both
`File::try_from_pypi` and `File::try_from_pyx` turn index-response URLs
into locations. That covers resolution-time downloads and the URLs
recorded in `uv.lock` at once, so no reverse mapping is needed at install
time.
Unlike the `UV_INDEX_PROXIES` mappings, these are prefix mappings, since
every artifact has a distinct path below the base. Matching respects path
boundaries, so a `/packages` base does not match `/packages-internal`.
Only absolute artifact URLs are rewritten. An index advertising relative
URLs resolves them against the index base at `to_url()` time and would
still point at the proxy.
Verified end-to-end against the internal mirror: `uv lock` records
`files.pythonhosted.org` URLs while `source.registry` stays on the proxy,
the recorded hashes match the proxy-served bytes, and the `.metadata`
sidecar is fetched from `files.pythonhosted.org`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `xattr -d com.apple.quarantine` line is self-explanatory; the comment just added noise to the copy-pasteable block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep
uv.lockfree of proxy-specific URLs, so the same lockfile works across environments that reach PyPI through different mirrors. Rebased onto upstream0.12.16; supersedes #3 (which targeted0.12.0).Context: astral-sh#6349. When
UV_DEFAULT_INDEXpoints at an internal mirror, upstreamuv lockbakes that mirror's host into both thesource.registryfield and every artifact URL, creating noisy diffs and breaking portability.Two independent mappings, because the canonical hosts differ — the index lives at
pypi.org/simplewhile artifacts live atfiles.pythonhosted.org/packages.UV_INDEX_PROXIES— registry URLsexport UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simpleAfter resolution,
Lock::rewrite_proxy_urlsreplaces every matching proxy registry URL with its canonical counterpart insource.registry. The canonical URLs are also injected into thesatisfies()check so subsequentuv lockruns recognize the lockfile as up-to-date instead of re-resolving, and are mapped back to the proxy at install time so fetches still hit a reachable mirror.UV_ARTIFACT_PROXIES— wheel and sdist URLsexport UV_ARTIFACT_PROXIES=https://files.pythonhosted.org/packages:https://pypi-proxy.example.com/packagesA proxy index advertises artifact URLs on its own host, so every wheel and sdist URL in
uv.lockpointed at the proxy and every download went through it.The rewrite happens in
FileLocation::new, the single point where bothFile::try_from_pypiandFile::try_from_pyxturn index-response URLs into locations. That covers resolution-time downloads and the URLs recorded inuv.lockat once, so — unlike the registry mapping — no reverse mapping is applied: artifacts are fetched directly from the canonical host.Two differences from the registry mapping worth knowing:
/packagesbase does not match/packages-internal/....to_url()time and would still point at the proxy.All fork changes are tagged with
// fork:comments so rebasers can find them quickly.Upstream drift since 0.12.0
Three changes in upstream required adapting this fork:
uv-resolverto a newuv-lockcrate.url_preservation.rsmoves with it tocrates/uv-lock/src/lock/.Lock::by_idnow maps to aPackageIndexnewtype rather than a bareusize, so theby_idrebuild after rewritingPackageIds wraps the index.Lock::find_by_idwas removed. The test that checksby_idstill resolves after a rewrite now goes throughLock::packagewith theby_idindex, which is what the removed helper did internally.Diff surface
.github/workflows/fork-release.ymlcrates/uv-lock/src/lock/url_preservation.rscrates/uv-distribution-types/src/artifact_proxies.rscrates/uv-distribution-types/src/index_url.rscrates/uv/src/commands/project/lock.rscrates/uv-lock/src/lock/mod.rscrates/uv-distribution-types/src/file.rscrates/uv-distribution-types/src/lib.rsTest plan
cargo clippy -p uv-lock -p uv-distribution-types --all-targets— clean, no warnings.cargo clippy -p uv --all-targets— clean, no warnings.cargo test -p uv-lock --lib url_preservation— 6 passed.cargo test -p uv-distribution-types --lib— 70 passed (includes 6artifact_proxiestests).cargo fmt --check— clean.End-to-end run against a containerized index pair
Two toy PEP 503 indexes run in a container, each serving the same wheel but advertising artifact URLs on its own host: a "canonical" host on
:8101and a "proxy" host on:8102. Resolution is pointed at the proxy only:source = { registry = "http://localhost:8101/simple" }— the canonical host, not the proxy that actually resolved it.url = "http://localhost:8101/packages/toy_pkg-1.0.0-py3-none-any.whl", with the hash of the proxy-served bytes.[proxy] GET /simple/toy-pkg/) while the artifactHEAD/GEThit the canonical host — confirming artifacts bypass the proxy.uv lockdoes not re-resolve. ReportsExisting uv.lock satisfies workspace requirements, and the lockfile is byte-identical afterwards. This closes the item left unverified in fork: preserve canonical URLs in uv.lock across re-locks (v0.12.0) #3.uv syncinstalls from the canonicalized lockfile, fetching the wheel from the canonical host.linux/amd64container againstuv-x86_64-unknown-linux-gnu.tar.gzfrom thepr-4release (uv 0.12.16 (182ecec52)), reproduces every result above: canonical registry and artifact URLs, index traffic to the proxy, artifact traffic to the canonical host, and a seconduv lockthat leaves the lockfile untouched.Known gap
There is no integration test under
crates/uv/tests/it/for either rewrite — coverage is the unit tests plus the end-to-end run above.AGENTS.mdprefers integration tests, and the dev-dependencies needed to build one (wiremock,hyper, as already used intests/it/network.rs) are available; worth adding if this fork lives longer than expected.🤖 Generated with Claude Code