crates.io 403 on python-requests UA is breaking nixpkgs rust builds
nixpkgs `rustPlatform.fetchCargoVendor` uses python's `requests` library to vendor crates from crates.io, and it never sets an explicit user-agent, so every request goes out as`python-requests/<version>`. Recently these requests started getting rejected with 403.
This affects a very large chunk of the Rust ecosystem on NixOS right now, essentially any Rust package that isn't already cached.
It looks like `https://crates.io/api/v1/crates/<name>/<version>/download` returns `403 Forbidden` for certain `python-requests/<version>` user-agents. Current observed behavior:
python-requests/2.32.5 -> 403
python-requests/2.33.1 -> 403
python-requests/2.32.3 -> 302
(empty UA) -> 302
foo/1.0 -> 302
I put up fix at https://github.com/NixOS/nixpkgs/pull/512735 that sets an identifying UA (`nixpkgs fetchCargoVendor (https://github.com/NixOS/nixpkgs)`), which aligns with crates.io's documented UA policy and bypasses the blocklist. But landing it in a usable form takes time:
- Review + merge into `staging`
- Propagation through `staging` → `staging-next` → `master` (since `rustPlatform.fetchCargoVendor` is a mass-rebuild trigger)
- Binary cache rebuild
- Channel bump to `nixos-unstable` / release branches
Realistically we're looking at **weeks** before affected users have a working `fetchCargoVendor` via channels, and in the meantime every user on an affected nixpkgs revision sees broken Rust builds.
Would it be possible to **temporarily unblock the specific `python-requests/<version>` UAs that nixpkgs is currently sending in the wild** (2.32.5 and 2.33.1 at least, not sure if more are needed)? Just long enough to cover the window until our PR lands and reaches users.
8 条评论