ITADN

Universal lockfile drops aarch64 wheel when x86_64 wheel has +cpu local version suffix (pytorch_cpu index)

#3154Closedalxhslm 创建于 2026-04-16
question
A
alxhslmcommented
## Bug When generating a universal lockfile with `resolves_to_sources` routing torch to the pytorch_cpu index (`https://download.pytorch.org/whl/cpu`) for all platforms, pex includes the macOS arm64 and Linux x86_64 wheels but silently drops the Linux aarch64 wheel. ## Context The pytorch_cpu index hosts CPU-only wheels with inconsistent version tagging: | Platform | Wheel | Version | |---|---|---| | macOS arm64 | `torch-2.5.1-cp311-none-macosx_11_0_arm64.whl` | `2.5.1` | | Linux x86_64 | `torch-2.5.1+cpu-cp311-cp311-linux_x86_64.whl` | `2.5.1+cpu` | | Linux aarch64 | `torch-2.5.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl` | `2.5.1` | Only x86_64 has the `+cpu` local version suffix. macOS and aarch64 are plain `2.5.1`. ## What happens With this Pants config (using pex v2.92.3): ```toml [python-repos] indexes = [ "https://pypi.org/simple/", "pytorch_cpu=https://download.pytorch.org/whl/cpu" ] [python.resolves_to_sources] inference = [ "pytorch_cpu=torch; sys_platform == 'darwin'", "pytorch_cpu=torch; sys_platform == 'linux' and platform_machine == 'x86_64'", "pytorch_cpu=torch; sys_platform == 'linux' and platform_machine == 'aarch64'", ] [environments-preview.names] # aarch64 docker environment so pex targets this platform linux_arm64_python311 = "src/ci/images:linux_arm64_python311" ``` The generated lockfile contains: - ✅ `torch-2.5.1-cp311-none-macosx_11_0_arm64.whl` (from pytorch_cpu) - ✅ `torch-2.5.1+cpu-cp311-cp311-linux_x86_64.whl` (from pytorch_cpu) - ❌ `torch-2.5.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl` — **missing** Other packages (aiohttp, casadi, etc.) DO get aarch64 wheels in the same lockfile, confirming pex is targeting aarch64. It's specifically torch that's dropped. ## What should happen All three wheels should be included. Per PEP 440, `2.5.1+cpu` and `2.5.1` are equal when ignoring local version — pex already demonstrates this by including both the macOS `2.5.1` and x86_64 `2.5.1+cpu` wheels in the same lockfile entry. The aarch64 `2.5.1` wheel should be treated the same way as the macOS `2.5.1` wheel. ## Additional observations We also tried: - **Two-line config** (`sys_platform != 'darwin'` covering all Linux): same result, no aarch64 wheel - **Letting aarch64 fall through to PyPI** (only routing darwin + x86_64 to pytorch_cpu): still no aarch64 wheel, even though PyPI has `torch-2.5.1` for aarch64. It seems like once pex resolves torch from pytorch_cpu for one platform, it won't resolve from a different index for another platform. - **Darwin-only routing** (all Linux falls through to PyPI): this DOES include the aarch64 wheel from PyPI — but also pulls in the x86_64 CUDA wheel and 12 nvidia packages, which is what we're trying to avoid. ## Workaround Direct URL requirements in the requirements file bypass index resolution entirely and work correctly: ``` torch @ https://download.pytorch.org/whl/cpu/torch-2.5.1%2Bcpu-cp311-cp311-linux_x86_64.whl ; sys_platform == 'linux' and platform_machine == 'x86_64' torch @ https://download.pytorch.org/whl/cpu/torch-2.5.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl ; sys_platform == 'linux' and platform_machine == 'aarch64' torch @ https://download.pytorch.org/whl/cpu/torch-2.5.1-cp311-none-macosx_11_0_arm64.whl ; sys_platform == 'darwin' ``` This produces a lockfile with all three CPU-only wheels and no nvidia packages — but requires manually updating URLs when upgrading torch. ## Environment - pex: v2.92.3 - Pants: 2.30.0a0 - Python: 3.11 - Host: macOS arm64 - Targets: macOS arm64 + Linux x86_64 + Linux aarch64 (via docker_environment) ## Related - pex-tool/pex#3147 — `platform_machine` marker fix (resolved in v2.92.3, thank you!)
关闭于 2026-04-17 6 条评论