pex3 lock sync does not work with multi-resolve universal lockfile + pytorch
bug
Using latest version of Pex (`2.96.1`), given a simple requirement set with just `pytorch`:
```
torch>=2.9.1,<3.0.0; sys_platform != 'darwin'
torch>=2.9.1,<3.0.0; sys_platform == 'darwin'
```
(note the duplicated requirement is to force a separate resolve for MacOS)
Using this command to create a universal lockfile with pytorch's CPU wheels:
```
pex3 lock create \
--requirement requirements.txt \
--index "https://pypi.org/simple/" \
--index "pytorch=https://download.pytorch.org/whl/cpu" \
--source "pytorch=torch" \
--indent 2 \
-o default.lock \
--style universal \
--interpreter-constraint "==3.12.*"
```
Resolution completes successfully, and as expected the resulting lockfile has two resolves, one for MacOS with `torch==2.12.1` and one for the rest with `torch==2.12.1+cpu`.
However, it is now not possible to update this lockfile with `pex3 lock sync`, as even with _no change at all_ (so it should be a no-op) it fails:
```
pex3 lock sync \
--requirement requirements.txt \
--index "https://pypi.org/simple/" \
--index "pytorch=https://download.pytorch.org/whl/cpu" \
--source "pytorch=torch" \
--indent 2 \
--lock default.lock \
--style universal \
--interpreter-constraint "==3.12.*"
```
```
ERROR: Given the lock requirements:
torch<3.0.0,>=2.9.1; sys_platform != "darwin"
torch<3.0.0,>=2.9.1; sys_platform == "darwin"
The following lock update constraints could not all be satisfied:
cuda-bindings==13.3.1
cuda-pathfinder==1.5.5
cuda-toolkit==13.0.2
filelock==3.29.4
fsspec==2026.6
jinja2==3.1.6
markupsafe==3.0.3
mpmath==1.3
networkx==3.6.1
nvidia-cublas==13.1.1.3
nvidia-cuda-cupti==13.0.85
nvidia-cuda-nvrtc==13.0.88
nvidia-cuda-runtime==13.0.96
nvidia-cudnn-cu13==9.20.0.48
nvidia-cufft==12.0.0.61
nvidia-cufile==1.15.1.6
nvidia-curand==10.4.0.35
nvidia-cusolver==12.0.4.66
nvidia-cusparse==12.6.3.3
nvidia-cusparselt-cu13==0.8.1
nvidia-nccl-cu13==2.29.7
nvidia-nvjitlink==13.0.88
nvidia-nvshmem-cu13==3.4.5
nvidia-nvtx==13.0.85
setuptools==81
sympy==1.14
torch==2.12.1+cpu
triton==3.7.1
typing-extensions==4.15
Encountered 1 error updating default.lock:
1.) cp312-cp312-manylinux_2_39_x86_64: pid 3769 -> /home/tmarback/.cache/pex/venvs/3/c6117f5843d015aabb1f1ab35ff7b68a02dda062/0de1795ad4486f45ee94cecd983e9905b6b11dc9/bin/python /home/tmarback/.cache/pex/venvs/3/c6117f5843d015aabb1f1ab35ff7b68a02dda062/0de1795ad4486f45ee94cecd983e9905b6b11dc9/pex --disable-pip-version-check --exists-action a --no-input --no-python-version-warning --isolated --log /tmp/pex-pip-log.ocb2zz2e/pip.log -q --cache-dir /home/tmarback/.cache/pex/pip/3/23.2/pip_cache install --no-clean --dry-run --ignore-installed --report /home/tmarback/.cache/pex/downloads/1/.tmp/resolver_report.i241ac5_/universal-mac-4244e55870f3fe801640657563437be4427eafac/pip-report.json --requirement /home/tmarback/torch-test/pex_lock_split.fqwxxwv9.requirements2.txt --constraint /tmp/lock_update.lkqiyyte.constraints.txt torch<3.0.0,>=2.9.1; sys_platform == "darwin" --index-url https://pypi.org/simple --extra-index-url https://pypi.org/simple/ --retries 5 --timeout 15 exited with 1 and STDERR:
pip: ERROR: Cannot install torch<3.0.0 and >=2.9.1 because these package versions have conflicting dependencies.
pip: ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
pip:
pip: The conflict is caused by:
pip: The user requested torch<3.0.0 and >=2.9.1
pip: The user requested (constraint) torch==2.12.1+cpu
pip:
pip: To fix this you could try to:
pip: 1. loosen the range of package versions you've specified
pip: 2. remove package versions to allow pip attempt to solve the dependency conflict
```
This does _not_ happen if there is only one entry for `torch` in `requirements.txt` without the environment markers (so only one resolve is generated), but then the lockfile does not work on MacOS.
Sidenote, the same happens if `requirements.txt` only has `torch` once but there is a `constraints.txt` with the environment markers instead.
4 条评论