-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Description
pip 25.0 was recently released (changelog). It includes this change:
The inclusion of packaging 24.2 changes how pre-release specifiers with
<and>behave. Including a pre-release version with these specifiers now implies accepting pre-releases (e.g.,<2.0devcan include1.0rc1). To avoid implying pre-releases, avoid specifying them (e.g., use<2.0). The exception is!=, which never implies pre-releases. (#13163)
Relevant packaging change: pypa/packaging#794
This slight behavior change could result in some changed pip solves that affect RAPIDS.
For example, consider:
docker run \
--rm \
-it python:3.12 \
/bin/bash -c 'pip install "pip<25"; pip install --dry-run "cython>=0.29.8,<3.0.0rc1"'pip <25.0findscython==0.29.37pip >=25.0finds the pre-releasecython==3.0.0b3
The presence of <...rc1 tells pip "pre-releases are OK, even if --pre was not passed to pip install", and then 3.0.0b3 satisfies this particular constraint because of the rules described in PEP 404 (link)):
Within a numeric release (1.0, 2.7.3), the following suffixes are permitted and MUST be ordered as shown:
.devN, aN, bN, rcN, <no suffix>, .postN
RAPIDS projects currently use such constraints in situations where it's not actually desirable to get pre-releases. For example, cudf 25.02 has these runtime dependencies:
numpy>=1.23,<3.0a0
pandas>=2.0,<2.2.4dev0
With pip>25.0, these would tell pip that pre-releases of numpy or pandas are acceptable, which is not really the intention and could have negative side-effects (for example, regular releases tend to be more reliable as they've gone through more extensive testing).
We should audit the dependency constraints across RAPIDS and remove modifies like dev, a, b, and rc wherever allowing pre-releases is undesirable.
Benefits of this work
- reduced risk of issues caused by unintentionally picking up pre-releases
- fasts
pipsolves - avoids surprises for users (e.g. no
numpypre-releases sneaking into environments transitively via acudfdependency)
Acceptance Criteria
- RAPIDS projects only use pre-release specifiers for dependencies where they explicitly want to allow pre-releases
- (e.g., like within-RAPIDS specs like
dask-cuda==25.2.*,>=0.0.0a0used to pull in nightlies during development
- (e.g., like within-RAPIDS specs like
Approach
@bdice @vyasr and I talked about this offline and agreed the right fix is probably to change specs like this:
# before
pandas>=2.0,<2.2.4dev0
# after
pandas>=2.0,<2.2.4
Where possible, use rapids-reviser to update these specs.
Notes
N/A