Skip to content

Remove alpha specs on non-RAPIDS dependencies #144

@jameslamb

Description

@jameslamb

Description

pip 25.0 was recently released (changelog). It includes this change:

The inclusion of packaging 24.2 changes how pre-release specifiers with < and > behave. Including a pre-release version with these specifiers now implies accepting pre-releases (e.g., <2.0dev can include 1.0rc1). To avoid implying pre-releases, avoid specifying them (e.g., use <2.0). The exception is !=, which never implies pre-releases. (#13163)

Relevant packaging change: pypa/packaging#794

This slight behavior change could result in some changed pip solves that affect RAPIDS.

For example, consider:

docker run \
  --rm \
  -it python:3.12 \
  /bin/bash -c 'pip install "pip<25"; pip install --dry-run "cython>=0.29.8,<3.0.0rc1"'
  • pip <25.0 finds cython==0.29.37
  • pip >=25.0 finds the pre-release cython==3.0.0b3

The presence of <...rc1 tells pip "pre-releases are OK, even if --pre was not passed to pip install", and then 3.0.0b3 satisfies this particular constraint because of the rules described in PEP 404 (link)):

Within a numeric release (1.0, 2.7.3), the following suffixes are permitted and MUST be ordered as shown:
.devN, aN, bN, rcN, <no suffix>, .postN

RAPIDS projects currently use such constraints in situations where it's not actually desirable to get pre-releases. For example, cudf 25.02 has these runtime dependencies:

numpy>=1.23,<3.0a0
pandas>=2.0,<2.2.4dev0

(cudf/dependencies.yaml)

With pip>25.0, these would tell pip that pre-releases of numpy or pandas are acceptable, which is not really the intention and could have negative side-effects (for example, regular releases tend to be more reliable as they've gone through more extensive testing).

We should audit the dependency constraints across RAPIDS and remove modifies like dev, a, b, and rc wherever allowing pre-releases is undesirable.

Benefits of this work

  • reduced risk of issues caused by unintentionally picking up pre-releases
  • fasts pip solves
  • avoids surprises for users (e.g. no numpy pre-releases sneaking into environments transitively via a cudf dependency)

Acceptance Criteria

  • RAPIDS projects only use pre-release specifiers for dependencies where they explicitly want to allow pre-releases

Approach

@bdice @vyasr and I talked about this offline and agreed the right fix is probably to change specs like this:

# before
pandas>=2.0,<2.2.4dev0

# after
pandas>=2.0,<2.2.4

Where possible, use rapids-reviser to update these specs.

Notes

N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions