-
Notifications
You must be signed in to change notification settings - Fork 98
rename 'sklearn' conda dependency to 'scikit-learn' #743
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
/ok to test |
aha!! Think this would explain it!
There is no package called |
/ok to test |
Looks like this worked...
|
For my curiosity/education: is there a way to tell the tool used to build conda packages that if one of the dependencies doesn't exist, it should error/warn (very loudly)? It seems hard to imagine a situation where I add a dependency to my package's dependencies list and I am Ok with it not existing. Basically, I was wondering how it happened that it was hard to spot that (For PyPI there is a |
"not existing" means "not found in the channels There are cases where that's helpful. For example, The CUDA
This is one of many failure modes that are intended to be caught by the And we have that in many recipes across RAPIDS... cuvs/conda/recipes/cuvs/meta.yaml Lines 72 to 76 in af81a61
... but skip it by passing Lines 28 to 31 in af81a61
... because conda builds run on CPU-only runners, and so many RAPIDS packages cannot be The most reliable way to catch problems like this given those constraints is to force You may want to follow conversation about that at these places: |
Thanks, this fix makes sense. Strict channel priority will help us a lot... |
/merge |
Fixes #6403 This project publishes a conda package, `cuml-cpu`, which does what it sounds like... allows the use of cuML on systems without a GPU. This proposes some updates to packaging for `cuml-cpu`: * fixes importing in CPU-only environment (broken in 25.04, see #6403) * enables import tests during conda builds, to reduce the risk of such issues going undetected in the future ## Notes for Reviewers ### Why all these changes in Python code? See some of the challenges I faced documented in #6400 (comment). In short, `import cuml` when it was installed via `cuml-cpu` will break at import time whenever modules imported with `cuml.internals.safe_imports.gpu_only_import()` are used in any of the following ways: * type hints * decorators * any other module-level direct use Like this: ```text cuml.internals.safe_imports.UnavailableError: cudf is not installed in non GPU-enabled installations ``` ### How long has this been broken? What's the root cause? It seems like something changed within 25.04... earlier versions of cuML are not affected by these issues: #6403 (comment) I don't know what the root cause is. Maybe some changes to `cuml`'s top-level imports in 25.04 is now pulling in the modules with these problems at runtime, when previously it wasn't? I'm really not sure. ### Benefits of these Changes This adds a bit of test coverage in CI, minimally verifying that `cuml-cpu` is installable and that `import cuml` works in an environment without a GPU. Inspired by: * similar changes in `cuvs`: rapidsai/cuvs#750 * this conversation I recently had with @betatim : rapidsai/cuvs#743 (comment) ### How I tested this Saw stuff like this in `conda-python-build` jobs, confirming that the import tests were running and passing: ```text BUILD START: ['cuml-cpu-25.04.00a137-py310_250312_g153b21870_137.conda'] ... import: 'cuml' ... Resource usage statistics from testing cuml-cpu: ... Time elapsed: 0:00:10.0 ... TEST END: /tmp/conda-bld-output/linux-64/cuml-cpu-25.04.00a137-py310_250312_g153b21870_137.conda ``` Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Gil Forsyth (https://github.com/gforsyth) - Simon Adorf (https://github.com/csadorf) - Tim Head (https://github.com/betatim) URL: #6400
rapidsai#695 introduced a conda dependency on `"sklearn"`. There is no `sklearn`... the package is called `scikit-learn`. This fixes that. ## Notes for Reviewers ### How this fixes CI On rapidsai#738, @rhdong was facing the following problem. The `conda-cpp-build` jobs were producing packages with version `25.04.00a94`: ```text BUILD START: ['libcuvs-25.04.00a94-cuda11_250303_g2ffe160_94.conda', 'libcuvs-static-25.04.00a94-cuda11_250303_g2ffe160_94.conda', 'libcuvs-examples-25.04.00a94-cuda11_250303_g2ffe160_94.conda', 'libcuvs-tests-25.04.00a94-cuda11_250303_g2ffe160_94.conda'] ``` ([logs link](https://github.com/rapidsai/cuvs/actions/runs/13636442011/job/38116419908?pr=738#step:9:489)) But at test time, they were getting `25.05.00a84`: ```text ... libcuvs 25.04.00a84 cuda11_250224_ga1e0cc0_84 rapidsai-nightly ``` ([logs link](https://github.com/rapidsai/cuvs/actions/runs/13636442011/job/38125502858?pr=738#step:9:4583)) It looks like this `sklearn` dependency was the root cause, as explained in rapidsai#743 (comment) Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Ben Frederickson (https://github.com/benfred) - Bradley Dice (https://github.com/bdice) URL: rapidsai#743
#695 introduced a conda dependency on
"sklearn"
. There is nosklearn
... the package is calledscikit-learn
.This fixes that.
Notes for Reviewers
How this fixes CI
On #738, @rhdong was facing the following problem. The
conda-cpp-build
jobs were producing packages with version25.04.00a94
:(logs link)
But at test time, they were getting
25.05.00a84
:(logs link)
It looks like this
sklearn
dependency was the root cause, as explained in #743 (comment)