Add quadrants - #34661
Conversation
|
Hi! This is the friendly automated conda-forge-linting service. I just wanted to let you know that I linted all conda-recipes in your PR ( |
|
@conda-forge/help-python-c, ready for review! The Quadrants recipe passed the complete matrix before the branch update. Current main is now merged cleanly as head 6d3a12b, with no changes under recipes/quadrants; fresh CI is running. |
|
@traversaro, this is the Quadrants split from #33293. Thanks again for offering to take a look. |
|
Thanks a lot for working on this! Quadrants is really an interesting library. I did a bit of research on this on my own on this (see https://chatgpt.com/share/6a92d65f-17b0-83eb-b8b5-f31edf899a58), so most of my inline comments are based on that. |
|
One thing that I wonder is how to handle the gpu specific dependencies. quadrants has this really nice design in which a single package variant (as one in PyPI) can target vulkan, rocm and cuda, as the required library are then loaded at runtime. However, that means that if a user just install I think a reasonable design could be to switch this recipe to be multiple output, and add the following outputs:
I think for the vulkan support, probably it is small enough that we can just depend on it directly in the This would permit easily for users to install the dependencies required to run on gpu if they want, but it would have a blind spot: a user that would just install To deal with it, we could think of having a quadrant-base package that is the one that actually contains the quandrants file, and then having the
In this way a, users that install in an environment with |
It does not provide wheels, or there is something actually preventing it to work? |
|
@traversaro, on the GPU dependency/output question: I traced the runtime loaders and tested this in a fresh environment before deciding on the output layout. The core CUDA backend only dynamically loads the NVIDIA driver ( Given that evidence, I kept |
|
@traversaro, on Linux AArch64 CUDA: it appears to be an upstream support/documentation and wheel-availability limitation, not a hard source-code guard. The CUDA backend emits PTX through LLVM and dynamically loads the host driver; it has no compile-time CUDA toolkit dependency or AArch64 exclusion. I therefore enabled CUDA for |
I see, thanks for the analysis! Give that, I agree in avoiding the quadrants-core/quadrants split, but as agreed please add quadrants-rocm and quadrants-cuda outputs, and for rocm please keep 5.2 as minimum version as documented in quadrants readme? |
|
Implemented on the current head.
Adding these outputs here, rather than immediately restructuring the generated feedstock, lets staged-recipes validate and create all three package names together. The full Linux multi-output build and tests passed locally. Both convenience packages also passed explicit package tests, and their output sets render/solve correctly on Linux x86_64, Linux AArch64, Windows x86_64, macOS x86_64, and macOS arm64. Fresh CI is running now. |
This seems to a big an upstream bug we may want to solve/patch rather then just propagate in the metadata? |
|
Agreed. I removed the metadata cap and patched the loader instead in The patch adds the CUDA 13 cuBLAS SONAME after the existing CUDA 11/12 choices, so the current PyTorch-compatible preference is preserved while I validated more than loading: a fresh environment with |
|
@traversaro, thank you for the careful reviews and package-specific guidance here—you've contributed substantially to shaping this recipe. If you're interested, would you like to be listed as a co-maintainer of the resulting No pressure at all; I would only add you after explicit confirmation. If you are interested, adding you to the staged recipe before feedstock creation would be the cleanest point to do it. |
|
All requested recipe changes are implemented on
The CUDA 13 loader fix is in I also resolved the now-outdated backend thread: the build logic uses @traversaro, when convenient, could you please take another look? |
Thanks for the asking, indeed I am interested in this package. However, I think we can discuss about this after the feedstock has been created, I would prefer to avoid approving and merging a recipe in which I am a maintainer. |
|
That makes sense. I will leave the staged recipe maintainer list unchanged and revisit this after the feedstock has been created. Thank you for being open to it. |
|
All three remaining inline requests are addressed and the threads are resolved on b67b914. Current-head CI is fully green: all 12 checks passed, including native Linux, Windows, and macOS; dedicated CUDA 12.9 and CUDA 13.0 builds; rerender; skip/status; and both linters. Azure build: https://dev.azure.com/conda-forge/84710dde-1620-425b-80d0-4cf5baca359d/_build/results?buildId=1577212 The PR description now carries this current-head evidence. As discussed, the staged recipe maintainer list remains unchanged; we can revisit co-maintenance after feedstock creation. @traversaro, when convenient, could you please take another look? |
traversaro
left a comment
There was a problem hiding this comment.
Thanks a lot, great work!
Adds Quadrants 1.3.0 as a standalone staged recipe. This is split from #33293 so its native build and GPU backends can be reviewed independently.
Backend coverage:
The Linux AArch64 CUDA backend has no architecture guard or compile-time dependency on the CUDA toolkit; it emits PTX through LLVM and loads the host NVIDIA driver at runtime. Upstream currently documents this combination as unsupported, so CI can validate compilation and packaging but not GPU execution. Windows and Intel macOS were enabled in response to review so their native CI jobs can establish whether the source build works there.
Source and dependency handling:
The build logic now lives in
build.shandbuild.bat, usesCPU_COUNT, and documents why a separate Clang executable is needed to emit embedded LLVM bitcode. AMDGPU is enabled on Linux x86_64 only because upstream's implementation currently hard-disables it on macOS and supports only Linux in its runtime loader.Tests now include imports,
pip check, backend payload checks, a CPU kernel smoke test, and selected upstream Python tests (test_smoke.pyandtest_range_loops) forced to the CPU backend.GPU runtime packaging:
quadrantsremains the actual package. Its ordinary CUDA kernels need only the host NVIDIA driver, so installing it does not automatically pull a CUDA toolkit.quadrants-cudais an opt-innoarch: genericconvenience metapackage that pins to the matching Quadrants patch release and installs cuBLAS, cuSPARSE, and cuSOLVER for the optional linear-algebra APIs. Upstream PR #896 teaches Quadrants 1.3's dynamic loader to recognize the CUDA 13 cuBLAS SONAME while preserving its existing preference for PyTorch-provided CUDA 11/12 libraries.quadrants-rocmis an opt-innoarch: genericconvenience metapackage that pins to the matching Quadrants patch release and installship-runtime-amd >=5.2, matching upstream's documented minimum; that dependency currently limits installation to Linux x86_64.Local validation:
conda-smithy recipe-lint --conda-forge recipes/quadrantsshellcheck recipes/quadrants/build.shrattler-buildLinux x86_64 build with Python 3.13pip check, CPU smoke test, and three selected upstream test cases: all passedquadrants-cudaandquadrants-rocmbuild once asnoarch: genericpackages and passed explicit package testsb67b914c12passed all 12 checks: Linux, Windows, macOS, rerender, CUDA 12.9, CUDA 13.0, skip/status, staged-recipes linter, and conda-forge linter (Azure build 1577212)git diff --checkChecklist
recipe.yamlformat.