cuda.bindings: support multiple CTK release lines on main by rwgk · Pull Request #2737 · NVIDIA/cuda-python

rwgk · 2026-08-31T06:35:48Z

Description

Closes #1199.

This continues and supersedes #2675. That draft was automatically closed when
its temporary pull-request/2467 base was deleted after #2467 merged. Its head
branch is not maintainer-writable, so this replacement preserves the reviewed
history on a writable fork and targets current main directly.

What changes

Import the maintained CUDA 12.9 bindings tree from
238955935bd903ac72817c0dfdfe4f6a54ee6bb1:cuda_bindings as
cuda_bindings_12/.
Extend the selective-CI workplan from merged PR ci: activate dependency-aware package builds and tests #2467 with cu12 and cu13
variants while retaining its aggregate module gates.
Build, test, document, and release the matching CUDA 12.9 and CUDA 13
bindings/metapackage pairs from main.
Treat the historical 12.9.x branch as a read-only release record. Routine
and emergency CUDA 12 work happens on main; CI and releases no longer
source package artifacts from that branch.
Carry applicable bindings fixes that landed on main after the imported
snapshot, including free-threading-safe parameter packing, Cython/packaging
corrections, option and handle-helper fixes, and the generated NVML
memoryview fix.
Advance CUDA 12 development versions through setuptools-scm after a stable
v12.9.8 tag instead of forcing 12.9.8.dev0 forever.

Selective CI behavior

Change	Bindings built/tested	Matching `cuda-python` pair	Other bindings line
`cuda_bindings_12/**` source	CUDA 12	CUDA 12	reused when eligible
`cuda_bindings/**` source	CUDA 13	CUDA 13	reused when eligible
shared bindings consumers or CI infrastructure	CUDA 12 and 13	CUDA 12 and 13 as needed	n/a

Per-major decisions live under
modules.{bindings,core,python}.variants.{cu12,cu13}. Core's two build halves
are rebuilt together when either bindings line changes, so the merged core
wheel always comes from one source revision. Stable v12.9.* tags select the
CUDA 12 pair; v13.* tags select the CUDA 13 pair.

Explicit reviewer decisions

Please review and explicitly accept or reject both of these policies:

The two full package roots, cuda_bindings/ and cuda_bindings_12/, are an
intentional transitional design. main is the sole active source of truth,
and cuda_bindings_12/MAINTENANCE.md documents generation provenance and
maintenance rules, while ci/cuda-bindings-shared-files.json enforces byte
equality for the explicitly shared subset. This is not precedent for
staging multiple same-major CTK minors such as 13.4 and 13.5 in full
duplicated roots.
The historical 12.9.x branch receives no further backports. Routine and
emergency CUDA 12 fixes are made on main and applied to every applicable
bindings root there.

Generation provenance

Cybind commit 95d8bb525de46a9ff7ae40d759a98cbe50cf8391
reproduces 52 generated paths from the imported snapshot, including all 40
generated paths changed by 12.9.x: Update bindings to latest generator release #2604. Complete-root provenance remains partial
because that revision does not reproduce the snapshot's legacy runtime/v2
storage layout.

The later NVML memoryview fix is reproduced byte-for-byte from cybind commit
6def52ca508c9e14ef67f4ce26a0c677f3fbad72 with Doxygen 1.17.0:

CUDA_PATH=/usr/local/cuda-12.9 python -m cybind \
  --ctk-target-version 12.9 \
  --generate nvml \
  --output-dir /tmp/cuda-bindings-12-generated \
  --jobs 1

Validation

pre-commit run --all-files passes with lychee,
check-precommit-installed, and secret-scan-trufflehog explicitly skipped
python3 -m unittest ci/tools/tests/test_compute_ci_plan.py (6 tests)
CUDA 12 SCM helper tests (3 tests), covering pre-release fallback, the exact
stable tag, and post-tag behavior
shared-file checker, generated-file seals, Python/JavaScript/shell syntax,
JSON validation, and final history/tree checks
the prior cuda.bindings: build 12.9 and 13.x selectively from main #2675 head passed CUDA 12 source-build/import, Cython-extension,
dependency-metadata, and environment-routing smoke checks; this replacement
reruns the full current matrix

Checklist

New or existing tests cover these changes.
The documentation is up to date with these changes.

Moon migration belongs in PR NVIDIA#2659. Restore the pre-Moon selective-CI planner and workflows from 727ef59.

copy-pr-bot · 2026-08-31T06:35:52Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

rwgk · 2026-08-31T06:36:37Z

/ok to test b87d0a1

github-actions · 2026-08-31T06:56:51Z

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-2737/
https://nvidia.github.io/cuda-python/pr-preview/pr-2737/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-2737/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-2737/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

mdboom

I started to comment on some individual things, but then decided to stop because I think there is a more fundamental change that needs to be made across this whole PR (and then I'm happy to come back and review further).

/Today/ the "current" version is 13, and the backport version is 12. But at some point in the future that will switch to 14 and 13. This pervasively hardcodes those version numbers all over this codebase, especially in CI, but in a bunch of the release scripts as well, and even the cuda_bindings_12 directory name as indicators of current vs. backport.

Instead, we should use the config we already have in versions.yml and use that to drive the numbers everywhere. That way when it's time to move on, all that should be required is updating versions.yml, and copying/overwriting the existing cuda_bindings to cuda_bindings_backport (or whatever we want to call it), and move on. I'm sure there are many details I'm missing, but that should be the goal and design -- it would be preferable to reduce it to as close to that as possible. The problem with this as-is is that there are hundreds of context-sensitive places that would need to be updated to do that update -- we are creating a massive pile of technical debt to pay later. I'm sure an agent might get that X% correct, but I always think it's better to engineer for flexibility, especially for something we know will happen. If versions.yml (which requires using yq to parse etc.) makes this too difficult, we could explore a simple VARIABLE=value format which would parse as both bash variables and Python variables and probably be more convenient to use from the many places it is needed. There are really only two actual values in versions.yml today, so that should be fine.

I'm also a little concerned (without any testing-based evidence) that this will break when we tag the same commit with v13.x.y and v12.x.y, which will be the common case, in fact, IMHO, one of the real benefits of moving to this approach. We should get an agent to do a thorough investigation of that use case and make sure it is covered. Ideally, it would be nice for a single release run to do both releases simultaneously but it's not a deal breaker if it still requires kicking off two runs.

Also what is this (from the agent's PR description):

The later NVML memoryview fix is reproduced byte-for-byte from cybind commit
6def52ca508c9e14ef67f4ce26a0c677f3fbad72 with Doxygen 1.17.0:

If there is something like this that wasn't backported, let's deal with that separately so it's not an unrelated tag-along to this PR.

Also a note for future agent reviewers of this PR: The interesting part of this PR is the part outside of the cuda_bindings_backport or cuda_bindings_12 directory. Those are just direct copies from the 12.9.x branch, and any differences between that and the cuda_bindings directory are likely intentional. When reviewing, focus on the scaffolding / CI / overall structure.

mdboom · 2026-08-31T13:52:16Z

@@ -1,19 +1,15 @@
-# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.


It seems like this whole workflow because obsolete. I could imagine some automation to copy changes from the 13.x directory to the 12.x, but this workflow won't do that --- At a minimum, I don't see how it continues to work after this change.

mdboom · 2026-08-31T13:58:05Z

+      BUILD_PYTHON_CU12: ${{ inputs.workplan == '' || fromJSON(inputs.workplan).modules.python.variants.cu12.needs_build }}
+      BUILD_PYTHON_CU13: ${{ inputs.workplan == '' || fromJSON(inputs.workplan).modules.python.variants.cu13.needs_build }}


This naming is confusing, since it relates to building cuda-python not python. Maybe BUILD_CUDA_PYTHON_CU1?? (I know that would be inconsistent, but in this case I think it's worth it).

mdboom · 2026-08-31T14:22:06Z

+    "cuda/bindings/__init__.pxd",
+    "cuda/bindings/__init__.py",
+    "cuda/bindings/_internal/__init__.py",
+    "cuda/bindings/_internal/_fast_enum.py",


This is a generated file. Probably doesn't belong here.

mdboom · 2026-08-31T14:22:37Z

+    "cuda/bindings/__init__.py",
+    "cuda/bindings/_internal/__init__.py",
+    "cuda/bindings/_internal/_fast_enum.py",
+    "cuda/bindings/_internal/utils.pxd",


This file isn't generated but is cargo-culted from cybind. Also /could/ be closely tied to the generated files, so maybe doesn't belong here.

mdboom · 2026-08-31T14:24:25Z

These changes seem to regress the diagram?

Made this: #2740

mdboom · 2026-08-31T14:34:00Z

@@ -19,6 +19,7 @@ dependencies:
  - sphinx-copybutton
  - myst-nb
  - enum_tools
+  - furo


Reason for this change?

mdboom · 2026-08-31T14:41:57Z

+
+
+if __name__ == "__main__":
+    sys.exit(main())


I think it's fine for now that we copy the directory wholesale.

Eventually (after this PR), it would be better to keep files that we want to be shared shared in some other way (actual shared directories, or symlinks) rather than confirming that files are identical -- it will get annoying.

kkraus14 and others added 25 commits August 13, 2026 21:37

ci: activate dependency-aware package builds and tests

9cf09a9

ci: use paths-filter for selective CI planning

da7ebee

ci: refine selective path classification

42fe412

ci: compute selective workplan in Python

eaed263

ci: split selective tests by platform

5c798a7

ci: harden prior artifact downloads

e5c3610

ci: defer artifact download retries

1ccaaef

ci: simplify workplan path classification

ca29abd

ci: simplify workplan plumbing

44c6a3a

Merge main into agent/selective-ci-orchestration

fba3b4d

Derive reusable artifact platforms from test matrix

727ef59

ci: route selective CI through Moon

30da891

ci: separate editable installs from Moon tests

139c2c4

Revert Moon CI migration from PR NVIDIA#2467

ceaae51

Moon migration belongs in PR NVIDIA#2659. Restore the pre-Moon selective-CI planner and workflows from 727ef59.

cuda.bindings: build CUDA 12 and 13 from main

5c5b4bf

ci: isolate cuda-core source builds by CUDA major

dc9aee4

test: use Ruff security suppressions in CUDA 12 tests

ca252cf

build: align CUDA 12 cibuildwheel targets

e175708

bench: support CUDA 12 context creation

a65ccea

ci: use blobless source-smoke checkout

c980cc9

Merge main into CUDA 12/13 selective CI work

5d8bffa

cuda.bindings: sync CUDA 12 maintenance fixes

b506585

ci: guard shared CUDA bindings maintenance

ee46c3e

ci: advance CUDA 12 versions after release

bfef374

docs: avoid generated-file marker in maintenance guide

b87d0a1

rwgk added this to the cuda.bindings 13.5.0 & 12.9.10 milestone Aug 31, 2026

rwgk added enhancement Any code-related improvements CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module labels Aug 31, 2026

rwgk self-assigned this Aug 31, 2026

mdboom requested changes Aug 31, 2026

View reviewed changes

mdboom reviewed Aug 31, 2026

View reviewed changes

rwgk changed the title ~~cuda.bindings: build 12.9 and 13.x selectively from main~~ cuda.bindings: support multiple CTK release lines on main Aug 31, 2026

Navigation Menu

Uh oh!

cuda.bindings: support multiple CTK release lines on main - #2737

cuda.bindings: support multiple CTK release lines on main#2737
rwgk wants to merge 25 commits into
NVIDIA:mainfrom
rwgk:agent/cuda-bindings-12-on-main

rwgk commented Aug 31, 2026

Uh oh!

copy-pr-bot Bot commented Aug 31, 2026

Uh oh!

rwgk commented Aug 31, 2026

Uh oh!

github-actions Bot commented Aug 31, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

mdboom left a comment •

edited

Loading

Uh oh!

mdboom Aug 31, 2026

Uh oh!

mdboom Aug 31, 2026

Uh oh!

mdboom Aug 31, 2026

Uh oh!

mdboom Aug 31, 2026

Uh oh!

mdboom Aug 31, 2026

Uh oh!

mdboom Aug 31, 2026

Uh oh!

mdboom Aug 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -1,19 +1,15 @@
		# SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
		# SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.

		BUILD_PYTHON_CU12: ${{ inputs.workplan == '' \|\| fromJSON(inputs.workplan).modules.python.variants.cu12.needs_build }}
		BUILD_PYTHON_CU13: ${{ inputs.workplan == '' \|\| fromJSON(inputs.workplan).modules.python.variants.cu13.needs_build }}

Uh oh!

Conversation

rwgk commented Aug 31, 2026

Description

What changes

Selective CI behavior

Explicit reviewer decisions

Generation provenance

Validation

Checklist

Uh oh!

copy-pr-bot Bot commented Aug 31, 2026

Uh oh!

rwgk commented Aug 31, 2026

Uh oh!

github-actions Bot commented Aug 31, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

mdboom left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdboom Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

mdboom Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

mdboom Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

mdboom Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

mdboom Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

mdboom Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

mdboom Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mdboom left a comment •

edited

Loading