Skip to content

feat(launch): support programmatic dependency ordering - #228

Open
lucifer1004 wants to merge 1 commit into
NVlabs:mainfrom
lucifer1004:pr/programmatic-launch-attributes
Open

lucifer1004 wants to merge 1 commit into
NVlabs:mainfrom
lucifer1004:pr/programmatic-launch-attributes

Conversation

@lucifer1004

Copy link
Copy Markdown
Contributor

Motivation

CUDA 12.0+ programmatic dependent launch (PDL) lets a dependent grid start while the preceding grid in the same stream is still running, removing the full-grid serialization gap between back-to-back kernels. There is currently no way to attach launch attributes to cuTile-generated kernel launches, so consumers cannot opt into CU_LAUNCH_ATTRIBUTE_PROGRAMMATIC_STREAM_SERIALIZATION.

What this adds

  • cuda-core: a composable LaunchAttributes value (starting with programmatic_stream_serialization: bool) and launch_kernel_with_attributes, which encodes the attributes into a CUlaunchConfig and launches via cuLaunchKernelEx.
    • CUlaunchAttribute stays opaque in the bindgen output across toolkit revisions, so the attribute is encoded through its documented C layout (4-byte id at offset 0, 8-byte-aligned payload union at offset 8) — noted in a comment at the encoder.
  • cuda-async: AsyncKernelLaunch::set_launch_attributes. Launches with default attributes keep using the plain launch_kernel path unchanged; only non-default attributes take the cuLaunchKernelEx path.
  • cutile: generated kernel launchers gain a .launch_attributes(...) builder method (declared on the TileKernel trait), threading the value through to the async launch.

Example

kernel(args).grid(grid)
    .launch_attributes(LaunchAttributes {
        programmatic_stream_serialization: true,
    })
    .async_on(&stream)?;

The dependent kernel is responsible for establishing the device-side ordering edge (e.g. cudaGridDependencySynchronize or an application-level ready-flag protocol) before consuming prerequisite output — the attribute only permits the overlap.

Tests

Added cutile/tests/gpu/launch_attributes.rs: launches a basic add kernel with programmatic_stream_serialization: true and verifies results, exercising the cuLaunchKernelEx encoding end-to-end. Passes on an SM120 GPU; the default-attribute path is unchanged and covered by the existing launch tests.

Carry CUDA launch attributes through cuTile-generated launchers so dependent kernels can use programmatic stream serialization. Keep ordinary launches unchanged while exposing the opt-in PDL path used by prefill MoE.
@copy-pr-bot

copy-pr-bot Bot commented Aug 20, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant