feat(launch): support programmatic dependency ordering - #228
Open
lucifer1004 wants to merge 1 commit into
Open
lucifer1004 wants to merge 1 commit into
lucifer1004 wants to merge 1 commit into
Conversation
Carry CUDA launch attributes through cuTile-generated launchers so dependent kernels can use programmatic stream serialization. Keep ordinary launches unchanged while exposing the opt-in PDL path used by prefill MoE.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
CUDA 12.0+ programmatic dependent launch (PDL) lets a dependent grid start while the preceding grid in the same stream is still running, removing the full-grid serialization gap between back-to-back kernels. There is currently no way to attach launch attributes to cuTile-generated kernel launches, so consumers cannot opt into
CU_LAUNCH_ATTRIBUTE_PROGRAMMATIC_STREAM_SERIALIZATION.What this adds
LaunchAttributesvalue (starting withprogrammatic_stream_serialization: bool) andlaunch_kernel_with_attributes, which encodes the attributes into aCUlaunchConfigand launches viacuLaunchKernelEx.CUlaunchAttributestays opaque in the bindgen output across toolkit revisions, so the attribute is encoded through its documented C layout (4-byteidat offset 0, 8-byte-aligned payload union at offset 8) — noted in a comment at the encoder.AsyncKernelLaunch::set_launch_attributes. Launches with default attributes keep using the plainlaunch_kernelpath unchanged; only non-default attributes take thecuLaunchKernelExpath..launch_attributes(...)builder method (declared on theTileKerneltrait), threading the value through to the async launch.Example
The dependent kernel is responsible for establishing the device-side ordering edge (e.g.
cudaGridDependencySynchronizeor an application-level ready-flag protocol) before consuming prerequisite output — the attribute only permits the overlap.Tests
Added
cutile/tests/gpu/launch_attributes.rs: launches a basic add kernel withprogrammatic_stream_serialization: trueand verifies results, exercising thecuLaunchKernelExencoding end-to-end. Passes on an SM120 GPU; the default-attribute path is unchanged and covered by the existing launch tests.