Skip to content

CUDA: support non-contiguous rows in L2_NORM#26026

Open
MagicalFlames wants to merge 1 commit into
ggml-org:masterfrom
MagicalFlames:cuda-l2norm-noncontig-row
Open

CUDA: support non-contiguous rows in L2_NORM#26026
MagicalFlames wants to merge 1 commit into
ggml-org:masterfrom
MagicalFlames:cuda-l2norm-noncontig-row

Conversation

@MagicalFlames

Copy link
Copy Markdown

Overview

The L2_NORM CUDA kernel assumed contiguous rows: it read x[col] directly and was guarded by GGML_ASSERT(nb00 == ts0) plus ggml_is_contiguous_rows in supports_op. As a result, test-backend-ops cases with noncontig_rows=1 were reported as not supported.

This PR adds an inner-row stride to the kernel so it indexes elements via x[col*stride_col], passes the stride down from ggml_cuda_op_l2_norm, and relaxes the support check for L2_NORM.

Additional information

Testing with test-backend-ops test -o L2_NORM on an RTX 3050 (sm_86, CUDA 13.0) shows improvement from 20/20 tests (with 10 noncontig_rows cases skipped as not supported) to 30/30 passed, including all noncontig_rows=1 cases.

Requirements

@MagicalFlames
MagicalFlames requested a review from a team as a code owner July 23, 2026 03:27
@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning CUDA Related to the CUDA backend labels Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CUDA Related to the CUDA backend ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant