Skip to content

Convolution shape and GEMM arithmetic overflow int32 for large or extreme-parameter inputs #3611

@uqio

Description

@uqio

While building a Rust binding over mlx-c, a static review of the convolution path surfaced several int (int32) quantities that overflow for large or extreme-parameter convolutions. Signed-integer overflow is UB in C++, and in practice yields wrong shapes/sizes. Reporting in case these are worth widening to int64_t or guarding with a raised error.

Quantities computed in int:

  1. Forward output shape (conv_out_shape): per axis, dilation * (kernel - 1), the effective input input_dilation * (in - 1) + 1, and (effective_in + pad_lo + pad_hi - dilated_kernel) / stride + 1. A dilation/padding/stride near INT_MAX overflows before validation.

  2. Transposed prelude (conv_transpose_general): computes 1 + dilation * (weight_dim - 1), 2 * padding, and (in - 1) * stride + ... over the weight and parameters before the nested conv_general validates the rank, so an output_padding/dilation near INT_MAX overflows even for a tiny input.

  3. Negative-padding normalization: slicing every dimension forms dim + 1 / 0 - pad_lo in int32, overflowing for a dimension near INT_MAX.

  4. Metal implicit GEMM (implicit_gemm_conv_2D_gpu and the 3D path): implicit_M = N * oH * oW is an int32 product of the output spatial dims. For a conv whose batch x output-spatial exceeds INT_MAX this overflows — e.g. conv2d on input [1, 46341, 46341, 1] gives implicit_M = 46341 * 46341 = 2147488281 > 2147483647. The K dimension product(weight_spatial) * channels_per_group and the lcm(input_dilation, stride) jump-table sizes are similarly int32.

Most need either multi-GB tensors or extreme parameters, so they're unlikely in normal use, but they are reachable from the safe API and are UB rather than a clean error. Would you accept widening these to int64_t (or adding overflow checks)? Happy to help pin down exact locations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions