Incorrect Gather out-of-bounds handling of TensorRT 10.16.1.11 when running ONNX Gather on GPU

[ALinrunrun]

Description

TensorRT appears to silently accept out-of-bounds indices for ONNX Gather.

For the same ONNX model and input, ONNX Runtime reports an invalid argument error because the indices are outside the valid range. TensorRT instead builds and runs the model, returning zero rows for the out-of-bounds indices.

This appears to be a TensorRT execution issue for ONNX Gather bounds checking.

Environment

TensorRT Version: 10.16.1.11

NVIDIA GPU: N/A / not detected by nvidia-smi

NVIDIA Driver Version: N/A / nvidia-smi failed

CUDA Version: N/A / nvcc not found

CUDNN Version: N/A / torch.backends.cudnn.version() returned None

Operating System: Linux 6.17.0-20-generic x86_64, glibc 2.39

Python Version (if applicable): Python 3.11.15

Tensorflow Version (if applicable): N/A

PyTorch Version (if applicable): N/A

Baremetal or Container (if so, version): Baremetal / non-Docker environment (/proc/1/cgroup: 0::/init.scope)

Additional package versions:

ONNX Version: 1.21.0
ONNX Runtime Version: 1.25.1

Relevant Files

Model link: N/A

The ONNX model is generated inline by the minimal reproducible script below.

Steps To Reproduce

Commands or scripts:

import numpy as np
import onnx
import onnxruntime as ort
from onnx import helper, TensorProto
from _trt_helper import build_engine_from_onnx, run_engine

X = helper.make_tensor_value_info("x", TensorProto.FLOAT, [6, 2])
I = helper.make_tensor_value_info("i", TensorProto.INT32, [4])
Y = helper.make_tensor_value_info("y", TensorProto.FLOAT, [4, 2])

g = helper.make_graph(
    [helper.make_node("Gather", ["x", "i"], ["y"], axis=0)],
    "g",
    [X, I],
    [Y],
)

m = helper.make_model(g, opset_imports=[helper.make_opsetid("", 18)])
m.ir_version = 10
ob = m.SerializeToString()

x = (np.arange(12, dtype=np.float32) + 1).reshape(6, 2)
idx = np.array([1, 999, -42, 3], dtype=np.int32)

ort_err = False
try:
    ort.InferenceSession(
        ob,
        providers=["CPUExecutionProvider"],
    ).run(["y"], {"x": x, "i": idx})
except Exception as e:
    ort_err = True
    print("ORT errored:", str(e)[:120])

eng, _ = build_engine_from_onnx(ob)
trt_y = run_engine(
    eng,
    {"x": x, "i": idx},
    ["y"],
    [(4, 2)],
    [np.float32],
)["y"]

print("TRT silent  :", trt_y.tolist())

assert ort_err
assert (trt_y[1] == 0).all() and (trt_y[2] == 0).all()

Have you tried the latest release?: Yes, reproduced with TensorRT 10.16.1.11.

Attach the captured .json and .bin files from TensorRT's API Capture tool if you're on an x86_64 Unix system Not attached. The issue is reproducible from the self-contained Python script above.

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

ONNX Runtime rejects the same model input at runtime because the Gather indices are out of bounds. TensorRT runs the model silently and returns zero rows for the invalid indices.

Actual output:

ORT errored: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : indices element out of data bounds, idx=999 must be within the inclusive range [-6,5]
TRT silent  : [[3.0, 4.0], [0.0, 0.0], [0.0, 0.0], [7.0, 8.0]]

TensorRT silently returns [0.0, 0.0] for the invalid indices 999 and -42, while ONNX Runtime reports that the indices are out of bounds.