ConvInteger segfaults when x_zero_point is the empty string

[mcollinswisc]

Describe the issue

ConvInteger has two optional inputs: for x_zero_point and w_zero_point (in that order).

The empty string may be used to omit optional inputs:
https://github.com/onnx/onnx/blob/97a7fd9b033d4dca5d945e6b9210d0a165b2f537/onnx/common/ir_pb_converter.cc#L261
so in this case one could set a w_zero_point but not an x_zero_point by passing the empty string for that input name.

This currently causes a segfault in ONNXRuntime. I believe the CPU provider is not checking whether x_zero_point is empty:

onnxruntime/onnxruntime/core/providers/cpu/quantization/conv_integer.cc

Lines 42 to 46 in 101353c

	if (num_inputs >= 3) {
	const auto* X_Zero_Point = context->Input<Tensor>(2);
	ORT_ENFORCE(IsScalarOr1ElementVector(X_Zero_Point), "Must be a scalar or 1D tensor or size 1.");
	input_offset = *(X_Zero_Point->Data<uint8_t>());
	}

To reproduce

pip3 install numpy==1.26.4 onnx==1.16.2 onnxruntime==1.20.1

import numpy as np
import onnx
import onnx.numpy_helper
import onnxruntime

input_info = onnx.helper.make_tensor_value_info(
    name="X",
    shape=["batch", 1, 8, 8],
    elem_type=onnx.TensorProto.UINT8,
)
output_info = onnx.helper.make_tensor_value_info(
    name="Y",
    shape=["batch", 1, 6, 6],
    elem_type=onnx.TensorProto.INT32,
)

W = [
    [-1, 0, 1],
    [-2, 0, 2],
    [-1, 0, 1],
]
W = (np.array(W, np.int32) + 128).astype(np.uint8)
W = np.expand_dims(W, (0, 1))  # Singleton input & output channels
W = onnx.numpy_helper.from_array(W, name="W")

# x_zero_point = np.array(0, dtype=np.uint8)
# x_zero_point = onnx.numpy_helper.from_array(x_zero_point, name="x_zero_point")

w_zero_point = np.array(128, dtype=np.uint8)
w_zero_point = onnx.numpy_helper.from_array(w_zero_point, name="w_zero_point")

conv_integer = onnx.helper.make_node(
    op_type="ConvInteger",
    name="conv",
    # inputs=["X", "W", "x_zero_point", "w_zero_point"],
    inputs=["X", "W", "", "w_zero_point"],
    outputs=["Y"],
    kernel_shape=[3, 3],
    auto_pad="VALID",
)
graph = onnx.helper.make_graph(
    nodes=[conv_integer],
    # initializer=[W, x_zero_point, w_zero_point],
    initializer=[W, w_zero_point],
    inputs=[input_info],
    outputs=[output_info],
    name="test_graph",
)
model = onnx.helper.make_model(graph)
onnx.checker.check_model(model)

sess_opts = onnxruntime.SessionOptions()
# https://github.com/microsoft/onnxruntime/issues/16105#issuecomment-2453730904
sess_opts.add_session_config_entry("session.x64quantprecision", "1")
sess = onnxruntime.InferenceSession(
    model.SerializeToString(),
    sess_opts,
    provides=["CPUExecutionProvider"],
)

test_input = [
    [0, 0, 0, 0, 1, 1, 1, 1],
    [0, 0, 0, 0, 1, 1, 1, 1],
    [0, 0, 0, 0, 1, 1, 1, 1],
    [0, 0, 0, 0, 1, 1, 1, 1],
    [0, 0, 0, 0, 8, 8, 8, 8],
    [0, 0, 0, 0, 8, 8, 8, 8],
    [0, 0, 0, 0, 8, 8, 8, 8],
    [0, 0, 0, 0, 8, 8, 8, 8],
]
test_input = np.array(test_input, dtype=np.uint8)
test_input = np.reshape(test_input, [1, 1, 8, 8])
result = sess.run(["Y"], {"X": test_input})[0]

print(np.squeeze(result))

First result here comes from the commented-out workaround. The current code above produced the segfault:

Urgency

Not really urgent: passing a zero scalar initializer for x_zero_point is a straightforward and easy workaround.

Mostly just unfortunate for users to see a segfault & have to track it down.

Platform

Linux

OS Version

6.11.0-18-generic

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.20.1

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response