This pull request expands support for several ONNX operator fusion and optimization passes to include newer operator set (opset) versions, especially opset 22 and above. This ensures that the optimizer can handle models using the latest ONNX specifications, improving compatibility and optimization coverage.

The most important changes are:

Expanded opset version support for key operators:

• Updated checks in various fusion and optimization passes (e.g., Conv, Dropout, HardSigmoid, Transpose, Cast, IsInf, Reshape) to include newer opset versions such as 19, 20, 21, 22, 23, 24, and 25, where applicable. This affects fusion passes like Conv-Add, Conv-BN, Conv-Mul, Conv-Activation, BiasDropout, FastGelu, LayerNorm, Dropout elimination, GemmTranspose, IsInfReduceSum, and more. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25]

Fusion rule and selector registration updates:

• Modified fusion rule registration to account for the new opset versions, ensuring that pattern matchers and selectors for fusions like Conv+Activation and Conv+Add+Activation are registered for the expanded opset ranges. [1] [2]

Operator-specific compatibility logic:

• Enhanced logic for supported activation functions and other ops to include additional opset versions, such as Elu, HardSigmoid, LeakyRelu, Selu, Softplus, Softsign, ThresholdedRelu, and others, improving the range of fusable patterns.

These changes collectively improve the optimizer's ability to process and optimize models using the latest ONNX operator versions, making the system more robust and future-proof.

For LLM related patterns, attention fusion under onnxruntime/core/optimizer is not updated with latest opset version, as the patterns are legacy, and we are more relying on https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/python/tools/transformers and https://github.com/microsoft/onnxscript/tree/main/onnxscript/rewriter