[Feature Request] Improve Xnnpack execution provider capabilities and structure by calling the Xnnpack subgraph API instead of the operator AI when creating Xnnpack layer/kernel

[pl121lp]

Describe the feature request

Xnnpack execution provider currently calls the Xnnpack operatior APIs directly when creating a Xnnpack layer/kernel. For example: when creating conv layer the function "xnn_create_convolution2d_nhwc_f32" is called directly from conv_base.cc:CreateXnnpackKernel().

I gather it would be better for Xnnpack execution provider to use the Xnnpack subgraph API instead by calling the "define_" functions to create an xnnpack supported layer instead. For example: xnn_define_convolution_2d() which is part of the Xnnpack subgraph API.

Note that the Xnnpack runtime/execution capabilities does not have to be used (like Executorch and LiteRT does). Just use the Xnnpack subgraph API to handle creating of the Xnnpack layers.

By taking this approach the Xnnpack execution provider can be simplified since:

1. it does not have to handle data type mapping to correct Xnnpack kernel
2. have subgraph layer handle data validation
3. benefit from any remapping to other kernel type (this is especially useful for the 1x1 convolution kernels that can be mapped to fully connected layer... the subgraph layer would take care of this).
4. Implicit handling of reshape since appropriate reshape handling function is created during "define" call. This for instance will resolve current issues with 3Dx2D Matmul
5. automatic creation of packing layers which is needed to support ARM SME use cases
6. make the Xnnpack execution provider more futureproof as Xnnpack add new capabilities

Note that when using the subgraph API to create Xnnpack layers the actual created layer (xnn_operator type) object needs to be extracted from the subgraph object as follows: subgraph->nodes[x] ->op. (xnn_subgraph->xnn_node->xnn_operator)

Describe scenario use case

See above

[pl121lp]

Does anyone in core dev team have any feedback on this request?

We do have the subgraph approach (albeit limited paths and validation) working internally, and would be interested in sharing the work if there is an interest.

However, due to the quite large architectural changes required to the XNNPACK execution provider it would be interesting to hear what the overall thoughts are about this.