[GlobalOpt] Support Img2Col Transformation for Conv2D Including Quantized Types

[phoebesv]

Move the Conv2D to Img2Col transformation pass from Preprocessing to GlobalOptimization phase. This allows the pass to run after LinalgQuantizedConvToConvPass, enabling quantized convolutions to benefit from the img2col transformation.

[MaheshRavishankar]

Thanks, but we cannot have the global optimization depend on preprocessing pass pipelines. Adding Global optimization to run some pre-processing pass pipeline is an inversion of dependencies. Could you describe what you are trying to see if there is a different way to approach this. In general using im2col at this level is not the best way to handle convs.

[phoebesv]

@MaheshRavishankar Thanks for the clarification. We observed that ConvertConv2DToImg2ColPass only handles a limited set of conv ops, so quantized models using QuantizedConv are not covered. While QuantizedConvToConvPass can handle this, it currently resides in GlobalOptimization.

Our goal is to understand the intended lowering path for QuantizedConv and whether there is a more appropriate place to handle this without introducing dependency inversion. We’re open to alternative approaches and would appreciate guidance on the recommended direction.

[MaheshRavishankar]

@MaheshRavishankar Thanks for the clarification. We observed that ConvertConv2DToImg2ColPass only handles a limited set of conv ops, so quantized models using QuantizedConv are not covered. While QuantizedConvToConvPass can handle this, it currently resides in GlobalOptimization.

Our goal is to understand the intended lowering path for QuantizedConv and whether there is a more appropriate place to handle this without introducing dependency inversion. We’re open to alternative approaches and would appreciate guidance on the recommended direction.

I think one option here is to use the iree-preprocessing-pass-pipeline option to stitch together the exact sequence of passes that you want to call before the iree-compile pass pipeline kicks in. This should be able to call any passes registered in IREE.

Broadly if you have a proposal for moving some of the pre-processing pass pipelines into IREE global optimization phase we can definitely iterate on that. In this particular case, converting a convolution using im2col this early is typically seen as an anti-pattern due to the high replication this entails. We would prefer to try to handle the convolutions more natively and fix the backend code-generator to handle that better. That is a lot of work, so if you want to use the im2col approach as a stop-gap then I think the iree-preprocessing-pass-pipeline should get you what you need.

[phoebesv]

I think one option here is to use the iree-preprocessing-pass-pipeline option to stitch together the exact sequence of passes that you want to call before the iree-compile pass pipeline kicks in. This should be able to call any passes registered in IREE.

Broadly if you have a proposal for moving some of the pre-processing pass pipelines into IREE global optimization phase we can definitely iterate on that. In this particular case, converting a convolution using im2col this early is typically seen as an anti-pattern due to the high replication this entails. We would prefer to try to handle the convolutions more natively and fix the backend code-generator to handle that better. That is a lot of work, so if you want to use the im2col approach as a stop-gap then I think the iree-preprocessing-pass-pipeline should get you what you need.

@MaheshRavishankar Thank you for the suggestion. We agree that iree-preprocessing-pass-pipeline is useful for experimentation and prototyping.
While exploring this approach, we found that using it as a permanent solution has some practical limitations:

• Users must manually configure the pass sequence and understand internal dependencies
• The pipeline is fragile to changes in IREE's internal pass ordering
• It doesn't solve the ordering issue with LinalgQuantizedConvToConvPass (which runs in GlobalOptimization)

We're thinking that moving the im2col pass to GlobalOptimization with an opt-in flag could be a good fit for our scenario while avoiding the anti-pattern concern for backends that don't need it.

We'll prepare a proposal with more details and rationale. Happy to iterate based on your feedback.

[phoebesv]

Thanks for taking the time to review! @MaheshRavishankar @benvanik @qedawkins @IanWood1 Please feel free to let me know if there's anything I can do to help with the next steps.

[phoebesv]

Rebased only.

[rednoah91]

@MaheshRavishankar Could anyone help merge this PR : )?

[MaheshRavishankar]

Ok, let me check if CI passes. cc @hanhanW as well to merge if everything passes.

[hanhanW]

I can help trigger CI and merge it, please fix the failures.

[phoebesv]

Hi @hanhanW, the pre-commit workflow failure seems to be caused by a temporary GitHub Actions service issue during the setup phase.

Getting action download info
Failed to resolve action download info. Error: Service Unavailable
Retrying in 21.442 seconds
Failed to resolve action download info. Error: Service Unavailable
Retrying in 21.695 seconds
Error: Failed to resolve action download info.
Error: Failed to resolve action download info.

Re-running the workflow might resolve the problem.

[hanhanW]

Hi @hanhanW, the pre-commit workflow failure seems to be caused by a temporary GitHub Actions service issue during the setup phase.

Getting action download info
Failed to resolve action download info. Error: Service Unavailable
Retrying in 21.442 seconds
Failed to resolve action download info. Error: Service Unavailable
Retrying in 21.695 seconds
Error: Failed to resolve action download info.
Error: Failed to resolve action download info.

Re-running the workflow might resolve the problem.

ooops, it ran for 2 mins, so I did not click into the link. Re-running the jobs now.

[hanhanW]

Here is the new log from lint: https://github.com/iree-org/iree/actions/runs/22607978256/job/65965929632?pr=23278

You can run build_tools/bazel_to_cmake/bazel_to_cmake.py to fix cmake files.

[phoebesv]

Here is the new log from lint: https://github.com/iree-org/iree/actions/runs/22607978256/job/65965929632?pr=23278

You can run build_tools/bazel_to_cmake/bazel_to_cmake.py to fix cmake files.

@hanhanW Formatting fix applied. Thanks!