Skip to content

[TorchAO] [OVQuantizer] Update Torch AO Dependency and Refactor Executorch Tests#3842

Open
anzr299 wants to merge 78 commits intoopenvinotoolkit:developfrom
anzr299:an/executorch_tests
Open

[TorchAO] [OVQuantizer] Update Torch AO Dependency and Refactor Executorch Tests#3842
anzr299 wants to merge 78 commits intoopenvinotoolkit:developfrom
anzr299:an/executorch_tests

Conversation

@anzr299
Copy link
Collaborator

@anzr299 anzr299 commented Jan 15, 2026

Changes

Include tests for executorch, OVQuantizer
Remove OVQuantizer from NNCF
Move the tests in appropriate directory

Reason for changes

Migrate from torch.ao -> torchao

Notable differences in torch.ao and torchao implementation which affect test references:

  1. Constant folding in torch.ao produced the new constant with the name _frozen_param{index}. In the new torchao implementation it keeps the same name as the node being replaced with.
  2. The old deprecated XNNPACK Quantizer was being used before which is now changed to point to the new XNNPACKQuantizer implementation in https://github.com/pytorch/executorch/tree/main/backends/xnnpack/quantizer which now explicitly quantizes the conv_transpose2D operator and its weights.

Reference graphs changed due to above:

  1. tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/mobilenet_v3_small.dot
  2. tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/resnet18.dot
  3. tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/swin_v2_t.dot
  4. tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/synthetic_transformer.dot
  5. tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/unet.dot
  6. tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/vit_b_16.dot
  7. tests/executorch/data/ao_export_quantization_OpenVINOQuantizer/yolo11n_sdpa_block.dot
  8. tests/executorch/data/XNNPACKQuantizer/unet.dot (Due to difference Number 2)
  9. tests/executorch/data/XNNPACKQuantizer/unet_ref_qconfig.json ((Due to difference Number 2))

Related tickets

CVS-176783

Tests

PTQ Conformance - PTQ-789 - Pass
Examples - https://github.com/openvinotoolkit/nncf/actions/runs/21204888584 - Pass
WC conformance - https://github.com/openvinotoolkit/nncf/actions/runs/21204893707 - Pass

pytest tests/executorch results:
image
All tests in tests/executorch pass

@anzr299 anzr299 requested a review from a team as a code owner January 15, 2026 15:33
@anzr299 anzr299 marked this pull request as draft January 15, 2026 16:44
@github-actions github-actions bot added the API Public API-impacting changes label Jan 15, 2026
@github-actions github-actions bot added NNCF PT Pull requests that updates NNCF PyTorch NNCF PTQ Pull requests that updates NNCF PTQ labels Jan 15, 2026
@MaximProshin
Copy link
Collaborator

MaximProshin commented Jan 23, 2026

Agreed to merge it after the CF to mitigate any risks with a new dependency. @anzr299 @AlexanderDokuchaev

# Pytorch
torch==2.9.0
torchvision==0.24.0
torchao==0.14.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aamir, as we discussed it with Alexander Suslov, the main idea was to get rid of torch.ao and do not use any external dependencies (like torchao)
The task is to remove torch.ao imports and replace them with code in the nncf codebase, no external dependencies

Copilot AI review requested due to automatic review settings February 23, 2026 05:23
@github-actions github-actions bot removed the API Public API-impacting changes label Feb 23, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the codebase from using torch.ao to torchao as a dependency, updates the OpenVINOQuantizer to use the executorch implementation, and refactors the test structure for executorch tests. The migration addresses breaking changes in PyTorch's quantization API and aligns with the newer torchao package architecture.

Changes:

  • Migration of all torch.ao.quantization imports to torchao.quantization.pt2e
  • Removal of NNCF's internal OpenVINOQuantizer implementation in favor of executorch's version
  • Introduction of quantizer_params alongside compression_params for better parameter separation
  • Update of test reference files to reflect behavior changes in torchao vs torch.ao
  • Addition of torchao and executorch dependencies across requirements files

Reviewed changes

Copilot reviewed 58 out of 100 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/torch/requirements.txt Added torchao dependency
tests/torch/fx/test_model_transformer.py Updated imports from torch.ao to torchao
tests/post_training/test_quantize_conformance.py Added quantizer_params support and printing
tests/post_training/requirements.txt Added torchao and executorch, updated scikit-learn version
tests/post_training/pipelines/image_classification_base.py Updated imports and refactored parameter handling
tests/post_training/pipelines/base.py Added quantizer_params parameter throughout pipeline
tests/post_training/model_scope.py Added add_quantizer_compression_params function
tests/executorch/test_quantizer_compression.py Updated test data directory paths
tests/executorch/test_ptq.py Updated imports and test references
tests/executorch/requirements.txt Updated dependency versions
tests/executorch/data/* Multiple new reference files for test data
tests/cross_fw/shared/helpers.py Added torchao to backend packages
src/nncf/torch/quantization/strip.py Updated observer imports
src/nncf/quantization/algorithms/min_max/torch_fx_backend.py Updated observer imports
src/nncf/experimental/torch/fx/transformations.py Updated imports and comments
src/nncf/experimental/torch/fx/quantization/quantizer/torch_ao_adapter.py Updated imports and documentation
src/nncf/experimental/torch/fx/quantization/quantizer/openvino_quantizer.py File removed
src/nncf/experimental/torch/fx/quantization/quantizer/openvino_adapter.py Updated to use executorch's OpenVINOQuantizer
src/nncf/experimental/torch/fx/quantization/quantize_pt2e.py Updated imports and removed internal quantizer reference
src/nncf/experimental/torch/fx/quantization/quantize_model.py Updated imports
src/nncf/experimental/torch/fx/init.py Removed OpenVINOQuantizer export
src/nncf/common/tensor_statistics/collectors.py Updated comment reference
examples/*/requirements.txt Added torchao dependency
constraints.txt Added torchao==0.14.0 constraint

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot removed the NNCF PT Pull requests that updates NNCF PyTorch label Mar 9, 2026
…uires this version. For upgrade, need to update python version first
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

NNCF PTQ Pull requests that updates NNCF PTQ

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants