Skip to content

Aanuf/data free awq #3315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
May 9, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
488cacc
Support scale estimation inside GPTQ
alexsu52 Jun 10, 2024
ee64877
fix for INT4_ASYM
alexsu52 Sep 4, 2024
f22e411
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Sep 23, 2024
51b4d7b
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Sep 26, 2024
f66cd1e
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Sep 30, 2024
7ce5a53
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Oct 2, 2024
f74d156
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Nov 11, 2024
5288c79
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Nov 11, 2024
1becf15
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Nov 14, 2024
047d7d9
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Dec 10, 2024
c0c7e57
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Dec 16, 2024
b74dea1
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Dec 27, 2024
26a9a77
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Jan 7, 2025
25fcc2c
Merge remote-tracking branch 'upstream/develop' into develop
andreyanufr Feb 25, 2025
f6f4693
Data-free AWQ prototype.
andreyanufr Feb 25, 2025
19a64ac
Data free AWQ.
andreyanufr Feb 26, 2025
bf215d5
Fixed style.
andreyanufr Feb 26, 2025
566ebe7
Fixed shape of data item int test.
andreyanufr Feb 27, 2025
70e47c8
Fixed test case for E2M1.
andreyanufr Feb 27, 2025
6b3310b
Enable awq by default.
andreyanufr Mar 4, 2025
c13437f
Resolved merge conflict.
andreyanufr Apr 9, 2025
519727f
Return AWQ default value.
andreyanufr Apr 9, 2025
10920a2
Fixed debug code.
andreyanufr Apr 9, 2025
35f6a64
Added parameter to define usage of data-free AWQ.
andreyanufr Apr 16, 2025
83c7867
Added tet for data-free AWQ.
andreyanufr Apr 16, 2025
a17b896
Fixed merge conflict.
andreyanufr Apr 16, 2025
516fec7
Changed type hint.
andreyanufr Apr 17, 2025
48e7f1c
Added data-free AWQ test fot pytorch case.
andreyanufr Apr 17, 2025
57c3e4a
Applied suggestion.
andreyanufr Apr 17, 2025
cd9e4c1
Gave the parameter a clearer nameю
andreyanufr Apr 17, 2025
1938caf
Fixed merge conflict.
andreyanufr Apr 29, 2025
7a7af27
Fixed test.
andreyanufr Apr 29, 2025
8674915
Changed name of parameter.
andreyanufr Apr 30, 2025
56e5312
Fix.
andreyanufr Apr 30, 2025
5465fc5
Fix.
andreyanufr Apr 30, 2025
4a3a3a5
Fixed syle.
andreyanufr Apr 30, 2025
8da6472
1) Fixed bug with data-free AWQ and mixed-precision.
andreyanufr May 5, 2025
f8ec84a
Fixed test.
andreyanufr May 5, 2025
d9fea93
Merge remote-tracking branch 'upstream/develop' into aanuf/data_free_awq
andreyanufr May 6, 2025
31e4d47
Enabled torch test for data-free AWQ.
andreyanufr May 6, 2025
716d9ef
Fixed bug in test.
andreyanufr May 6, 2025
1ba3bb9
Merge remote-tracking branch 'upstream/develop' into aanuf/data_free_awq
andreyanufr May 7, 2025
6d68137
Expanded test for data-free awq.
andreyanufr May 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions nncf/quantization/advanced_parameters.py
Original file line number Diff line number Diff line change
Expand Up @@ -276,13 +276,17 @@ class AdvancedAWQParameters:
:type alpha_max: float
:param steps: The number of the steps in grid search.
:type steps: int
:param prefer_data_aware_scaling: Determines whether to use activations to calculate scales if
activations are presented.
:type prefer_data_aware_scaling: bool
"""

subset_size: int = 32
percent_to_apply: float = 0.002
alpha_min: float = 0.0
alpha_max: float = 1.0
steps: int = 100
prefer_data_aware_scaling: bool = True


@api()
Expand Down
10 changes: 8 additions & 2 deletions nncf/quantization/algorithms/weight_compression/algorithm.py
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,7 @@ def __init__(
awq_params.alpha_min,
awq_params.alpha_max,
awq_params.steps,
awq_params.prefer_data_aware_scaling,
)
if self._gptq:
gptq_params = self._advanced_parameters.gptq_params
Expand All @@ -323,7 +324,12 @@ def __init__(
self._data_aware_mixed_precision = (
self._sensitivity_metric != SensitivityMetric.WEIGHT_QUANTIZATION_ERROR and self._ratio != 1.0
)
self._data_aware_compression = self._awq or self._scale_estimation or self._lora_correction or self._gptq
self._data_aware_compression = (
(self._awq and self._advanced_parameters.awq_params.prefer_data_aware_scaling)
or self._scale_estimation
or self._lora_correction
or self._gptq
)

@property
def available_backends(self) -> list[BackendType]:
Expand Down Expand Up @@ -546,7 +552,7 @@ def apply(
nodes_to_compress = self.get_nodes_to_compress(graph)

statistics = None
if self._data_aware_mixed_precision or self._data_aware_compression:
if (self._data_aware_mixed_precision or self._data_aware_compression) and dataset:
matmul_nodes_to_compress = [
node for node in nodes_to_compress if node.metatype in self._backend_entity.matmul_metatypes
]
Expand Down
Loading
Loading