Aanuf/data free awq

[andreyanufr]

Changes

Data free AWQ: smooth down_proj input channels and merge extra scale to up_proj output channels.

New method is enabled when AWQ is enabled but the dataset is not available.

Reason for changes

[ljaljushkin]

The test coverage is poor for the new algorithm, that is going to be a default option.

Definitely, a functionality testing for data-free method is needed, that is capable to find an expected alpha (like in template test) and accuracy is improved (conformance test).

As far as I understood, this PR introduces a new behavior for weight compression with dataset and awq option. It would be great to check typical user scenarios and check whether the expected path is launched (e.g. via mocker.spy on specific function).

[alexsu52]

@andreyanufr, please fill the PR description

[nikita-savelyevv]

Could you please redefine self._data_aware_compression as following:

self._data_aware_compression = (self._awq and self._advanced_parameters.awq_params.is_data_aware) or self._scale_estimation or self._lora_correction or self._gptq

Then we can rollback this if statement to the original form.

Suggested change

	if (self._data_aware_mixed_precision or self._data_aware_compression) and dataset:
	if self._data_aware_mixed_precision or self._data_aware_compression:

[andreyanufr]

Done

[nikita-savelyevv]

I have to change my original suggestion here 🙂 After the recent changes (is_data_aware -> prefer_data_aware) I think it actually makes more sense to define

self._data_aware_compression = self._scale_estimation or self._lora_correction or self._gptq

Because otherwise it can happen that self._data_aware_compression is True, but data-aware won't actually be applied. This is in case self._awq is True and dataset is not provided.

And then we can do:

Suggested change

	if (self._data_aware_mixed_precision or self._data_aware_compression) and dataset:
	data_aware_awq = dataset and self._awq and self._advanced_parameters.awq_params.prefer_data_aware
	if self._data_aware_mixed_precision or self._data_aware_compression or data_aware_awq:

[alexsu52]

Suggested change

	:param prefer_data_aware: Determines whether to use activations to calculate scales if activations are presented.
	:type prefer_data_aware: bool
	"""
	subset_size: int = 32
	percent_to_apply: float = 0.002
	alpha_min: float = 0.0
	alpha_max: float = 1.0
	steps: int = 100
	prefer_data_aware: bool = True
	:param use_data_aware_scaling: Whether to use activation data for scale calculation when available.
	:type use_data_aware_scaling: bool
	"""
	subset_size: int = 32
	percent_to_apply: float = 0.002
	alpha_min: float = 0.0
	alpha_max: float = 1.0
	steps: int = 100
	use_data_aware_scaling: bool = True

What do you think about this?