Add quantization options by mht-sharma · Pull Request #102 · huggingface/optimum-amd

mht-sharma · 2024-03-11T07:18:38Z

Update quantization for Ryzen SDK 1.1

* update quantization configurations for ryzenai * add some combination checks * fix typo * change enable_dpu to enable_ipu_cnn

mht-sharma · 2024-06-04T15:21:26Z

Doc build fix in: #141

fxmarty · 2024-06-05T08:05:38Z

optimum/amd/ryzenai/configuration.py

-            Determines whether to generate a quantized model that is suitable for the DPU. If set to True, the quantization
-            process will create a model that is optimized for DPU computations.
-
+        format (Union[QuantFormat, str], defaults to `QuantFormat.QDQ`):


Suggested change

format (Union[QuantFormat, str], defaults to `QuantFormat.QDQ`):

format (`Union[QuantFormat, str]`, defaults to `QuantFormat.QDQ`):

fxmarty · 2024-06-05T08:06:17Z

optimum/amd/ryzenai/configuration.py

+              into the tensor. Supports a wider range of bit-widths and precisions.
+            - `QuantFormat.FixNeuron` (Experimental): Quantizes the model by inserting FixNeuron (a combination of
+              QuantizeLinear and DeQuantizeLinear) into the tensor. Experimental and not recommended for deployment.
+        calibration_method (Union[CalibrationMethod, str], defaults to `CalibrationMethod.MinMSE`):


fxmarty · 2024-06-05T08:06:25Z

optimum/amd/ryzenai/configuration.py

+            - `CalibrationMethod.MinMax`: Obtain quantization parameters based on minimum and maximum values of each tensor.
+            - `CalibrationMethod.Entropy`: Determine quantization parameters based on the entropy algorithm of each tensor's distribution.
+            - `CalibrationMethod.Percentile`: Calculate quantization parameters using percentiles of tensor values.
+        activations_dtype (QuantType, defaults to `QuantType.QUInt8`):


same (and all other args below)

fxmarty · 2024-06-05T08:08:04Z

optimum/amd/ryzenai/configuration.py

+    def check_dtype_and_format(dtype, dtype_name, format):
+        if dtype not in ["uint8", "int8"] and format not in ["vitisqdq"]:
+            raise ValueError(f'{dtype_name} is: "{dtype}", format must be "vitisqdq".')


I don't think this is needed, clearer to have it inlined in the post-init.

fxmarty · 2024-06-05T08:08:33Z

optimum/amd/ryzenai/configuration.py

+        mapping = {
+            "uint8": QuantType.QUInt8,
+            "int8": QuantType.QInt8,
+            "uint16": QuantType.QUInt16,
+            "int16": QuantType.QInt16,
+            "uint32": QuantType.QUInt32,
+            "int32": QuantType.QInt32,
+            "float16": QuantType.QFloat16,
+            "bfloat16": QuantType.QBFloat16,
+        }


Why not define a constant for this

fxmarty · 2024-06-05T08:17:07Z

optimum/amd/ryzenai/configuration.py

+            if self.activations_dtype not in ["uint8", "int8"]:
+                raise ValueError('ipu cnn configuration only support activations_dtype "uint8" and "int8".')
+            if self.weights_dtype not in ["int8"]:
+                raise ValueError('ipu cnn configuration only support weights_dtype "int8".')


Suggested change

raise ValueError('ipu cnn configuration only support weights_dtype "int8".')

raise ValueError(f'ipu cnn configuration only support weights_dtype "int8". Got: weights_dtype={self.weights_dtype}')

fxmarty · 2024-06-05T08:17:48Z

optimum/amd/ryzenai/configuration.py

+            if self.activations_dtype not in ["uint8", "int8"]:
+                raise ValueError('ipu cnn configuration only support activations_dtype "uint8" and "int8".')
+            if self.weights_dtype not in ["int8"]:
+                raise ValueError('ipu cnn configuration only support weights_dtype "int8".')


fxmarty · 2024-06-05T08:17:53Z

optimum/amd/ryzenai/configuration.py

+                raise ValueError('ipu cnn configuration only support calibration_method "nonoverflow" and "mse".')
+            if not (self.extra_options.activation_symmetric and self.extra_options.weight_symmetric):
+                raise ValueError(
+                    "ipu cnn configuration requires setting activation_symmetric and weight_symmetric to true."


fxmarty · 2024-06-05T08:17:58Z

optimum/amd/ryzenai/configuration.py

+            if self.format not in ["qdq"]:
+                raise ValueError('ipu cnn configuration only support format "qdq".')
+            if self.calibration_method not in ["nonoverflow", "mse"]:
+                raise ValueError('ipu cnn configuration only support calibration_method "nonoverflow" and "mse".')


fxmarty · 2024-06-05T08:18:40Z

optimum/amd/ryzenai/configuration.py

+    def to_diff_dict(self) -> dict:
+        """
+        Returns a dictionary of non-default values in the configuration.
+        """
+        non_default_values = {}
+        for option in fields(self):
+            if option.name == "extra_options":
+                extra_options_dict = getattr(self, option.name).to_diff_dict()
+                if extra_options_dict:
+                    non_default_values[option.name] = extra_options_dict
+            else:
+                value = getattr(self, option.name)
+
+                if value != option.default and value not in ({}, []):
+                    if option.name == "execution_providers" and value == ["CPUExecutionProvider"]:
+                        continue
+
+                    if isinstance(value, Enum):
+                        value = value.name
+                    elif isinstance(value, list):
+                        value = [elem.name if isinstance(elem, Enum) else elem for elem in value]
+
+                    non_default_values[option.name] = value
+        return non_default_values


To me what would be more interesting is a method to compare two configs.

Is the compare method for loading quantization params from config.json file?

mht-sharma and others added 23 commits March 6, 2024 18:37

add pipeline

afd8367

add options

3afe2bc

added diff dict

ceef198

added diff dict

51334f6

Merge branch 'main' into add_quantization_options

ea88c53

removed fpn

591ee62

removed fpn

f044274

add docsttring

44891a6

updated init models

5bf1818

updated init models

be8c954

updated docs

6174113

Merge branch 'main' into add_quantization_options

3568ca7

Merge branch 'main' into add_quantization_options

86d038f

fix config

acf90b0

update quantization configurations for ryzenai (vai_q_onnx) (#117)

1721308

* update quantization configurations for ryzenai * add some combination checks * fix typo * change enable_dpu to enable_ipu_cnn

Merge branch 'main' into add_quantization_options

1c5cd94

fix style

db3ca00

fix options

0d8b1a3

fix options

4385b77

add tests

94b3f80

add config options

4684dea

fix style

8cc538f

Merge branch 'main' into add_quantization_options

7faa79e

fix token

41143d9

fxmarty reviewed Jun 5, 2024

View reviewed changes

mht-sharma added 3 commits June 5, 2024 15:15

addressed comments

409d43c

addressed comments

c3e2554

fix docstring

e67ed2f

	format (Union[QuantFormat, str], defaults to `QuantFormat.QDQ`):
	format (`Union[QuantFormat, str]`, defaults to `QuantFormat.QDQ`):

	raise ValueError('ipu cnn configuration only support weights_dtype "int8".')
	raise ValueError(f'ipu cnn configuration only support weights_dtype "int8". Got: weights_dtype={self.weights_dtype}')

Conversation

mht-sharma commented Mar 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mht-sharma commented Jun 4, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mht-sharma commented Mar 11, 2024 •

edited

Loading