Skip to content

When quantizing baichuan2-7B, I encountered an error: AttributeError: 'RMSNorm' object has no attribute 'epsilon'. #20

@Flyipig

Description

@Flyipig

Here are the details of the traceback:
Traceback (most recent call last):
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/examples/smoothquant_model.py", line 70, in
main()
File "/HOME/scz0oo5/.conda/envs/AUTOSmooth/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/examples/smoothquant_model.py", line 65, in main
int8_model = quant_model_class.from_float(model, decoder_layer_scales, quant_config)
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/models/baichuan.py", line 561, in from_float
int8_module.model = Int8BaichuanModel.from_float(
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/models/baichuan.py", line 363, in from_float
int8_module.layers[i] = Int8BaichuanLayer.from_float(
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/models/baichuan.py", line 284, in from_float
int8_module.input_layernorm = Int8BaichuanRMSNorm.from_float(
File "/data/run01/scz0oo5/AutoSmoothQuant/autosmoothquant/models/baichuan.py", line 54, in from_float
int8_norm = Int8BaichuanRMSNorm(module.weight.numel(), module.epsilon)
File "/HOME/scz0oo5/.conda/envs/AUTOSmooth/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1688, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'RMSNorm' object has no attribute 'epsilon'

I guess this issue is related to the version of PyTorch being used. Could you please advise on the specific version of PyTorch required for using AUTOSmoothQuant?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions