Issues about QuaRot on Dream-Base

[hhhhhhy5]

Thanks for the excellent work.
It seems that there is something wrong with QuaRot on Dream-base. I modified w_bits and a_bits to 4 and ran command below:
python $DIRPATH/QuaRot/fake_quant/main.py --model $MODEL_PATH \ --a_bits 4 --v_bits 16 --k_bits 16 --w_bits 4 --w_clip \ --rotate --tasks piqa \
only got 52.77 accuracy, which is nearly random.
To further investigate, I ran fp 16 without "--rotate", that is python $DIRPATH/QuaRot/fake_quant/main.py --model $MODEL_PATH \ --a_bits 16 --v_bits 16 --k_bits 16 --w_bits 16 \ --tasks piqa \
got the right accuracy 74.
But I ran fp16 with "--rotate", that is python $DIRPATH/QuaRot/fake_quant/main.py --model $MODEL_PATH \ --a_bits 16 --v_bits 16 --k_bits 16 --w_bits 16 --rotate \ --tasks piqa \
only got 53.53.

Have you tested quarot on Dream?

[FelixMessi]

Hi, thanks for your question.

There are indeed some bugs in QuaRot when applied to Dream models. We are planning to fix these issues in the next update. In the meantime, we also welcome pull requests if you happen to identify a solution.

Thanks for your interest!