Search results

How to run the eval.py in real quant mode?

Status: Open.

#26 In ChenMnZ/PrefixQuant;

· quic-jouachen opened on Mar 15, 2025

qwen2.5支持吗

Status: Open.

#24 In ChenMnZ/PrefixQuant;

· aqe670 opened on Jan 23, 2025

Possible PrefixQuant issues on Smaller LLMs?

Status: Open.

#23 In ChenMnZ/PrefixQuant;

· sasha-hailo opened on Jan 13, 2025

Question about Preventing Outlier Tokens during Inference

Status: Open.

#21 In ChenMnZ/PrefixQuant;

· fingerk28 opened on Dec 16, 2024

Not quant Q layer?

Status: Open.

#19 In ChenMnZ/PrefixQuant;

· laomao0 opened on Dec 13, 2024

evaluate error

Status: Open.

#18 In ChenMnZ/PrefixQuant;

· z18256199275 opened on Dec 12, 2024

Why does finer granularity in quantization result in more overhead?

Status: Open.

#17 In ChenMnZ/PrefixQuant;

· lsjlsj5846 opened on Dec 12, 2024

how to process prefix token again in decoding?

Status: Open.

#16 In ChenMnZ/PrefixQuant;

· ArtyZe opened on Nov 30, 2024

Question about outlier tokens of MoE, like DeepSeek-v2?

Status: Open.

#13 In ChenMnZ/PrefixQuant;

· L1aoXingyu opened on Nov 15, 2024

AttributeError: 'WrappedPrefixCausalLM' object has no attribute 'generate'

Status: Open.

#11 In ChenMnZ/PrefixQuant;

· vengdeng opened on Nov 11, 2024