File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -79,9 +79,9 @@ Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherA
7979| --------------------- | ----------------------------------------- | --------------------- | -------- | ---------------- |
8080| Qwen2.5 1.5B Instruct | none (` float16 ` ) | 16.00 | macOS | 12.21 |
8181| Qwen2.5 1.5B Instruct | [ 4-bit quantized] [ p-4bit ] | 4.50 | macOS | 14.79 |
82- | Qwen2.5 1.5B Instruct | none (` float16 ` ) | 16.00\* | iOS | 12.21 |
83- | Qwen2.5 1.5B Instruct | [ 4-bit palettized (group size 8)] [ p-4bit ] | 4.02 \* | iOS | 14.64 |
82+ | Qwen2.5 1.5B Instruct | none (` float16 ` ) | 16.00 | iOS | 12.21 |
83+ | Qwen2.5 1.5B Instruct | [ 4-bit palettized (group size 8)] [ p-4bit ] | 4.63 \* | iOS | 14.64 |
8484
85- \* BPW excludes the Embedding which is quantized to INT8 per-tensor.
85+ \* BPW includes the Embedding which is quantized to INT8 per-tensor.
8686
8787[ p-4bit ] : ../README.md#quantization-options
Original file line number Diff line number Diff line change @@ -79,16 +79,16 @@ Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherA
7979
8080| Model | Compression | Bits Per Weight (BPW) | Platform | Perplexity Score |
8181| ---------- | ---------------------------------------------------- | --------------------- | -------- | ---------------- |
82- | Qwen3 0.6B | none (` float16 ` ) | 16.00\* | iOS | 26.16 |
83- | Qwen3 0.6B | [ Mixed 4-bit/8-bit palettized] [ mixed-4bit-8bit-yaml ] | 4.88 \* | iOS | 30.90 |
82+ | Qwen3 0.6B | none (` float16 ` ) | 16.00 | iOS | 26.16 |
83+ | Qwen3 0.6B | [ Mixed 4-bit/8-bit palettized] [ mixed-4bit-8bit-yaml ] | 5.71 \* | iOS | 30.90 |
8484| Qwen3 4B | none (` float16 ` ) | 16.00 | macOS | 16.41 |
8585| Qwen3 4B | [ 4-bit quantized] [ presets-info ] | 4.50 | macOS | 18.33 |
86- | Qwen3 4B | none (` float16 ` ) | 16.00\* | iOS | 16.41 |
87- | Qwen3 4B | [ Mixed 4-bit/8-bit palettized] [ qwen3-4b-mixed-yaml ] | 4.56 \* | iOS | 18.80 |
86+ | Qwen3 4B | none (` float16 ` ) | 16.00 | iOS | 16.41 |
87+ | Qwen3 4B | [ Mixed 4-bit/8-bit palettized] [ qwen3-4b-mixed-yaml ] | 4.89 \* | iOS | 18.80 |
8888| Qwen3 8B | none (` float16 ` ) | 16.00 | macOS | 12.19 |
8989| Qwen3 8B | [ 4-bit quantized] [ presets-info ] | 4.50 | macOS | 12.90 |
9090
91- \* BPW excludes the Embedding which is quantized to INT8 per-tensor.
91+ \* BPW includes the Embedding which is quantized to INT8 per-tensor.
9292
9393[ presets-info ] : ../README.md#quantization-options
9494[ mixed-4bit-8bit-yaml ] : qwen3_0_6b_mixed_4bit_8bit.yaml
You can’t perform that action at this time.
0 commit comments