Skip to content

Commit 7aa051f

Browse files
committed
add bpw with embedding included
1 parent 04312a6 commit 7aa051f

2 files changed

Lines changed: 8 additions & 8 deletions

File tree

models/qwen2/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -79,9 +79,9 @@ Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherA
7979
| --------------------- | ----------------------------------------- | --------------------- | -------- | ---------------- |
8080
| Qwen2.5 1.5B Instruct | none (`float16`) | 16.00 | macOS | 12.21 |
8181
| Qwen2.5 1.5B Instruct | [4-bit quantized][p-4bit] | 4.50 | macOS | 14.79 |
82-
| Qwen2.5 1.5B Instruct | none (`float16`) | 16.00\* | iOS | 12.21 |
83-
| Qwen2.5 1.5B Instruct | [4-bit palettized (group size 8)][p-4bit] | 4.02\* | iOS | 14.64 |
82+
| Qwen2.5 1.5B Instruct | none (`float16`) | 16.00 | iOS | 12.21 |
83+
| Qwen2.5 1.5B Instruct | [4-bit palettized (group size 8)][p-4bit] | 4.63\* | iOS | 14.64 |
8484

85-
\* BPW excludes the Embedding which is quantized to INT8 per-tensor.
85+
\* BPW includes the Embedding which is quantized to INT8 per-tensor.
8686

8787
[p-4bit]: ../README.md#quantization-options

models/qwen3/README.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -79,16 +79,16 @@ Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherA
7979

8080
| Model | Compression | Bits Per Weight (BPW) | Platform | Perplexity Score |
8181
| ---------- | ---------------------------------------------------- | --------------------- | -------- | ---------------- |
82-
| Qwen3 0.6B | none (`float16`) | 16.00\* | iOS | 26.16 |
83-
| Qwen3 0.6B | [Mixed 4-bit/8-bit palettized][mixed-4bit-8bit-yaml] | 4.88\* | iOS | 30.90 |
82+
| Qwen3 0.6B | none (`float16`) | 16.00 | iOS | 26.16 |
83+
| Qwen3 0.6B | [Mixed 4-bit/8-bit palettized][mixed-4bit-8bit-yaml] | 5.71\* | iOS | 30.90 |
8484
| Qwen3 4B | none (`float16`) | 16.00 | macOS | 16.41 |
8585
| Qwen3 4B | [4-bit quantized][presets-info] | 4.50 | macOS | 18.33 |
86-
| Qwen3 4B | none (`float16`) | 16.00\* | iOS | 16.41 |
87-
| Qwen3 4B | [Mixed 4-bit/8-bit palettized][qwen3-4b-mixed-yaml] | 4.56\* | iOS | 18.80 |
86+
| Qwen3 4B | none (`float16`) | 16.00 | iOS | 16.41 |
87+
| Qwen3 4B | [Mixed 4-bit/8-bit palettized][qwen3-4b-mixed-yaml] | 4.89\* | iOS | 18.80 |
8888
| Qwen3 8B | none (`float16`) | 16.00 | macOS | 12.19 |
8989
| Qwen3 8B | [4-bit quantized][presets-info] | 4.50 | macOS | 12.90 |
9090

91-
\* BPW excludes the Embedding which is quantized to INT8 per-tensor.
91+
\* BPW includes the Embedding which is quantized to INT8 per-tensor.
9292

9393
[presets-info]: ../README.md#quantization-options
9494
[mixed-4bit-8bit-yaml]: qwen3_0_6b_mixed_4bit_8bit.yaml

0 commit comments

Comments
 (0)