Skip to content

Commit b949460

Browse files
stikvesstikves
committed
Merge branch 'main' into sukru/diffusion-metadata-alignment
2 parents dca8a40 + d83f0b0 commit b949460

51 files changed

Lines changed: 1420 additions & 2416 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 23 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,26 +12,37 @@ concurrency:
1212

1313
jobs:
1414
lint:
15-
runs-on: ubuntu-latest
15+
if: github.repository == 'apple/coreai-models'
16+
runs-on: [self-hosted, macos, tahoe, ARM64]
17+
timeout-minutes: 15
1618
steps:
17-
- uses: actions/checkout@v4
18-
- uses: astral-sh/setup-uv@v4
19+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
20+
- name: Ensure uv
21+
run: |
22+
command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
23+
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
1924
- run: uv tool install ruff
2025
- run: ruff check python/src/ python/tests/
2126
- run: ruff format --check python/src/ python/tests/
2227

2328
swift-format:
24-
runs-on: ubuntu-latest
29+
if: github.repository == 'apple/coreai-models'
30+
runs-on: [self-hosted, macos, tahoe, ARM64]
31+
timeout-minutes: 15
32+
env:
33+
DEVELOPER_DIR: /Applications/Xcode-latest.app/Contents/Developer
2534
steps:
26-
- uses: actions/checkout@v4
27-
- uses: swift-actions/setup-swift@v2
28-
with:
29-
swift-version: "6.0"
30-
- run: swift-format lint --strict --recursive swift/Sources/ swift/Tests/
35+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
36+
- run: swift format lint --strict --recursive swift/Sources/ swift/Tests/
3137

3238
python-test:
33-
runs-on: ubuntu-latest
39+
if: github.repository == 'apple/coreai-models'
40+
runs-on: [self-hosted, macos, tahoe, ARM64]
41+
timeout-minutes: 60
3442
steps:
35-
- uses: actions/checkout@v4
36-
- uses: astral-sh/setup-uv@v4
43+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
44+
- name: Ensure uv
45+
run: |
46+
command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
47+
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
3748
- run: uv run pytest python/tests/test_model_units -x -q

‎Package.resolved‎

Lines changed: 10 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Package.swift‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ let package = Package(
3939
dependencies: [
4040
.package(url: "https://github.com/apple/swift-argument-parser", from: "1.2.0"),
4141
.package(url: "https://github.com/huggingface/swift-transformers", from: "1.1.0"),
42+
.package(url: "https://github.com/mlc-ai/xgrammar", branch: "main"),
4243
],
4344
targets: [
4445
.target(
@@ -98,9 +99,13 @@ let package = Package(
9899
),
99100

100101
// CXGrammar C bridge
101-
.binaryTarget(
102+
.target(
102103
name: "CXGrammar",
103-
path: "swift/Sources/CoreAILanguageModels/lib/CXGrammar.xcframework"
104+
dependencies: [
105+
.product(name: "XGrammar", package: "xgrammar")
106+
],
107+
path: "swift/Sources/lib/CXGrammar",
108+
publicHeadersPath: "include"
104109
),
105110

106111
// MARK: Executable targets
@@ -229,5 +234,6 @@ let package = Package(
229234
]
230235
),
231236
],
232-
swiftLanguageModes: [.v6]
237+
swiftLanguageModes: [.v6],
238+
cxxLanguageStandard: .cxx17
233239
)

‎models/gemma3/README.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -79,11 +79,11 @@ Defaults: 512 prompt tokens, 1024 generation tokens, 5 trials. Override with `-p
7979

8080
Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherAI/wikitext_document_level) dataset computed using the [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/wikitext/README.md) with the Core AI PyTorch models.
8181

82-
| Model | Compression | Platform | Perplexity Score |
83-
| ----------- | ------------------------- | -------- | ---------------- |
84-
| Gemma 3 4B | none (`float16`) | macOS | 17.90 |
85-
| Gemma 3 4B | [4-bit quantized][p-4bit] | macOS | 19.28 |
86-
| Gemma 3 12B | none (`float16`) | macOS | 11.24 |
87-
| Gemma 3 12B | [4-bit quantized][p-4bit] | macOS | 11.75 |
82+
| Model | Compression | Bits Per Weight (BPW) | Platform | Perplexity Score |
83+
| ----------- | ------------------------- | --------------------- | -------- | ---------------- |
84+
| Gemma 3 4B | none (`float16`) | 16.00 | macOS | 17.90 |
85+
| Gemma 3 4B | [4-bit quantized][p-4bit] | 4.50 | macOS | 19.28 |
86+
| Gemma 3 12B | none (`float16`) | 16.00 | macOS | 11.24 |
87+
| Gemma 3 12B | [4-bit quantized][p-4bit] | 4.50 | macOS | 11.75 |
8888

8989
[p-4bit]: ../README.md#quantization-options

‎models/mistral/README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -69,9 +69,9 @@ Defaults: 512 prompt tokens, 1024 generation tokens, 5 trials. Override with `-p
6969

7070
Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherAI/wikitext_document_level) dataset computed using the [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/wikitext/README.md) with the Core AI PyTorch models.
7171

72-
| Model | Compression | Platform | Perplexity Score |
73-
| ------------------- | ------------------------------------------ | -------- | ---------------- |
74-
| Mistral 7B Instruct | none (`float16`) | macOS | 8.29 |
75-
| Mistral 7B Instruct | [4-bit quantized][p-4bit] | macOS | 8.41 |
72+
| Model | Compression | Bits Per Weight (BPW) | Platform | Perplexity Score |
73+
| ------------------- | ------------------------------------------ | --------------------- | -------- | ---------------- |
74+
| Mistral 7B Instruct | none (`float16`) | 16.00 | macOS | 8.29 |
75+
| Mistral 7B Instruct | [4-bit quantized][p-4bit] | 4.50 | macOS | 8.41 |
7676

7777
[p-4bit]: ../README.md#quantization-options

‎models/mixtral/README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -69,9 +69,9 @@ Defaults: 512 prompt tokens, 1024 generation tokens, 5 trials. Override with `-p
6969

7070
Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherAI/wikitext_document_level) dataset computed using the [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/wikitext/README.md) with the Core AI PyTorch models.
7171

72-
| Model | Compression | Platform | Perplexity Score |
73-
| ------------ | ---------------------------------------------------- | -------- | ---------------- |
74-
| Mixtral 8x7B | none (`float16`) | macOS | 5.72 |
75-
| Mixtral 8x7B | [4-bit quantized][p-4bit] | macOS | 6.19 |
72+
| Model | Compression | Bits Per Weight (BPW) | Platform | Perplexity Score |
73+
| ------------ | ---------------------------------------------------- | --------------------- | -------- | ---------------- |
74+
| Mixtral 8x7B | none (`float16`) | 16.00 | macOS | 5.72 |
75+
| Mixtral 8x7B | [4-bit quantized][p-4bit] | 4.50 | macOS | 6.19 |
7676

7777
[p-4bit]: ../README.md#quantization-options

‎models/qwen2/README.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -75,11 +75,13 @@ Defaults: 512 prompt tokens, 1024 generation tokens, 5 trials. Override with `-p
7575

7676
Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherAI/wikitext_document_level) dataset computed using the [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/wikitext/README.md) with the Core AI PyTorch models.
7777

78-
| Model | Compression | Platform | Perplexity Score |
79-
| --------------------- | ----------------------------------------- | -------- | ---------------- |
80-
| Qwen2.5 1.5B Instruct | none (`float16`) | macOS | 12.21 |
81-
| Qwen2.5 1.5B Instruct | [4-bit quantized][p-4bit] | macOS | 14.79 |
82-
| Qwen2.5 1.5B Instruct | none (`float16`) | iOS | 12.21 |
83-
| Qwen2.5 1.5B Instruct | [4-bit palettized (group size 8)][p-4bit] | iOS | 14.64 |
78+
| Model | Compression | Bits Per Weight (BPW) | Platform | Perplexity Score |
79+
| --------------------- | ----------------------------------------- | --------------------- | -------- | ---------------- |
80+
| Qwen2.5 1.5B Instruct | none (`float16`) | 16.00 | macOS | 12.21 |
81+
| Qwen2.5 1.5B Instruct | [4-bit quantized][p-4bit] | 4.50 | macOS | 14.79 |
82+
| Qwen2.5 1.5B Instruct | none (`float16`) | 16.00 | iOS | 12.21 |
83+
| Qwen2.5 1.5B Instruct | [4-bit palettized (group size 8)][p-4bit] | 4.63\* | iOS | 14.64 |
84+
85+
\* BPW includes the Embedding which is quantized to INT8 per-tensor.
8486

8587
[p-4bit]: ../README.md#quantization-options

‎models/qwen3/README.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -77,16 +77,18 @@ Defaults: 512 prompt tokens, 1024 generation tokens, 5 trials. Override with `-p
7777

7878
Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherAI/wikitext_document_level) dataset computed using the [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/wikitext/README.md) with the Core AI PyTorch models.
7979

80-
| Model | Compression | Platform | Perplexity Score |
81-
| ---------- | ---------------------------------------------------- | -------- | ---------------- |
82-
| Qwen3 0.6B | none (`float16`) | iOS | 26.16 |
83-
| Qwen3 0.6B | [Mixed 4-bit/8-bit palettized][mixed-4bit-8bit-yaml] | iOS | 30.90 |
84-
| Qwen3 4B | none (`float16`) | macOS | 16.41 |
85-
| Qwen3 4B | [4-bit quantized][presets-info] | macOS | 18.33 |
86-
| Qwen3 4B | none (`float16`) | iOS | 16.41 |
87-
| Qwen3 4B | [Mixed 4-bit/8-bit palettized][qwen3-4b-mixed-yaml] | iOS | 18.80 |
88-
| Qwen3 8B | none (`float16`) | macOS | 12.19 |
89-
| Qwen3 8B | [4-bit quantized][presets-info] | macOS | 12.90 |
80+
| Model | Compression | Bits Per Weight (BPW) | Platform | Perplexity Score |
81+
| ---------- | ---------------------------------------------------- | --------------------- | -------- | ---------------- |
82+
| Qwen3 0.6B | none (`float16`) | 16.00 | iOS | 26.16 |
83+
| Qwen3 0.6B | [Mixed 4-bit/8-bit palettized][mixed-4bit-8bit-yaml] | 5.71\* | iOS | 30.90 |
84+
| Qwen3 4B | none (`float16`) | 16.00 | macOS | 16.41 |
85+
| Qwen3 4B | [4-bit quantized][presets-info] | 4.50 | macOS | 18.33 |
86+
| Qwen3 4B | none (`float16`) | 16.00 | iOS | 16.41 |
87+
| Qwen3 4B | [Mixed 4-bit/8-bit palettized][qwen3-4b-mixed-yaml] | 4.89\* | iOS | 18.80 |
88+
| Qwen3 8B | none (`float16`) | 16.00 | macOS | 12.19 |
89+
| Qwen3 8B | [4-bit quantized][presets-info] | 4.50 | macOS | 12.90 |
90+
91+
\* BPW includes the Embedding which is quantized to INT8 per-tensor.
9092

9193
[presets-info]: ../README.md#quantization-options
9294
[mixed-4bit-8bit-yaml]: qwen3_0_6b_mixed_4bit_8bit.yaml

‎models/qwen3_moe/README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -51,9 +51,9 @@ Defaults: 512 prompt tokens, 1024 generation tokens, 5 trials. Override with `-p
5151

5252
Perplexity score on the [`WikiText-2`](https://huggingface.co/datasets/EleutherAI/wikitext_document_level) dataset computed using the [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/wikitext/README.md) with the Core AI PyTorch models.
5353

54-
| Model | Compression | Platform | Perplexity Score |
55-
| ---------------------------- | ------------------------- | -------- | ---------------- |
56-
| Qwen3 Coder 30B-A3B Instruct | none (`float16`) | macOS | 11.06 |
57-
| Qwen3 Coder 30B-A3B Instruct | [4-bit quantized][p-4bit] | macOS | 11.90 |
54+
| Model | Compression | Bits Per Weight (BPW) | Platform | Perplexity Score |
55+
| ---------------------------- | ------------------------- | --------------------- | -------- | ---------------- |
56+
| Qwen3 Coder 30B-A3B Instruct | none (`float16`) | 16.00 | macOS | 11.06 |
57+
| Qwen3 Coder 30B-A3B Instruct | [4-bit quantized][p-4bit] | 4.50 | macOS | 11.90 |
5858

5959
[p-4bit]: ../README.md#quantization-options

‎python/src/coreai_models/diffusion/pipeline.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,9 @@ async def _async_export_diffusion(config: DiffusionExportConfig) -> dict[str, st
137137
_save_tokenizer(config.hf_model_id, output_path, hf_pipe, overwrite=config.overwrite)
138138

139139
# 4. Write pipeline.json
140-
_write_metadata_json(hf_pipe, config.hf_model_id, pipeline_type, output_path, config.compression, results)
140+
_write_metadata_json(
141+
hf_pipe, config.hf_model_id, pipeline_type, output_path, config.compression, results
142+
)
141143

142144
# Summary
143145
logger.info("=== Export Summary ===")

0 commit comments

Comments
 (0)