Skip to content

Commit c62419c

Browse files
ephetersonepheterson
authored
Native Swift ML engine (default, with automatic venv fallback) (#108)
1 parent b9611ba commit c62419c

36 files changed

Lines changed: 2006 additions & 41 deletions

‎.github/workflows/native-bundle.yml‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
name: native-bundle
2+
# Build the relocatable native ML bundle on a macOS runner and upload it as an
3+
# artifact (workflow_dispatch) or a release asset (on release). The metallib is
4+
# pinned to the mlx version that matches mlx-swift 0.31.6 (validated 0.31.1), so
5+
# the shipped bundle is the one that was tested, not a mismatched newer metallib.
6+
on:
7+
workflow_dispatch:
8+
inputs:
9+
version:
10+
description: version string for the tarball name
11+
default: "1.6.0"
12+
13+
jobs:
14+
build:
15+
# macos-15 ships Xcode 16 / Swift 6, so mlx-swift resolves to the pinned
16+
# 0.31.6 (Package.resolved). Older Swift silently falls back to 0.29.x.
17+
runs-on: macos-15
18+
steps:
19+
- uses: actions/checkout@v4
20+
21+
- name: onnxruntime
22+
run: brew install onnxruntime
23+
24+
- name: metallib matching mlx-swift 0.31.6
25+
run: |
26+
pip3 install --break-system-packages "mlx==0.31.1" || pip3 install "mlx==0.31.1"
27+
# mlx 0.31.1 splits into namespace packages (mlx + mlx-metal), so
28+
# mlx.__file__ is None; find the metallib under mlx.__path__ instead.
29+
MLX_DIR="$(python3 -c 'import mlx; print(list(mlx.__path__)[0])')"
30+
MLX_METALLIB="$(find "$MLX_DIR" -name mlx.metallib 2>/dev/null | head -1)"
31+
[ -n "$MLX_METALLIB" ] || MLX_METALLIB="$(find /opt/homebrew /Library/Frameworks/Python.framework -name mlx.metallib 2>/dev/null | head -1)"
32+
test -f "$MLX_METALLIB" || { echo "no mlx.metallib found (mlx dir: $MLX_DIR)"; exit 1; }
33+
echo "found metallib: $MLX_METALLIB"
34+
echo "MLX_METALLIB=$MLX_METALLIB" >> "$GITHUB_ENV"
35+
36+
- name: build bundle
37+
env:
38+
RELEASE_TAG: ${{ github.event.release.tag_name }}
39+
INPUT_VERSION: ${{ inputs.version }}
40+
run: |
41+
VERSION="${RELEASE_TAG:-${INPUT_VERSION:-1.6.0}}"
42+
VERSION="${VERSION#v}"
43+
cd native-ml && bash scripts/release_bundle.sh "$VERSION"
44+
45+
- name: upload artifact
46+
uses: actions/upload-artifact@v4
47+
with:
48+
name: native-bundle
49+
path: native-ml/immich-ml-native-*.tar.gz
50+
if-no-files-found: error

‎.github/workflows/update-homebrew.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,43 @@ on:
88
required: true
99

1010
jobs:
11+
build-native:
12+
# Build the native ML bundle on macOS (Swift + Metal) and attach it to the
13+
# release, so the formula can install a prebuilt, ad-hoc-signed binary (no
14+
# notarization; brew does not quarantine its downloads). macos-15 = Swift 6,
15+
# so mlx-swift resolves to the validated 0.31.x.
16+
runs-on: macos-15
17+
outputs:
18+
sha256: ${{ steps.pkg.outputs.sha256 }}
19+
steps:
20+
- uses: actions/checkout@v6
21+
with:
22+
ref: ${{ inputs.tag }}
23+
- name: onnxruntime
24+
run: brew install onnxruntime
25+
- name: metallib matching mlx-swift
26+
run: |
27+
pip3 install --break-system-packages "mlx==0.31.1" || pip3 install "mlx==0.31.1"
28+
MLX_DIR="$(python3 -c 'import mlx; print(list(mlx.__path__)[0])')"
29+
MLX_METALLIB="$(find "$MLX_DIR" -name mlx.metallib 2>/dev/null | head -1)"
30+
[ -n "$MLX_METALLIB" ] || MLX_METALLIB="$(find /opt/homebrew -name mlx.metallib 2>/dev/null | head -1)"
31+
test -f "$MLX_METALLIB" || { echo "no mlx.metallib found"; exit 1; }
32+
echo "MLX_METALLIB=$MLX_METALLIB" >> "$GITHUB_ENV"
33+
- name: Build + upload native bundle asset
34+
id: pkg
35+
env:
36+
TAG: ${{ inputs.tag }}
37+
GH_TOKEN: ${{ github.token }}
38+
GH_REPO: ${{ github.repository }}
39+
run: |
40+
VERSION="${TAG#v}"
41+
cd native-ml && bash scripts/release_bundle.sh "$VERSION"
42+
TARBALL="$(ls immich-ml-native-*.tar.gz)"
43+
echo "sha256=$(shasum -a 256 "$TARBALL" | cut -d' ' -f1)" >> "$GITHUB_OUTPUT"
44+
gh release upload "$TAG" "$TARBALL" --clobber
45+
1146
update-tap:
47+
needs: build-native
1248
runs-on: ubuntu-latest
1349
env:
1450
TAG: ${{ inputs.tag }}
@@ -54,9 +90,13 @@ jobs:
5490
ML_SHA: ${{ steps.ml.outputs.sha256 }}
5591
REPO: ${{ github.repository }}
5692
OWNER: ${{ github.repository_owner }}
93+
NATIVE_SHA: ${{ needs.build-native.outputs.sha256 }}
5794
run: |
5895
export SRC_URL="https://github.com/${REPO}/archive/refs/tags/v${VERSION}.tar.gz"
5996
export ML_URL="https://github.com/${OWNER}/immich-ml-metal/archive/${ML_COMMIT}.tar.gz"
97+
export NATIVE_URL="https://github.com/${REPO}/releases/download/${TAG}/immich-ml-native-${VERSION}-macos-arm64.tar.gz"
98+
# ML models are fetched by the accelerator on first native start, not
99+
# at install time (see _maybe_fetch_native_models), so no model URL here.
60100
bash acc/scripts/render-formula.sh
61101
ruby -c "$OUT"
62102

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Changelog
22

3+
## 1.6.0 - 2026-07-16
4+
5+
### Added
6+
- **Native Swift ML engine, the new default (with automatic Python-venv fallback).** The ML service (CLIP visual and text, face detection and recognition, OCR) can now run as a single native Swift binary instead of the ~1.5 GB Python venv (torch, mlx, onnxruntime, opencv, insightface) and the mlx-pin crash surface that came with it (#38, #103). CLIP runs on `mlx-swift` with the same mlx-community weights; face embedding runs the identical InsightFace ArcFace model through onnxruntime's C API; detection and OCR use Apple's Vision framework. Same models and weights, so embeddings stay in the same space as an existing Immich index and face clusters: no re-index, no re-cluster. Validated against the Python service on real library photos (CLIP visual cosine ~0.996, CLIP text 1.0, face embedding ~0.999 bbox-matched, OCR identical). The native engine is preferred automatically; if its bundle or models are missing, or it fails to start or become healthy, the accelerator falls back to the Python venv so ML is never left down. On a fresh install the models (~740MB) download once in the background on first native start, so ML runs on the Python engine for a few minutes until they arrive. **To switch back to the Python engine**, set `"ml_engine": "python"` in `~/.immich-accelerator/config.json` and restart the accelerator (`brew services restart epheterson/immich-accelerator/immich-accelerator`); set it back to `"native"` or remove the key to return. Distributed as an ad-hoc-signed relocatable bundle over Homebrew, so there is no notarization step and no Gatekeeper friction.
7+
38
## 1.5.32 - 2026-07-15
49

510
### Fixed

‎README.md‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -165,16 +165,28 @@ See **[docs/split-deployment.md](docs/split-deployment.md)** for the full guide:
165165

166166
## ML service
167167

168-
The ML service is a managed fork of [immich-ml-metal](https://github.com/sebastianfredette/immich-ml-metal) by [@sebastianfredette](https://github.com/sebastianfredette), included as a git submodule. It replaces Immich's Docker ML container with native macOS inference. Upstream changes are reviewed before merging.
168+
The ML service runs Immich's CLIP, face, and OCR inference natively on Apple Silicon.
169169

170170
| Task | Hardware | Framework |
171171
|------|----------|-----------|
172-
| CLIP embeddings | GPU (Metal) | MLX |
173-
| Face detection | Neural Engine | Apple Vision |
174-
| Face recognition | CPU / CoreML | InsightFace ONNX |
172+
| CLIP embeddings (image + text) | GPU (Metal) | mlx-swift |
173+
| Face detection + landmarks | Neural Engine | Apple Vision |
174+
| Face recognition | CPU | InsightFace ArcFace (onnxruntime) |
175175
| OCR | Neural Engine | Apple Vision |
176176

177-
Contributions to the ML service are made via [upstream PRs](https://github.com/sebastianfredette/immich-ml-metal/pulls).
177+
As of 1.6.0 this runs as a **native Swift engine**: a single binary with the models and libraries bundled, no Python. It replaces the ~1.5 GB Python venv (torch, mlx, onnxruntime, opencv, insightface) and the dependency-pin fragility that came with it. It uses the same weights and models as the Python service, so embeddings stay in the same space as an existing Immich search index and face clusters (no re-index, no re-cluster).
178+
179+
The native engine is the default and is health-checked at startup. If its bundle or models are missing, or it fails to start, the accelerator automatically falls back to the Python service so ML is never left down. On a brand-new install the models (~740MB) are downloaded once in the background on first native start, so ML runs on the Python engine for a few minutes until they arrive, then switches to native automatically.
180+
181+
**Switching back to the Python engine.** If you want to force the Python service (for example to compare results, or if native misbehaves), set `ml_engine` in `~/.immich-accelerator/config.json`:
182+
183+
```json
184+
{ "ml_engine": "python" }
185+
```
186+
187+
Then restart the accelerator (`brew services restart epheterson/immich-accelerator/immich-accelerator`, or `immich-accelerator stop` then `start`). Set it back to `"native"` (or remove the key) and restart to return to native. Confirm which engine is live with `immich-accelerator ml-test` and check `ml.log`.
188+
189+
The Python engine is a managed fork of [immich-ml-metal](https://github.com/sebastianfredette/immich-ml-metal) by [@sebastianfredette](https://github.com/sebastianfredette), included as a git submodule; upstream changes are reviewed before merging, and contributions are made via [upstream PRs](https://github.com/sebastianfredette/immich-ml-metal/pulls).
178190

179191
## Running as a service (recommended)
180192

‎VERSION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1.5.32
1+
1.6.0

0 commit comments

Comments
 (0)