v1.5.30: camera RAW thumbnails via the vips shim (#99) (#102)

Author: epheterson

CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## 1.5.30 - 2026-07-14
+
+### Fixed
+- **Camera RAW thumbnails failed** (#99): Canon CR2 (and other camera RAW such as Nikon NEF, Sony ARW, Adobe DNG, Fujifilm RAF) died in `AssetGenerateThumbnails` with `tiff2vips: Old-style JPEG compression support is not configured`. Sharp's prebuilt libvips on macOS lacks old-style-JPEG support and has no dcraw/libraw loader, so it cannot read a RAW file's embedded image. The HEIC decode shim now also detects RAW by extension and pre-decodes it via Homebrew `vips` (fuller libtiff/libjpeg for TIFF-based RAW like CR2/NEF/ARW/DNG, plus libraw for the rest), the same libvips Docker Immich uses, before handing the result to Sharp. Works headless; OCR and face are untouched (RAW decode affects thumbnail generation only). Verified on an M4 Mac Mini with a real Canon CR2: bundled Sharp alone fails with the tiff2vips error, and with the shim it produces a valid thumbnail. Closes another macOS-only decode gap after the HEIC fix (v1.5.28). Reported by [@shtefko](https://github.com/shtefko).
+- **Headless HEIC decode was silently falling back to `sips`**: Sharp's prebuilt libvips exports a bogus `VIPSHOME` (the GitHub-runner path it was built in, e.g. `/Users/runner/work/sharp-libvips/...`) into the process environment when it loads. The decode shim spawned the Homebrew `vips` with that inherited `VIPSHOME`, so `vips` looked for its loader modules (libheif for HEIC, the RAW loaders) in a nonexistent directory and failed with `VipsForeignLoad: ... is not a known file format`, falling back to Apple's `sips`. Because `sips` needs a logged-in GUI session, on a headless Mac the primary `vips` path (the entire point of the v1.5.28 HEIC fix) never actually ran, so HEIC thumbnails quietly depended on a desktop session. The shim now drops a `VIPSHOME` that points at a nonexistent directory before spawning a decoder, so `vips` uses its own compiled-in module path and decodes HEIC and RAW headless as intended; an intentional, existing `VIPSHOME` is preserved. Surfaced while adding RAW support and verified on a headless M4 Mac Mini (HEIC now decodes via `vips`, not `sips`).
+
+### Performance
+- The `vips` decode intermediate is now written as a lossless deflate-compressed TIFF (roughly 40% smaller than uncompressed) to trim peak memory and temp I/O when decoding large RAW frames. Pixels are unchanged; Sharp reads the deflate TIFF the same way.
+
 ## 1.5.29 - 2026-07-14
 
 ### Changed

README.md

@@ -221,11 +221,13 @@ The native worker runs Immich's unmodified code. The ffmpeg and image processing
 | **ML: Face recognition** | ONNX Runtime | ONNX Runtime with CoreML | Same model, CoreML acceleration. Numerically close embeddings. |
 | **ML: OCR** | PaddleOCR via ONNX | Apple Vision framework (Neural Engine) | Different engine. Vision framework OCR is generally more accurate for Latin text, may differ for CJK. |
 | **HEIC decode** | libvips built with libde265 | Homebrew `vips` (libvips + libde265), then Sharp | Sharp's prebuilt libvips on macOS has no HEVC decoder, so iPhone HEICs are pre-decoded by the Homebrew `vips` (the same libvips + libde265 Docker uses) before Sharp processes them. Works headless; pixels match Docker. Apple ImageIO (`sips`) is a last-resort fallback for a logged-in desktop only. |
+| **Camera RAW decode** | libvips with fuller libtiff/libjpeg + libraw | Homebrew `vips`, then Sharp | Sharp's prebuilt libvips on macOS lacks old-style-JPEG and dcraw/libraw support, so Canon CR2/CR3, Nikon NEF, Sony ARW, Adobe DNG and other RAW originals fail thumbnail generation (`tiff2vips: Old-style JPEG compression support is not configured`, or a `multiband -> srgb` colourspace error). They are pre-decoded by the Homebrew `vips` (fuller libtiff/libjpeg for TIFF-based RAW, plus libraw for the rest), the same libvips Docker uses, before Sharp. Works headless; matches Docker. |
 
 ### What this means in practice
 
 - **Thumbnails, previews, and video**: Identical to Docker. Same jellyfin-ffmpeg binary, same `tonemapx` HDR tone mapping, same output. VideoToolbox hardware encoding is faster but visually equivalent.
 - **HEIC photos**: Thumbnails generate correctly. The default iPhone format (HEVC-coded HEIC, often tiled) is decoded by the Homebrew `vips` (libvips + libde265, a formula dependency) since Sharp's bundled libheif is AVIF-only. This works with no logged-in GUI session (a headless Mac Mini), and output matches Docker. Apple ImageIO (`sips`) remains a fallback but needs a GUI session, so it isn't relied on.
+- **Camera RAW photos** (Canon CR2/CR3, Nikon NEF, Sony ARW, Adobe DNG, and other RAW formats): Thumbnails generate correctly. Sharp's bundled libvips on macOS can't decode these (no old-style-JPEG support, no dcraw/libraw), so they're pre-decoded by the Homebrew `vips` (fuller libtiff/libjpeg plus libraw), the same libvips Docker uses. Works headless.
 - **CLIP search**: Search results are equivalent but not identical. A search that returns 20 results in Docker will return ~18-20 of the same results natively, possibly in slightly different order.
 - **Face grouping**: Faces are detected and grouped correctly. The grouping boundaries may differ slightly (e.g., a borderline face might be grouped differently).
 - **OCR**: Text extraction is at least as good as Docker for English/Latin text.

VERSION

@@ -1 +1 @@
-1.5.29
+1.5.30

immich_accelerator/__main__.py

@@ -3873,10 +3873,11 @@ def cmd_start(args):
             f"{existing} {require_arg}".strip() if existing else require_arg
         )
 
-    # HEIC decode shim (issue #62 follow-up): Sharp's prebuilt libvips ships
-    # without an HEVC decoder (AVIF-only), so iPhone HEICs fail to decode and
-    # never get thumbnails. This preload wraps the `sharp` module to route
-    # HEVC-HEIC file paths through Apple's ImageIO (`sips`) before Sharp.
+    # HEIC + camera-RAW decode shim (issues #62, #99): Sharp's prebuilt libvips
+    # ships without an HEVC decoder (AVIF-only) and without a dcraw/libraw
+    # loader, so iPhone HEICs and Canon/Nikon/Sony RAW files fail to decode and
+    # never get thumbnails. This preload wraps the `sharp` module to route those
+    # file paths through Homebrew libvips (to a lossless TIFF) before Sharp.
     # Same --require interposition; Immich's source on disk is untouched.
     heic_shim = Path(__file__).parent / "hooks" / "heic_decode_shim.js"
     if heic_shim.exists():

immich_accelerator/hooks/heic_decode_shim.js

@@ -1,20 +1,29 @@
 // heic_decode_shim.js
 //
-// Runtime interposition for HEIC decoding in Immich's media pipeline.
+// Runtime interposition for HEIC and camera-RAW decoding in Immich's media
+// pipeline.
 //
-// The native worker uses Sharp's prebuilt libvips (8.17.x), whose bundled
-// libheif is compiled WITHOUT an HEVC decoder — it supports AVIF only. So
-// every HEIC original (the default iPhone format) fails to decode with a
-// "bad seek" / "compression format has not been built in" error, and no
-// thumbnail is produced (issue #62 follow-up). Docker Immich isn't affected
-// because it builds libvips with libde265.
+// The native worker uses Sharp's prebuilt libvips (8.17.x). Two format families
+// it cannot decode on macOS, both of which Docker Immich handles fine:
+//
+//   1. HEVC-HEIC (the default iPhone format): Sharp's bundled libheif is
+//      compiled WITHOUT an HEVC decoder (AVIF only), so every HEIC original
+//      fails with "bad seek" / "compression format has not been built in" and
+//      no thumbnail is produced (issue #62). Docker builds libvips with
+//      libde265, so it is unaffected.
+//   2. Camera RAW (Canon CR2/CR3, Nikon NEF, Sony ARW, Adobe DNG, and friends):
+//      Sharp's bundled libtiff/libjpeg lack old-style-JPEG support and it has no
+//      dcraw/libraw loader, so a CR2 dies in AssetGenerateThumbnails with
+//      "tiff2vips: Old-style JPEG compression support is not configured"
+//      (issue #99). Docker's libvips (fuller libtiff plus libraw) reads it.
 //
 // Fix: preload this module via `NODE_OPTIONS=--require ...` and wrap the
-// `sharp` module export. When Sharp is handed a HEVC-family HEIC file path, we
-// transcode it to a lossless intermediate with an external decoder and feed
-// that buffer to the real Sharp instead. Everything else (JPEG, PNG, AVIF,
-// buffers, streams) passes through untouched. Immich's source on disk is never
-// modified — same interposition pattern as the ffmpeg wrapper and pg_dump shim.
+// `sharp` module export. When Sharp is handed a HEVC-HEIC or camera-RAW file
+// path, we transcode it to a lossless TIFF with Homebrew's fuller libvips and
+// feed that buffer to the real Sharp instead. Everything else (JPEG, PNG, AVIF,
+// ordinary TIFF, buffers, streams) passes through untouched. Immich's source on
+// disk is never modified, the same interposition pattern as the ffmpeg wrapper
+// and pg_dump shim.
 //
 // Decoder preference (first that yields a valid TIFF wins):
 //   1. libheif via `vips autorot` (Homebrew libvips built WITH libde265) — a
@@ -73,8 +82,14 @@ const SIPS = findBin('IMMICH_ACCELERATOR_SIPS', ['/usr/bin/sips']);
 const DECODERS = [
     {
         name: 'vips',
+        // Write the intermediate TIFF with lossless deflate compression. Sharp
+        // ingests the whole file into a Node Buffer, and a camera RAW decodes to
+        // a large frame (tens of MB uncompressed, ~40% smaller deflated), so this
+        // trims peak memory and tmp I/O for no pixel change and negligible CPU
+        // (the decode dominates). The magic bytes are unaffected, so isTiff still
+        // gates it. sips (fallback) has no equivalent option and is left plain.
         bin: VIPS,
-        args: (input, out) => ['autorot', input, out],
+        args: (input, out) => ['autorot', input, `${out}[compression=deflate]`],
     },
     {
         name: 'sips',
@@ -86,7 +101,7 @@ const DECODERS = [
 // One-time startup line so the worker log shows the shim is active and which
 // HEIC decoders it resolved (invaluable when HEIC thumbnails misbehave).
 process.stderr.write(
-    '[immich-accelerator] heic-decode-shim active ' +
+    '[immich-accelerator] heic/raw decode shim active ' +
     `(decoders: vips=${VIPS || 'no'}, sips=${SIPS || 'no'})\n`
 );
 
@@ -131,6 +146,44 @@ function isHevcHeicPath(input) {
     return brands !== null && brands.some((b) => HEVC_BRANDS.has(b));
 }
 
+// Camera-RAW extensions that Sharp's bundled libvips cannot decode on macOS
+// (its libtiff/libjpeg lack old-style-JPEG support and it has no dcraw/libraw
+// loader) but Homebrew libvips can. RAW is matched by EXTENSION, not magic:
+// most RAW containers are TIFF-based, so their magic bytes don't reliably
+// separate them from an ordinary TIFF that Sharp already handles. Routing these
+// through `vips autorot` mirrors Docker Immich's own libvips+libraw path; if a
+// file with one of these extensions is not actually decodable RAW, the decoder
+// fails the TIFF check and we fall back to the real Sharp (no worse than now).
+// Generic `.raw` is intentionally omitted: it is too ambiguous (also raw audio,
+// disk images, etc.) to intercept safely.
+const RAW_EXTS = new Set([
+    'cr2', 'cr3', 'crw',          // Canon
+    'nef', 'nrw',                 // Nikon
+    'arw', 'sr2', 'srf',          // Sony
+    'raf',                        // Fujifilm
+    'orf', 'ori',                 // Olympus
+    'rw2',                        // Panasonic
+    'pef',                        // Pentax
+    'dng',                        // Adobe / generic
+    'rwl',                        // Leica
+    'dcr', 'kdc', 'k25',          // Kodak
+    'mrw',                        // Minolta
+    'x3f',                        // Sigma
+    '3fr', 'fff',                 // Hasselblad
+    'mos', 'iiq',                 // Leaf / Phase One
+    'erf',                        // Epson
+    'mef',                        // Mamiya
+    'mdc',                        // Minolta / Agfa
+    'ari', 'cap',                 // Arri / others
+]);
+
+function isRawPath(input) {
+    if (typeof input !== 'string' || input.length === 0) return false;
+    const dot = input.lastIndexOf('.');
+    if (dot < 0) return false;
+    return RAW_EXTS.has(input.slice(dot + 1).toLowerCase());
+}
+
 // The decode runs synchronously in the sharp() constructor, so a hung decoder
 // stalls the worker's event loop. Cap the TOTAL time across all decoders (not
 // per-decoder) so trying the fallback can't multiply the worst case.
@@ -148,6 +201,26 @@ function isTiff(buf) {
     );
 }
 
+// Sharp's prebuilt libvips exports VIPSHOME into the process environment,
+// pointing at the build tree it was compiled in (a GitHub-runner path baked into
+// the binary, e.g. /Users/runner/work/sharp-libvips/...). When we spawn the
+// Homebrew `vips`, it inherits that VIPSHOME and hunts for its loader modules
+// (libheif for HEIC, etc.) under that bogus directory, so the decode fails with
+// "not a known file format" and silently falls back to sips (which needs a GUI
+// session, so it produces nothing headless). If VIPSHOME points at a directory
+// that does not exist, drop it from the child's env so the spawned vips uses its
+// own compiled-in module path. A real, existing VIPSHOME (an intentional
+// operator override) is left untouched.
+function decoderEnv() {
+    const home = process.env.VIPSHOME;
+    if (home && !fs.existsSync(home)) {
+        const env = { ...process.env };
+        delete env.VIPSHOME;
+        return env;
+    }
+    return process.env;
+}
+
 // Decode a HEIC file to a lossless TIFF buffer, trying each available decoder in
 // preference order until one yields a valid TIFF. Returns the buffer, or null if
 // every decoder is unavailable or fails (caller falls back to the real Sharp, so
@@ -168,6 +241,7 @@ function decodeToBuffer(input) {
             cp.execFileSync(dec.bin, dec.args(input, tmp), {
                 stdio: ['ignore', 'ignore', 'pipe'],
                 timeout: remaining,
+                env: decoderEnv(),
             });
             const buf = fs.readFileSync(tmp);
             if (isTiff(buf)) return buf;
@@ -186,8 +260,8 @@ function decodeToBuffer(input) {
     }
     if (attempted === 0) {
         process.stderr.write(
-            '[immich-accelerator] No HEIC decoder found (install vips for ' +
-            'headless HEIC support); falling back to Sharp.\n'
+            '[immich-accelerator] No HEIC/RAW decoder found (install vips for ' +
+            'headless HEIC and camera-RAW support); falling back to Sharp.\n'
         );
     }
     return null;
@@ -196,7 +270,9 @@ function decodeToBuffer(input) {
 // Wrap the real Sharp factory so HEVC-HEIC paths are pre-decoded.
 function wrapSharp(realSharp) {
     function sharp(input, options) {
-        if (isHevcHeicPath(input)) {
+        // isRawPath is a pure string check; isHevcHeicPath reads the ftyp box.
+        // Test the cheap one first so RAW paths route with no filesystem I/O.
+        if (isRawPath(input) || isHevcHeicPath(input)) {
             const buf = decodeToBuffer(input);
             if (buf !== null) {
                 return realSharp(buf, options);

tests/test_fresh_install.py

@@ -1379,9 +1379,10 @@ def test_create_app_succeeds_with_minimal_deps(self, tmp_path):
 
 
 class TestHeicDecodeShim:
-    """The HEIC decode shim (issue #62 follow-up) interposes on require('sharp')
-    to route HEVC-HEIC paths through Apple's ImageIO. These guard the JS logic
-    without needing real Sharp/sips (the real decode is verified on-device)."""
+    """The HEIC/RAW decode shim (issues #62, #99) interposes on require('sharp')
+    to route HEVC-HEIC paths (by ftyp brand) and camera-RAW paths (by extension)
+    through Homebrew libvips. These guard the JS logic without needing real
+    Sharp/vips (the real decode is verified on-device)."""
 
     SHIM = REPO_ROOT / "immich_accelerator" / "hooks" / "heic_decode_shim.js"
 
@@ -1425,8 +1426,11 @@ def _fake_sharp(self, tmp_path):
 
     def _stub(self, path_, body):
         # Decoder stubs: the output path is always the last argument (vips
-        # `autorot in out`, sips `... --out out`).
-        path_.write_text('#!/bin/bash\nout="${@: -1}"\n' + body)
+        # `autorot in out[compression=deflate]`, sips `... --out out`). vips
+        # accepts a trailing `[options]` suffix on the save target and writes to
+        # the base path; strip it (`[[]` is a glob for a literal `[`) so the stub
+        # writes where the shim reads back.
+        path_.write_text('#!/bin/bash\nout="${@: -1}"\nout="${out%%[[]*}"\n' + body)
         path_.chmod(0o755)
 
     # A minimal little-endian TIFF header (II*\0) — enough for the shim's isTiff
@@ -1531,6 +1535,69 @@ def test_falls_through_to_sips_when_vips_produces_empty(self, tmp_path):
         assert "INPUT_TYPE:buffer" in result.stdout, result.stdout
         assert sips_used.exists(), "must fall through to sips when vips output is empty"
 
+    def test_routes_raw_by_extension_and_leaves_ordinary_tiff_alone(self, tmp_path):
+        """Camera RAW is matched by EXTENSION (issue #99): a .cr2/.nef path is
+        decoded to a Buffer via vips, while an ordinary .tif (which Sharp handles
+        natively) passes through untouched as a string. Guards against the
+        extension set over-matching plain TIFFs."""
+        node = self._node_or_skip()
+        self._fake_sharp(tmp_path)
+        vips = tmp_path / "vips_stub.sh"
+        self._stub(vips, self._TIFF)
+        cr2 = tmp_path / "photo.cr2"
+        cr2.write_bytes(b"content-irrelevant-extension-is-what-matters")
+        nef = tmp_path / "photo.nef"
+        nef.write_bytes(b"II*\x00whatever")
+        plain_tif = tmp_path / "scan.tif"
+        plain_tif.write_bytes(b"II*\x00ordinary-tiff-sharp-handles-this")
+
+        result = self._run(
+            tmp_path,
+            node,
+            {"IMMICH_ACCELERATOR_VIPS": str(vips)},
+            [str(cr2), str(nef), str(plain_tif)],
+        )
+        assert result.returncode == 0, f"driver failed: {result.stderr}"
+        types = [ln for ln in result.stdout.splitlines() if ln.startswith("INPUT_TYPE:")]
+        assert types == [
+            "INPUT_TYPE:buffer",
+            "INPUT_TYPE:buffer",
+            "INPUT_TYPE:string",
+        ], result.stdout
+
+    def test_strips_nonexistent_vipshome_from_decoder_env(self, tmp_path):
+        """Sharp's prebuilt libvips exports a bogus VIPSHOME (its GitHub-runner
+        build path) into the environment. If the spawned Homebrew vips inherits
+        it, vips looks for its loader modules there, fails ("not a known file
+        format"), and decode silently falls to sips (GUI-only). The shim drops a
+        VIPSHOME pointing at a nonexistent dir but keeps a real one."""
+        node = self._node_or_skip()
+        self._fake_sharp(tmp_path)
+        marker = tmp_path / "vipshome_seen"
+        vips = tmp_path / "vips_stub.sh"
+        self._stub(vips, 'echo "[${VIPSHOME-UNSET}]" > ' + str(marker) + '\n' + self._TIFF)
+        heic = tmp_path / "photo.heic"
+        heic.write_bytes(self._HEIC_BYTES)
+
+        bogus = tmp_path / "does-not-exist"
+        r = self._run(
+            tmp_path, node,
+            {"IMMICH_ACCELERATOR_VIPS": str(vips), "VIPSHOME": str(bogus)},
+            [str(heic)],
+        )
+        assert r.returncode == 0, r.stderr
+        assert marker.read_text().strip() == "[UNSET]", "nonexistent VIPSHOME must be stripped"
+
+        real = tmp_path / "real-vipshome"
+        real.mkdir()
+        r2 = self._run(
+            tmp_path, node,
+            {"IMMICH_ACCELERATOR_VIPS": str(vips), "VIPSHOME": str(real)},
+            [str(heic)],
+        )
+        assert r2.returncode == 0, r2.stderr
+        assert marker.read_text().strip() == "[" + str(real) + "]", "existing VIPSHOME must be preserved"
+
 class TestPgKeepaliveShim:
     """The pg keepalive shim sets keepAlive on Immich's Postgres connections so
     a stateful firewall between worker and a remote DB can't reap idle