Unable to upload files that are larger than 2 GB on web

[vishwa5854]

The bug

If an upload of a file that is larger than 2GB is initiated on webapp, it would fail. I have looked at the console logs or errors in developer console and seems like it is an issue from the SubtleCrypto library function.

Here's the exact error I see

Error calculating sha1 file=20250204_142925.mp4) TypeError: SubtleCrypto.digest: ArrayBuffer branch of (ArrayBufferView or ArrayBuffer) can't be an ArrayBuffer or an ArrayBufferView larger than 2 GB
    J Immutable
[HQvrpDmE.js:1:3984](https://photos.cruger.in/_app/immutable/chunks/HQvrpDmE.js)
[handleError]: Unable to upload file TypeError: SubtleCrypto.digest: ArrayBuffer branch of (ArrayBufferView or ArrayBuffer) can't be an ArrayBuffer or an ArrayBufferView larger than 2 GB
    J Immutable
 J@https://photos.cruger.in/_app/immutable/chunks/HQvrpDmE.js:1:3688
[CeBM9qXI.js:1:643](https://photos.cruger.in/_app/immutable/chunks/CeBM9qXI.js)
Error calculating sha1 file=20250204_142925.mp4) TypeError: SubtleCrypto.digest: ArrayBuffer branch of (ArrayBufferView or ArrayBuffer) can't be an ArrayBuffer or an ArrayBufferView larger than 2 GB
    J Immutable
[HQvrpDmE.js:1:3984](https://photos.cruger.in/_app/immutable/chunks/HQvrpDmE.js)
[handleError]: Unable to upload file TypeError: SubtleCrypto.digest: ArrayBuffer branch of (ArrayBufferView or ArrayBuffer) can't be an ArrayBuffer or an ArrayBufferView larger than 2 GB
    J Immutable
 J@https://photos.cruger.in/_app/immutable/chunks/HQvrpDmE.js:1:3688

However it gets uploaded with no issues on the mobile app (I'm guessing it is using a different crypto library that doesn't have this issue)

The OS that Immich Server is running on

Debian

Version of Immich Server

v1.125.6

Version of Immich Mobile App

1.125.5.build.181

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - '127.0.0.1:2283:2283'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: >-
        pg_isready --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" || exit 1;
        Chksum="$$(psql --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" --tuples-only --no-align
        --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')";
        echo "checksum failure count is $$Chksum";
        [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: >-
      postgres
      -c shared_preload_libraries=vectors.so
      -c 'search_path="$$user", public, vectors'
      -c logging_collector=on
      -c max_wal_size=2GB
      -c shared_buffers=512MB
      -c wal_compression=on
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=America/New_York

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=***************

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. Create a video that is very large specifically greater than 2GB
2. Try to upload it to immich using the web app on phone or desktop
3. You will see an error message saying upload failed
   ...

Relevant log output

Error calculating sha1 file=20250204_142925.mp4) TypeError: SubtleCrypto.digest: ArrayBuffer branch of (ArrayBufferView or ArrayBuffer) can't be an ArrayBuffer or an ArrayBufferView larger than 2 GB
    J Immutable
[HQvrpDmE.js:1:3984](https://photos.cruger.in/_app/immutable/chunks/HQvrpDmE.js)
[handleError]: Unable to upload file TypeError: SubtleCrypto.digest: ArrayBuffer branch of (ArrayBufferView or ArrayBuffer) can't be an ArrayBuffer or an ArrayBufferView larger than 2 GB
    J Immutable
 J@https://photos.cruger.in/_app/immutable/chunks/HQvrpDmE.js:1:3688
[CeBM9qXI.js:1:643](https://photos.cruger.in/_app/immutable/chunks/CeBM9qXI.js)
Error calculating sha1 file=20250204_142925.mp4) TypeError: SubtleCrypto.digest: ArrayBuffer branch of (ArrayBufferView or ArrayBuffer) can't be an ArrayBuffer or an ArrayBufferView larger than 2 GB
    J Immutable
[HQvrpDmE.js:1:3984](https://photos.cruger.in/_app/immutable/chunks/HQvrpDmE.js)
[handleError]: Unable to upload file TypeError: SubtleCrypto.digest: ArrayBuffer branch of (ArrayBufferView or ArrayBuffer) can't be an ArrayBuffer or an ArrayBufferView larger than 2 GB
    J Immutable
 J@https://photos.cruger.in/_app/immutable/chunks/HQvrpDmE.js:1:3688

Additional information

No response

[schuhbacca]

Please attempt over local IP.

[schuhbacca]

Your proxy is misconfigured or you are using cloudflare. There's probably some upload limit in the chain.

[vishwa5854]

Well I configured my DNS to use local IP whenever I am home, and the nginx reverse proxy is configured to accept requests in LAN with the same cert, so it's definitely not cloudflare. Let me share my nginx configuration and maybe something is wrong over there.

DNS Lookup

z@fedora:~$ nslookup photos.cruger.in
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	photos.cruger.in
Address: 10.0.0.20

Nginx Config

server{
    listen 443 ssl http2; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/photos.cruger.in/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/photos.cruger.in/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

    # OCSP stapling
    ssl_stapling on;
    ssl_stapling_verify on;

    http2_body_preread_size 256k;

    keepalive_timeout 75s;
    keepalive_requests 100;

    server_name photos.cruger.in;
    
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_pass http://127.0.0.1:5854;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        
        client_max_body_size 0;
        client_body_buffer_size 512k;
        proxy_read_timeout 86400s;
    }
}
server{
    if ($host = photos.cruger.in) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    server_name photos.cruger.in;
    listen 80;
    return 404; # managed by Certbot
}

[vishwa5854]

Since client_max_body_size is set to 0 it should be fine with any payload, I have uploaded big files before and it didn't happen before. Is there something I am missing here?

[Mraedis]

nginx has a default limit of 2GB for buffers, try
proxy_request_buffering off;

[vishwa5854]

Okay, let me try that. Also a quick side question how the heck did it work on mobile without any issues then? 😅

[mmomjian]

Please upgrade to 1.125.7 as well, there is a patch for this

[vishwa5854]

Ohhh, can you link me to the code change please?

[bo0tzz]

tbc this isn't a proxy issue but a rare case of #15739

[vishwa5854]

so basically on LAN the crypto module will altogether skip the hashing part hence it works on LAN since the browser considers it as insecure. Got it! Thanks for linking me to the issue.

[mmomjian]

It seems like you’re using CloudFlare and SSL, is that correct? In that case this error is probably due to the CloudFlare max payload size, not the HTTP context

[vishwa5854]

as mentioned above, whenever I am in my home, my DNS resolves to my local server IP, and there is an instance of nginx running with certificates for same public domain in the LAN, so it wouldn't go via cloudflare when I am home.

[vishwa5854]

It wouldn't have worked for mobile as well if it went through cloudflare since they limit only upto 100 MB

[mmomjian]

I see you are using a real letsencrypt cert. I'm surprised you faced this bug.