Manually paused jobs are restarted even if --pause-immich-jobs is false

[mihalyr]

I am just trying immich-go as I like some of the features it provides and seems to be very fast. However, it is constantly failing with random errors without any proper explanation in the logs either on server or the immich-go logs (which just say 400 or EOF, it could be that my system is choking, though).

I also noticed that one of the failures was caused by immich-go not pausing jobs correctly so some concurrent jobs did interfere with the process. To avoid this, I decided to manually pause all my jobs until I upload my collection. Now, immich-go fails with these EOFs randomly so I have to keep retrying, but every time it fails/stops it restart all my manually paused jobs even if I set --pause-immich-jobs=false. My expectation is that if it did not touch any job to pause, it should not touch any job to resume either.

My current workaround is to just not give it the admin api key. It works, but it still tries to access the /job path, but since it is admin only, it gets a 403.

[mihalyr]

Regarding those EOFs and random failures, first I thought my system might be choking so reduced concurrency to 1, but the same thing happened and realized it always happens after 10GB new data is processed. First failure was at 10GB, then I retried on same input, then it got to 20GB, now I tried again from start with 1 thread, stopped at 30GB.

And every time it stops it sends the command, tries to at least, even with --pause-immich-jobs=false and without admin api key set:

2026-01-22 23:27:51 ERR Immich Job command sent resume=thumbnailGeneration err=SendJobCommand, PUT, http://localhost:2283/api/jobs/thumbnailGeneration, 403 Forbidden
Forbidden

Regarding those failures every 10GB all I see in logs are the following before it stops:

2026-01-22 23:27:36 ERR server error file=Photos:myphotos/.picasaoriginals/pic1.jpg error=CopyAsset, PUT, http://localhost:2283/api/assets/copy, 500 Internal Server Error
Failed to copy asset

2026-01-22 23:27:36 ERR Error err=CopyAsset, PUT, http://localhost:2283/api/assets/copy, 500 Internal Server Error
Failed to copy asset

2026-01-22 23:27:36 ERR server error file=Photos:myphotos/.picasaoriginals/pic2.jpg error=AssetUpload, POST, http://localhost:2283/api/assets

io: read/write on closed pipe
2026-01-22 23:27:36 ERR Error err=AssetUpload, POST, http://localhost:2283/api/assets

io: read/write on closed pipe
2026-01-22 23:27:39 ERR failed to create album err=CreateAlbum, POST, http://localhost:2283/api/albums, 400 Bad Request

 album=Folder / SubFolder

And this one is related on the server side:

[Nest] 23  - 01/22/2026, 11:27:36 PM   ERROR [Api:ErrorInterceptor~g7emdt6b] Unknown error: Error: ENOENT: no such file or directory, copyfile '/data/upload/24ab820b-3031-4715-a389-bbf02b911039/ec/63/ec639b10-3fb0-4a97-ba03-8ea398ecf21c.jpg.xmp' -> '/data/upload/24ab820b-3031-4715-a389-bbf02b911039/63/d2/63d2e6bc-5ce2-4b16-9d89-81e1c891bf88.jpg.xmp'
Error: ENOENT: no such file or directory, copyfile '/data/upload/24ab820b-3031-4715-a389-bbf02b911039/ec/63/ec639b10-3fb0-4a97-ba03-8ea398ecf21c.jpg.xmp' -> '/data/upload/24ab820b-3031-4715-a389-bbf02b911039/63/d2/63d2e6bc-5ce2-4b16-9d89-81e1c891bf88.jpg.xmp'
    at async Object.copyFile (node:internal/fs/promises:621:10)

It is unclear what is happening and why exactly at the 10GB boundary.

[mihalyr]

I believe that server error about copyfile xmp from above is a red herring and only a side effect of the extract metadata job that was not paused at the time. Earlier when I ran it with the extract metadata paused too, it also failed at 10GB but the server didn't have that xmp error, instead it logged this:

[Nest] 23  - 01/22/2026, 10:52:02 PM    WARN [Api:FileUploadInterceptor] Request error while uploading file, cleaning up
[Nest] 23  - 01/22/2026, 10:52:02 PM    WARN [Api:FileUploadInterceptor] Error: aborted
    at abortIncoming (node:_http_server:811:17)
    at socketOnClose (node:_http_server:805:3)
    at Socket.emit (node:events:530:35)
    at TCP.<anonymous> (node:net:346:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  code: 'ECONNRESET'
}
[Nest] 23  - 01/22/2026, 10:52:03 PM    WARN [Api:FileUploadInterceptor] Request error while uploading file, cleaning up
[Nest] 23  - 01/22/2026, 10:52:03 PM    WARN [Api:FileUploadInterceptor] Error: aborted
    at abortIncoming (node:_http_server:811:17)
    at socketOnClose (node:_http_server:805:3)
    at Socket.emit (node:events:530:35)
    at TCP.<anonymous> (node:net:346:12)
    at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
  code: 'ECONNRESET'
}

So, the server says it was a request error and aborted. On the client side we just see the connection drop:

2026-01-22 22:52:02 ERR server error file=Folder:SubFolder/PHOTO1.JPG error=AssetUpload, POST, http://localhost:2283/api/assets
Post "http://localhost:2283/api/assets": EOF

2026-01-22 22:52:02 ERR Error err=AssetUpload, POST, http://localhost:2283/api/assets
Post "http://localhost:2283/api/assets": EOF

2026-01-22 22:52:02 ERR server error file=Folder:SubFolder/PHOTO2.JPG error=AssetUpload, POST, http://localhost:2283/api/assets

2026-01-22 22:52:02 ERR Error err=AssetUpload, POST, http://localhost:2283/api/assets

2026-01-22 22:52:02 ERR server error file=Folder:SubFolder/PHOTO3.JPG error=AssetUpload, POST, http://localhost:2283/api/assets
Post "http://localhost:2283/api/assets": EOF

2026-01-22 22:52:02 ERR Error err=AssetUpload, POST, http://localhost:2283/api/assets
Post "http://localhost:2283/api/assets": EOF

2026-01-22 22:52:02 ERR server error file=Folder:SubFolder/PHOTO4.JPG error=AssetUpload, POST, http://localhost:2283/api/assets

2026-01-22 22:52:02 ERR Error err=AssetUpload, POST, http://localhost:2283/api/assets

This is on a Synology, Immich running in Docker on the same host, so sending over localhost only. The images are uploaded from the NAS HDD to Immich that stores them on the same HDD with its DB on a separate SSD.

[mihalyr]

And one last info, when immich-go fails tries to resume jobs it didn't even pause, but it doesn't have permission (no admin key) then it also fails to write the asset tracking report in the log on finish. Because the reporting happens after it tries to resume jobs. Ideally, it should not even try to resume jobs it didn't stop, but as a quick workaround maybe we could just handle gracefully the 403s to avoid the unhandled error, so not giving an admin key gives at least an option to prevent it interfering with jobs.