-
-
Notifications
You must be signed in to change notification settings - Fork 188
Description
I tried to upload google takeout zip file and it was failing due to missing metadata for some of the files. It only uploaded partial set of files from the zip. So I reset the immich server and used the from-folder option was able to upload all files. I was able to upload over 1TB of data successfully.
However later I realized that there is an option for -u in the from-google-photos as well. So I tried using that and reupload the existing zip file so it can populate the missing metadata without uploading all files again. It works up to some extend, however it fails later in the stage after processing some random set of files.
I am running immich-go version 0.31.0 on MacOS Darwin Arm64 platform. My immich-server is running 2.4.1. I have set the concurrent-tasks = 1 to avoid any server side issue but it still fails.
Following is the example of failure. This only fails after having many successful API completed. eg.
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1203.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1203.JPG
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1205.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1205.JPG
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1206.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1206.JPG
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1207.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1207.JPG
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1208.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1208.JPG
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1211.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1211.JPG
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1212.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1212.JPG
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1213.HEIC
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1213.HEIC
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1225.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_1225.JPG
2025-12-31 22:09:02 WRN discarded local duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_0988.JPG reason=already present in input as
unknown
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_3081.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_3081.JPG
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_5141.HEIC
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_5141.HEIC
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_5186.HEIC
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_5186.HEIC
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_5256.HEIC
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_5256.HEIC
2025-12-31 22:09:02 INF server has duplicate file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_5268.JPG
2025-12-31 22:09:02 INF metadata updated file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_5268.JPG
2025-12-31 22:09:02 ERR server error file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_0542.JPG error=io: read/write on closed pipe
2025-12-31 22:09:02 ERR Error err=io: read/write on closed pipe
2025-12-31 22:09:02 ERR server error file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_0989.JPG error=AssetUpload, POST, https://immich-server/api/assets
Post "https://immich-server/api/assets": io: read/write on closed pipe
2025-12-31 22:09:02 ERR server error file=takeout-20251222T185233Z-3-002:Takeout/Google Photos/Art /IMG_0479.JPG error=AssetUpload, POST, https://immich-server/api/assets
Post "https://immich-server/api/assets": io: read/write on closed pipe
I also enabled api-trace to get more details.
Method: POST, URL: https://immich-server/api/assets
Timestamp: 2025-12-31T22:09:01.795-08:00
Headers:
Content-Type: multipart/form-data; boundary=93c8f5b3cade130883901fc6cbaf64d16d4ab4518ddfbe712dce6e539999
X-Api-Key: *********
Accept: application/json
X-Immich-Checksum: 1wvlp3hi0NirDiT+ukaeuuJFQT8=
Body:
[Multipart]
--- Part 1 ---
Content-Disposition: form-data; name="fileCreatedAt"
Content:
2024-04-25T16:48:43.000Z
--- Part 2 ---
Content-Disposition: form-data; name="fileModifiedAt"
Content:
2026-01-01T06:09:01.794Z
--- Part 3 ---
Content-Disposition: form-data; name="isReadOnly"
Content:
false
--- Part 4 ---
Content-Disposition: form-data; name="deviceAssetId"
Content:
IMG_0542.JPG-1076161
--- Part 5 ---
Content-Disposition: form-data; name="deviceId"
Content:
MacStudio-M1-Max.local
--- Part 6 ---
Content-Disposition: form-data; name="assetType"
Content:
image
--- Part 7 ---
Content-Disposition: form-data; name="isFavorite"
Content:
false
--- Part 8 ---
Content-Disposition: form-data; name="fileExtension"
Content:
.JPG
--- Part 9 ---
Content-Disposition: form-data; name="duration"
Content:
00:00:00.000000
--- Part 10 ---
Content-Disposition: form-data; name="visibility"
Content:
timeline
--- Part 11 ---
Content-Disposition: form-data; name="assetData"; filename="IMG_0542.JPG"
Content-Type: application/octet-stream
Error reading part data: unexpected EOF
Error reading part 11: multipart: NextPart: EOF
+---- response ---------------------------------------------------
Timestamp: 2025-12-31T22:09:02.794-08:00
Duration: 999ms
Status: 200 OK: 200
Headers:
Etag: "42-UWN2RfeaDCYSXBMijLlIGFAG59Q"
Vary: Accept-Encoding
X-Immich-Cid: osjqytbm
X-Powered-By: Express
Content-Length: 66
Content-Type: application/json; charset=utf-8
Date: Thu, 01 Jan 2026 06:09:02 GMT
Body:
[JSON]
{"status":"duplicate","id":"9d440e44-5722-4453-a045-2bc7a75119da"}
Failed API request/response log
/---- client #1 request #119 ---------------------------------------------------
Method: POST, URL: https://immich-server/api/assets
Timestamp: 2025-12-31T22:09:01.82-08:00
Headers:
X-Api-Key: *******
Accept: application/json
X-Immich-Checksum: QLb7bHUA82PceLGMFvrRE9Qfhi0=
Content-Type: multipart/form-data; boundary=02ee280d50ec344a2292f8aaabea7c4e933e14e0a2e13c0813e4b01752fd
Body:
[Multipart]
--- Part 1 ---
Content-Disposition: form-data; name="deviceAssetId"
Content:
IMG_0400.JPG-912826
--- Part 2 ---
Content-Disposition: form-data; name="fileCreatedAt"
Content:
2026-01-01T06:09:01.820Z
--- Part 3 ---
Content-Disposition: form-data; name="fileModifiedAt"
Content:
2026-01-01T06:09:01.820Z
--- Part 4 ---
Content-Disposition: form-data; name="isFavorite"
Content:
false
--- Part 5 ---
Content-Disposition: form-data; name="fileExtension"
Content:
.JPG
--- Part 6 ---
Content-Disposition: form-data; name="isReadOnly"
Content:
false
--- Part 7 ---
Content-Disposition: form-data; name="visibility"
Content:
timeline
--- Part 8 ---
Content-Disposition: form-data; name="deviceId"
Content:
MacStudio-M1-Max.local
--- Part 9 ---
Content-Disposition: form-data; name="assetType"
Content:
image
--- Part 10 ---
Content-Disposition: form-data; name="duration"
Content:
00:00:00.000000
--- Part 11 ---
Content-Disposition: form-data; name="assetData"; filename="IMG_0400.JPG"
Content-Type: application/octet-stream
Error reading part data: unexpected EOF
Error reading part 11: multipart: NextPart: EOF
+---- response ---------------------------------------------------
Timestamp: 2025-12-31T22:09:02.797-08:00
Duration: 978ms
Error: io: read/write on closed pipe
\---- client #1 request #119 ---------------------------------------------------
Why does it fail after some files? Is there a way to import metadata only?
I am running immich-server behing traefik proxy. I don't see any logs on the immich-server side or treafik side.