Thousands of local discard duplicate when I dont have any duplicates at all

[Nurul9615]

Hi,

I recently tried to use immich-go to import my google takeout photos into Immich running on my NAS. I have 10 files of 50GB each.
According to my Google storage quota, its actually 271GB used by Google Photos, but the takeout is nearly 500GB. I have another person on Google Family with 159GB so that might also be why, but not sure why I'd have their library exported on my own personal takeout.

Regardless, after immich-go ran for about 10 hours, it found 37,000 local duplicates. So only 83GB worth of data actually got imported. I'm pretty sure I don't have duplicates as I do actually have 271GB worth of unique content, so where is this coming from?

Thanks

immich-go upload from-google-photos   --server=http://xxx.xxx.xxx.xxx:2283   
--api-key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx   --concurrent-tasks=4  
 --client-timeout=60m   --pause-immich-jobs=true   --on-errors=continue   --session-tag   
/mnt/ugreen/takeouts/takeout-*.zip

2025-12-17 19:13:15 INF Immich Job command sent resume=metadataExtraction
2025-12-17 19:13:15 INF Immich Job command sent resume=videoConversion
2025-12-17 19:13:15 INF Immich Job command sent resume=faceDetection
2025-12-17 19:13:15 INF Immich Job command sent resume=smartSearch
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF Asset Tracking Report:
2025-12-17 19:13:15 INF =====================
2025-12-17 19:13:15 INF Total Assets:      46162  (499.1 GB)
2025-12-17 19:13:15 INF   Processed:        7385  (82.6 GB)
2025-12-17 19:13:15 INF   Discarded:        1676  (14.8 GB)
2025-12-17 19:13:15 INF   Errors:             47  (790.7 MB)
2025-12-17 19:13:15 INF   Pending:         37054  (400.9 GB)
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF ⚠  WARNING: 37054 assets did not reach a final state!
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF Event Report:
2025-12-17 19:13:15 INF =============
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF Discovery (Assets):
2025-12-17 19:13:15 INF   discovered image                   :   36584  (101.3 GB)
2025-12-17 19:13:15 INF   discovered video                   :    9538  (397.4 GB)
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF Discovery (Non-Assets):
2025-12-17 19:13:15 INF   discovered sidecar                 :   47894  (34.1 MB)
2025-12-17 19:13:15 INF   discovered unsupported file        :      72  (16.5 KB)
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF Asset Lifecycle (PROCESSED):
2025-12-17 19:13:15 INF   uploaded successfully              :    7380  (82.6 GB)
2025-12-17 19:13:15 INF   server asset upgraded              :       3  (2.1 MB)
2025-12-17 19:13:15 INF   metadata updated                   :    7378  (228.6 KB)
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF Asset Lifecycle (DISCARDED):
2025-12-17 19:13:15 INF   server has duplicate               :       1  (119.7 KB)
2025-12-17 19:13:15 INF   discarded filtered                 :     543  (761.6 MB)
2025-12-17 19:13:15 INF   discarded local duplicate          :   38001  (413.0 GB)
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF Asset Lifecycle (ERROR):
2025-12-17 19:13:15 INF   server error                       :      51
2025-12-17 19:13:15 INF 
2025-12-17 19:13:15 INF Processing Events:
2025-12-17 19:13:15 INF   associated metadata                :    9068
2025-12-17 19:13:15 INF   missing metadata                   :     187
2025-12-17 19:13:15 INF   stacked                            :      21
2025-12-17 19:13:15 INF   added to album                     :    5681
2025-12-17 19:13:15 INF   tagged                             :   14755
2025-12-17 19:13:15 INF 
2025-12-17 20:29:43 ERR Some errors have occurred. Look at the log file for details

[nurul@nurul-xxx]$ ls -la /mnt/ugreen/takeouts/
total 523412856
drwxr-xr-x 2 nurul nurul           0 Dec 17 09:13 .
drwxr-xr-x 2 nurul nurul           0 Dec 17 09:12 ..
-rwxr-xr-x 1 nurul nurul 53576174699 Dec 16 22:25 takeout-20251215T195509Z-3-001.zip
-rwxr-xr-x 1 nurul nurul 53576174699 Dec 16 22:26 takeout-20251215T195509Z-3-002.zip
-rwxr-xr-x 1 nurul nurul 53576174699 Dec 16 23:05 takeout-20251215T195509Z-3-003.zip
-rwxr-xr-x 1 nurul nurul 53576174699 Dec 16 23:05 takeout-20251215T195509Z-3-004.zip
-rwxr-xr-x 1 nurul nurul 53576174699 Dec 16 23:38 takeout-20251215T195509Z-3-005.zip
-rwxr-xr-x 1 nurul nurul 53576174699 Dec 16 23:37 takeout-20251215T195509Z-3-006.zip
-rwxr-xr-x 1 nurul nurul 53576174699 Dec 17 00:02 takeout-20251215T195509Z-3-007.zip
-rwxr-xr-x 1 nurul nurul 53576174699 Dec 17 00:09 takeout-20251215T195509Z-3-008.zip
-rwxr-xr-x 1 nurul nurul 53682561194 Dec 17 07:51 takeout-20251215T195509Z-3-009.zip
-rwxr-xr-x 1 nurul nurul 53682561194 Dec 17 07:49 takeout-20251215T195509Z-3-010.zip

[Nectulus]

Hi,

I see you closed this - were you able to figure it out? I think I'm having the same issue.

I just did a fresh Immich install so it's completely empty. I did my first Google Photos takeout upload and out of 11000+ files it failed 640 duplicates. But it's a complete fresh and empty Immich server so I have no duplicates =/

Thanks

[Nurul9615]

Hi,

I see you closed this - were you able to figure it out? I think I'm having the same issue.

I just did a fresh Immich install so it's completely empty. I did my first Google Photos takeout upload and out of 11000+ files it failed 640 duplicates. But it's a complete fresh and empty Immich server so I have no duplicates =/

Thanks

Hi,

Apologies for the late response.

I solved this by running the 'immich-go upload' command directly on my UGREEN NAS by SSHing into it. Previously I was running it from my local laptop where the --server argument was set to the IP of the Immich NAS host. So it was probably a network bottleneck.