Hash files to be able to repair path changes

[oledid]

Before I found Immich, I did a thorough search of options. Among them, an app called stash which I obviously could not use for my private photo and video collection. I seriously tought about forking it and removing all unwanted stuff, but then I found Immich instead.

I noticed a great feature in stash however.

They use file hashing, and connect the file metadata to the hash. So when they do a library scan, they first save the file with the path. And then another job run which scans the file and hash it. And they link the file path against the hash.

The great thing here is: If you move your library files in any way, the app will be able to repair the path automatically because of the file hash, so that the metadata gets attached to the correct file. If you for example rename a folder in an "external library", as immich calls it, it gets fixed automatically.

[mmomjian]

I love this idea. We already store the SHA1 sum of the file in the database as assets.checksum. This would allow for some kind of "self-healing" of the assets and could be a big step towards supporting other storage systems in the future, like S3. Perhaps there could be some kind of "scan for missing files", where we will search for missing assets in a new folder by checksum.

[njhsi]

support this idea too. but I've just got to know that for assets.checksum it's not the case to external library which might be checksum'd of the file path.

file checksumming for all assets is quite needed to me, better using xxhsum, so I could reuse all my backup metadata in future in immich..

[mmomjian]

I don't see the value of using a checksum program that is not installed by default on most Linux distributions. We are not hashing with the volume that sha1sum vs some other program is really worthwhile.

[oledid]

If in doubt that sha1 is good enough, it is a possibility to check again with a stronger hash function only for actual sha1 matches. So you don't have to use the stronger/slower hash function for all files.

[mbethke]

SHA1 is way more than good enough for this purpose. Even if you took an MD5 and truncated it to 64 bit for easier DB storage you'd need about 6 million files in your collection before you had a one in a million chance for a single collision.

[flammermann]

I also love this idea, especially because I have a huge external library, which I would like to reorder without having to repeat Face Detection and Face Recognition.

Does anyone know if there is any progress on this topic or if it is on the roadmap?