Emoji's in albumn names

[Jazza-231]

Simply put, if you have an emoji in an albumn name, it fails:

164217_473.jpg in Soph💖 failed :(
PowerShell doesnt work :( - report that to @TheLastGimbus on GitHub:

No more context is needed really. I love this tool, thank you @TheLastGimbus

[Wacheee]

I have created a PR that addresses this issue on Windows:
#390

If you want to test it, I have released a version that includes this fix along with other features and pull requests I have submitted:
https://github.com/Wacheee/GooglePhotosTakeoutHelper/releases/tag/v3.6.0-wacheee

If you try it, could you please provide feedback on the PR: #390?

[Jazza-231]

Thank you I will test this, I am also looking into another issue with photos not having their metadata found (there is about 2k photos in date unknown or whatever the folder is called) but this could be a google takeout issue - i will let you know

[Wacheee]

Thank you I will test this, I am also looking into another issue with photos not having their metadata found (there is about 2k photos in date unknown or whatever the folder is called) but this could be a google takeout issue - i will let you know

This may be the same issue discussed here: #353, which prevents the script from reading JSON files with dates. There's also a pull request awaiting review that addresses this problem: #375.

The release version I posted also includes this fix (the script will prompt to remove a suffix from JSON files).

[Jazza-231]

Yes that's most likely the issue, I'll test them both

[Jazza-231]

Brilliant, that has fixed both issues!

I have a question related to google photos in general, if I send these files to my new phone and turn on google photos sync, will it detect that the local files are the same as the backed up files and not back them all up again? i did this 1. as a backup and 2. as a transfer (i wanted to transfer all photos including ones not locally on my phone)

[Wacheee]

Nice, I'm glad to hear that it works for you. Regarding your question, I'm not sure, but maybe Google Photos won't detect that the local files are the same as the ones you backed up before because the metadata varies when you download them and then upload them to the phone. However, I'll test it first with a few files to see what happens in the end.

If you can, could you please provide feedback on these two PRs to help support their approval: #390 and #375
Thank you!

[Jazza-231]

@Wacheee To answer my question, I managed to dig up an old phone of mine to test it, and it worked amazingly. All photos kept the right order, google photos detected them and didn't reupload them, gallery kept the right order, all metadata like the location and stuff was exactly the same, even the albumns detected them. I'm doing a large scale test right now with about 4GB of media using GPTH and Syncthing. I am yet to attempt a locked folder test, this I expect it will not automatically grab the photos and remove them from the general file system (which is what the locked folder does, I think it encrypts them so other apps can't access em)

More info:
GPTH settings, vaguely
All in one folder
Remove the extras from json file names
Reverse shortcut for albumns

A script I wrote to delete all shortcuts from the ALL_PHOTOS folder

Edit:
On conclusion of this large scale test, I forgot of one crucial detail... the -edited suffix on all photos which have been...well, edited. I don't know if the original and edited versions are stored locally or not to be entirely honest, but I am personally happy with simply stripping the -edited tag and overwriting the original file, chances are if I edited it and haven't thought about it since imma be happy with the edit. I'll look into the android file system now for how the edited and original files are stored, maybe there's a special file name we can use to make it properly interact with the files.

Edit 2:
I couldn't find the way edited files are stored sadly. But, the locked folder is not automatically ingested, but does not reupload, and does remove from the main memory of the phone, when the images are manually added back to the locked folder. I would use syncthing to back those up to a seperate folder from all the others, then go through that folder, pressing each one - which opens it in google photos, then send them to the locked folder that way.

[Wacheee]

@Wacheee To answer my question, I managed to dig up an old phone of mine to test it, and it worked amazingly. All photos kept the right order, google photos detected them and didn't reupload them, gallery kept the right order, all metadata like the location and stuff was exactly the same, even the albumns detected them. I'm doing a large scale test right now with about 4GB of media using GPTH and Syncthing. I am yet to attempt a locked folder test, this I expect it will not automatically grab the photos and remove them from the general file system (which is what the locked folder does, I think it encrypts them so other apps can't access em)

More info: GPTH settings, vaguely All in one folder Remove the extras from json file names Reverse shortcut for albumns

A script I wrote to delete all shortcuts from the ALL_PHOTOS folder

Edit: On conclusion of this large scale test, I forgot of one crucial detail... the -edited suffix on all photos which have been...well, edited. I don't know if the original and edited versions are stored locally or not to be entirely honest, but I am personally happy with simply stripping the -edited tag and overwriting the original file, chances are if I edited it and haven't thought about it since imma be happy with the edit. I'll look into the android file system now for how the edited and original files are stored, maybe there's a special file name we can use to make it properly interact with the files.

Thanks for the info! About the edited photos, it's a bit complicated. I think every app handles the original and edited photos differently. If you edited the photo using Google Photos, it creates a new copy with the suffix "-edited" (but in your language, in my case, "-editado"). The original remains hidden within Google Photos, so you only see one version in the app. However, if you do a Takeout, you'll see that both the original and the edited version exist as separate files (with the suffix).

This can be a problem with GPTH because there's only one JSON file for both images. So, if the photo doesn't have a date and the script has to use the JSON file to retrieve it, the version with the suffix will fail (but this happens very rarely because, in most cases, the date is either stored within the file itself or in its filename).

[Jazza-231]

@Wacheee I'm unsure where to contact you other than here, but you seem to get things done (thank you!), so I have a feature idea:

In my case where I want to transfer from an old phone to a new one, going back to google photos, knowing the device folder (where the image was stored on the android device) (under googlePhotosOrigin > it can be mobileUpload or webUpload etc, then > deviceFolder > localFolderName > "Instagram"), so I can trasnfer these files to the same folder on the new phone.

Could this be an option, similar to the albumns option? Sorting not albumns into folders but the origin folder name into folders (with unknown and web uploads handled ofc)? I understand this is most likely the wrong place but hopefully my offence is not too agregious.

[Jazza-231]

I did some testing with the -edited suffix, even if you remove the original file before running gpth, and remove the -edited from the second image the metadata is still vastly different which means google doesn't recognise it - every photo I've edited has the same name and basic metadata, but more complex stuff including the Image Unique ID and more do not match. I don't think this is a solvable issue to be entirely honest.

Let me rewrite that, as I'm sure it's wrong.
I search for any images with -edited, find and delete their non -edited counterparts. I then rename the ones containing -edited to remove it.
Even with this happening before running GPTH, google photos does not properly pick up the photos. Downloading the photo using google photo's built in download button returns a file with the exact same metadata, bar file dates. This seems to imply that the photos should be accepted as identical, yet they are not. This may be a google photos issue, because even the photo downloaded directly in the google photos app isn't recognised, and the download button is still visible.

After even more testing, this is a google photos issue. In an albumn, even if it displays the edited version of the photo, the metadata it expects is that of the original version of the photo. I know this because the file size of the photo in the albumn is reported as 1019 kB, yet the file size of the edited version is only 234 kB, despite that being the version visually shown.

[Wacheee]

@Wacheee I'm unsure where to contact you other than here, but you seem to get things done (thank you!), so I have a feature idea:

In my case where I want to transfer from an old phone to a new one, going back to google photos, knowing the device folder (where the image was stored on the android device) (under googlePhotosOrigin > it can be mobileUpload or webUpload etc, then > deviceFolder > localFolderName > "Instagram"), so I can trasnfer these files to the same folder on the new phone.

Could this be an option, similar to the albumns option? Sorting not albumns into folders but the origin folder name into folders (with unknown and web uploads handled ofc)? I understand this is most likely the wrong place but hopefully my offence is not too agregious.

That is interesting, sadly, there are some inconsistencies with those fields. First, you mention that it can be either mobileUpload or webUpload, but I also see fromPartnerSharing, so there might be more possible field names.
Another issue is that localFolderName does not always contain the folder name. For example, when the photo is in the DCIM folder, the field is empty.
I think it is possible to have a feature with that option, but it would require some time and testing to avoid issues.

I recommend that you create a new "Issue" so that this request gains visibility.

[Wacheee]

After even more testing, this is a google photos issue. In an albumn, even if it displays the edited version of the photo, the metadata it expects is that of the original version of the photo. I know this because the file size of the photo in the albumn is reported as 1019 kB, yet the file size of the edited version is only 234 kB, despite that being the version visually shown.

Yes, I have seen that behavior before. I think Google Photos compresses images when you edit them, so be careful when deleting the original photo, as the edited one might have lower quality.