zip compat issue with GoldenCheetah's qzip

[mayfield]

This may be too esoteric to solve, but I'm hoping you might be able to help me figure this out.

ZIP bundles I'm creating are failing to parse properly with this zip library:
https://github.com/GoldenCheetah/GoldenCheetah/tree/master/contrib/qzip

I wish I could link to the main repo for "qzip" but I can't find it. Its very possibly not maintained outside that aforementioned repo. Unfortunately GoldenCheetah is one of my primary targets for zip import, so it's not a good look for me to have a compatibility issue with them.

The error triggered is "File does not exist" -> https://github.com/GoldenCheetah/GoldenCheetah/blob/master/src/Gui/RideImportWizard.cpp#L492

Also of interest is that GC (GoldenCheetah) does list the file(s) but still fails..

I've tried zipSync and Zip (streaming) but both fail in the same way. I tried with and without directory structures. Sidebar, I noticed that when I use Zip and I have a directory structure for the files, there is no entry for the directory. I thought maybe that was the issue, but after manually adding empty directory entries it still doesn't work, and also zipSync does create these entries (it uses ZipDeflate on them btw, which actually uses more space than ZipPassThrough).

In all cases if I take the zip file and unzip on my linux box, then rezip with
zip -r works.zip JenBelt the resulting zip based file works as expected. I should note that unzip and zipdetails think the fflate based files are fine too. As does every other program I've tried. Unfortunately this app this the most important one for me.

Here are the actual files and read outs from 'zipdetails -v'...

zipdetails-linux-zip.txt
linux.zip
zipdetails-zipSync.txt
fflate-zipSync.zip

[101arrowz]

Strange, I'll take a look at the source code and see if I can find what the issue is but I've never used GoldenCheetah before so I don't know how likely I'll be able to find anything. More than likely qzip, which as far as I can tell is copy-pasted from an internal part of Qt's source code, isn't spec compliant. If it's something obvious I'll send a fix when I have time.

Sidebar, I noticed that when I use Zip and I have a directory structure for the files, there is no entry for the directory.

This is actually intentional. Consumer OSes (Windows, Mac, Android, etc.) don't care at all whether or not you include the directories as zero-byte entries in the ZIP. As far as I can tell the empty files aren't required or even mentioned by the ZIP spec. I only do it for zipSync because some pedantic and esoteric software DOES care about, and zipSync is meant to be a convenience method for Zip, which is more imperative but more customizable.

You can also avoid wasting space in your ZIP if you use full filepaths instead of nested objects:

zipSync({
  'no/dirs/created/a.txt': strToU8('no empty entries created for each directory'),
  dirs: {
    'created.txt': strToU8('here there will be an empty "dirs/" entry created')
  }
})

[mayfield]

More than likely qzip, which as far as I can tell is copy-pasted from an internal part of Qt's source code, isn't spec compliant. If it's something obvious I'll send a fix when I have time.

I'm guessing the same thing. You're champion for even taking a look, much appreciated.

This is actually intentional. Consumer OSes (Windows, Mac, Android, etc.) don't care at all whether or not you include the directories as zero-byte entries in the ZIP. As far as I can tell the empty files aren't required or even mentioned by the ZIP spec. I only do it for zipSync because some pedantic and esoteric software DOES care about, and zipSync is meant to be a convenience method for Zip, which is more imperative but more customizable.

That makes perfect sense, thank you very much for the explanation!

[101arrowz]

Just downloaded Golden Cheetah on Windows 10 and it looks like 7-Zip and Windows Explorer's ZIP don't work either. Info-ZIP, as you mentioned, does work. If you can reproduce those issues, I'm guessing that this is an issue you might want to bring up with the Golden Cheetah devs as well.

Which brings me to my findings. It seems that GoldenCheetah doesn't like the empty directories. I think it tries to create a file for each entry in the archive, including the empty directory entry. Those end with slashes, so when the filesystem complains, qzip crashes and Golden Cheetah simply stops decompressing the archive. Basically if you have an empty directory entry, every file after that won't work.

EXCEPT, apparently qzip is able to handle files generated in Unix. ZIP files have a tag specifying the OS, I think what's happening is when qzip detects a Unix file and sees the Unix metadata, it somehow stops trying to create files for the directory entries. I'd recommend telling the Golden Cheetah maintainers to switch to a better ZIP library, say 7zip.

With all that being said, I found a solution. I know you mentioned that you tested without directory structures but I think you might be mistaken because the following worked for me:

const data = fflate.zipSync({
  // Without { JenBelt: { ... } }, this works fine
  'JenBelt/268741814.fit': new Uint8Array([...]) // your data
})

[mayfield]

Dude, you're awesome. You're right about the dir-less working. I just retested that, with the streaming interface actually, and it worked.

I already created an issue on GC's GH with a ref to this, but I'll update that with some more of the context you've provided.

Thank you again!