iOS save implementation

[alpha0010]

I have had some user reports of corrupt downloaded files (end of file truncated, I think), with my library react-native-file-access. Comparing implementation react-native-blob-courier DownloaderDelegate.swift#L102 and my library NetworkHandler.swift#L65 they differ in FileManager.default.copyItem() vs FileManager.default.moveItem() when download completes.

I am wondering if you have had a particular reason for choosing copy over move, or other thoughts on what might cause this corruption.

[edeckers]

Hi @alpha0010, thank you for your message! Unfortunately I don't have a clean cut answer for you.

It seems to me that there was a real good reason for me to use copy over move for the obvious performance penalty, but I can't find anything about it in my commit history nor do I remember any issues. So let's not rule out a moment of absent-mindedness either.

Can you point me to an error report? Maybe when I read that, some part of the description will stand-out to me and I can be of more help.

[alpha0010]

Trouble is, there is not really a stack trace to go on. Procedure this app takes is:

• Get a list of pdfs (300+ files)
• Loop through and download each
  • If server response is not 200, delete the file
• Some time later (maybe same session, maybe days later) try to read a pdf

On pdf read error, Sentry took a snapshot of data.

{
  "md5": "21f820408ccb544d1093a6ee815aabe5",
  "stat": {
    "lastModified": 1650477306617.6123,
    "path": "/var/mobile/Containers/Data/Application/5FF250E5-0251-44CB-A079-B9B7F84B903B/Documents/accomp-316.pdf",
    "filename": "accomp-316.pdf",
    "size": 573440,
    "type": "file"
  }
}

Comparing to server side:

$ ll 316.pdf 
-rw-rw-r-- 1 root root 803504 Jul  5  2020 316.pdf
$ md5sum 316.pdf 
f03688e9f4c797038c41f2183d84ed71  316.pdf
$ head -c 573440 316.pdf | md5sum
21f820408ccb544d1093a6ee815aabe5  -

Looks like it really is just the end of the file is lost.

[alpha0010]

Of course, there is the chance issue is server side. That code is decades old, which should mean it is stable. (Or maybe that it has finally broken.)

[alpha0010]

If the app detects an error when trying to read a pdf, and downloads it again, the new file works fine.

[edeckers]

Thank you for the extra information :)

Unfortunately this is still not much for me to work with, but the more I think about it, the less likely I feel that the problem lies with moveItem: it's an OS-handled action which you seem to use it just fine and works like it should in most cases. And I would definitly be baffled when it turns out using copyItem instead of moveItem would fix this particular issue. I am really curious about it though, so I'm really keen to know what you find out!

You probably investigated the steps below already, but if you didn't, these checks would be where I'd start:

1. Client/server connection stability
2. Client disk space and memory, 'interesting things' might happen when downloading this many files especially in parallel
3. Server load, logs (access + error)
4. Compare file cut-off sizes, see if you can find a pattern
5. Find out if there's some memory limit in your apache / nginx / haproxy / php-fpm etc. configuration that would cause files to semi-randomly cut off

I hope that doesn't come off as patronizing or something, just wanted to share my 2 cents in case it might help you!

[alpha0010]

Thanks for the ideas; I had not come up with all those (too much focus on the newer client side code). Will keep you posted if I discover anything.