Multi archive in one time

[yesvods]

Describe the request

Since remote storage has many disks. Just archive to one disk a time is not enough. Can I rsync to multi remote disk in one time? Thanks a lot!

Images

Additional comments

[randellhodges]

Maybe when the archive process looks for a drive with available storage, it could look for the rsync .tmp file (.whater.plot.XXXX) and if there is one there, skip that and look for another candidate. If it finds a drive without a tmp file, then copy to it.

This would even allow multiple machines running plotman archiving to a single host to coordinate the syncs and maximize network/drive bandwidth.

Of course, with opening up how archiving works, this might be a little harder to assume a way to track "tmp" files.

Quick proof of concept: #642

[syadnom]

I'd rather see archiver pick the remote directory with the most free space. update rsync to use --preallocate and it all should just work out without any extra logic.

If you have multiple plotters pushing to a single remote archive, this solves that as well because --preallocate will handle 'priority' and the additional plotting machines should be doing the same check.

[syadnom]

oh yeah, I'm not discounting the parallel archiving, I'm saying that if each archive process did the check that's basically solved all of this.