bpcp: reduce round trips for recursive tree copies

[dcpagotto]

hi all,

this updates the recursive remote copy path so bpcp can ask for a recursive dir listing in one request when the responder supports it. this should avoid the extra per-directory listing round trips on larger trees.

i kept the legacy behavior available with -x, and the client still falls back when talking to older responders. the responder also skips entries it cannot safely include instead of failing the whole listing.

main things changed:

• add recursive dir listing option for the existing extended request
• use it by default for bpcp -r remote copies
• keep -x for the old iterative traversal
• reject unsafe remote manifest paths
• document the new bpcp and cfdptest behavior

local checks run:

• cfdp dev build for x86_64-linux
• bpcp, bpcpd, cfdptest and libcfdp build/link with -wall -werror
• podchecker on updated manpages
• git diff --check

[iondev33]

Thank you for this PR. The efficiency improvement potential is high in your PR.

In the course of our investigation, we discovered other flaws (not directly related to what your PR is addressing) so at this time we will focus on patching the problem first, and then return to your PR for further testing and analysis. In the meantime, we are adding your PR on our shorter list of to-do. Thanks!

[zebastian]

let me know @iondev33 if you need help:

• merging dcpagottos part on top of the already added cfdp_rls_extended implementation
• implementation of the other parts of bpcp remove unnecessary round trips for tree copying #84 not present in the PR or the codebase yet: checksum/mtime based optional sending of files in an rsync-y way