Skip to content

Commit 483ece5

Browse files
committed
BF: job_templates: Call tar with --ignore-failed-read
After a command completes, it writes to "status.$subjob". If, after completing its command, a subjob sees that the status files for all the other subjobs are in, it claims responsibility for the post-processing step. For the datalad-run orchestrators, post-processing includes calling `find` to get a list of newly added files and then calling `tar` with these files as input. Given that the above procedure waits until each command exits, the hope is that all the output files are created and any temporary files will have been cleaned up. But we're hitting into cases [*] where apparently intermediate files are present for the `find` call but gone by the time `tar` is called. This leads to `tar` exiting with a non-zero status and the post-processing being aborted. Until someone has a better idea of how to deal with this, instruct `tar` to exit with zero even if an expected file isn't present. This allows post-processing to succeed and the incident will still show up in the captured stderr. [*] ReproNim#438 (comment)
1 parent eb6070c commit 483ece5

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

reproman/support/jobs/job_templates/runscript/includes/post-run.template.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,5 +29,6 @@ mkdir -p "$rootdir/outputs"
2929
tar \
3030
-C "$workdir" \
3131
--files-from "$metadir/togethome" \
32+
--ignore-failed-read \
3233
-cz \
3334
-f "$rootdir/outputs/$jobid.tar.gz"

0 commit comments

Comments
 (0)