Skip to content

Implement litani create-barrier #106

@karkhaz

Description

@karkhaz

As a user, I sometimes want to run a post-processing step after a bunch of jobs have finished. I want the post-processing job to run regardless of the dependencies' return codes, but I don't want to mask the dependencies' return codes with --ok-returns or --outcome-table. If the dependencies fail they should still fail on the report, but I want to run a post-processing step regardless.

What is needed in this case is a dummy program that polls run.json, waiting for all the dependent jobs to terminate. The dummy program doesn't actually take those jobs as inputs, it just watches run.json. When all dependent jobs are marked as complete, the dummy job terminates. The post-processing job can then take the dummy job as an input.

Acceptance criterea

  • Implement a new command, litani create-barrier
  • This command has an --inputs switch to specify a list of jobs that the barrier should wait on. (Probably: list of output files, not list of jobs?)
  • The command then terminates and prints a UUID. This UUID is the "output file" that the post-processing job would then depend on.
  • If we need to wait on a huge number of files, we cannot create a subprocess with that many arguments due to POSIX limitations. So the command should also be able to accept a list of inputs on stdin if --read-stdin is passed as an alternative to --inputs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions