Open
Description
Optimus supports figuring out dependencies automatically by parsing task assets. This logic of finding dependency is implemented per task behavior. Users can also choose to not depend on the task's inferred dependencies and pass them explicitly in job.yaml
specification. We need a way to ignore the task's automatically inferred dependencies explicitly.
We use the existing specification file to add a nonbreaking change as follows:
name: job1
dependencies:
- job: hello_world1
- job: hello_world3
- ignore_job: hello_world2
- ignore_job: hello_world4
In this case, if the task used in this job somehow infer hello_world2
as one of the upstream, we will choose to not treat it as an upstream dependency. Similarly, if infer logic did not find hello_world4
as one of the upstream, nothing happens and no error should be thrown.