The purpose of this issue is to ensure that the EC2 executor
- returns any stdout and stderr printed by tasks using the mechanism defined in #1380. To do this, the
run() implementation needs to
- retrieves the
stdout and stderr from the executor backend -- in the case of AWS Executors, by parsing Cloudwatch logs (see how Braket does this)
- printing those strings to
self.task_stdout and self.task_stderr.
- distinguish task runtime exceptions -- those raised in the executor backend -- from other exceptions originating from the interaction of Covalent and the executor backend. This is explained in more detail in #1390. When an runtime exception occurs, the
run() implementation should:
- Retrieve whatever
stdout and stderr messages have already been printed by the task
- Ensure that the exception traceback is appended to the task's
stderr.
- Print
stdout and stderr to self.task_stdout and self.task_stderr, respectively.
- Raise a
TaskRuntimeException.
For examples, see how the dask executor now deals with task runtime exceptions.
Note: These changes should be implemented in a backward-compatible manner -- so that the new AWSExecutors should work with Covalent 0.202.0post1 (the current stable AWSExecutors work with the latest develop).
Acceptance criteria:
The purpose of this issue is to ensure that the EC2 executor
run()implementation needs tostdoutandstderrfrom the executor backend -- in the case of AWS Executors, by parsing Cloudwatch logs (see how Braket does this)self.task_stdoutandself.task_stderr.run()implementation should:stdoutandstderrmessages have already been printed by the taskstderr.stdoutandstderrtoself.task_stdoutandself.task_stderr, respectively.TaskRuntimeException.For examples, see how the dask executor now deals with task runtime exceptions.
Note: These changes should be implemented in a backward-compatible manner -- so that the new AWSExecutors should work with Covalent 0.202.0post1 (the current stable AWSExecutors work with the latest develop).
Acceptance criteria:
self.task_stdoutandself.task_stderrrespectively, whereselfis the executor plugin instance.run()method raises aTaskRuntimeError.