Commit 790e1bd
committed
Extract task execution to TaskRunner
At its core a process is a function that accepts an input, executes a
task and returns an asynchronous result. TaskProcessor implements this
functionality together with the dataflow interface of the process, so
that a task can only be executed by feeding values through the process
input channels.
Move the execution of a task to a separate `TaskRunner` class:
- the resolution of the task inputs and outputs
- the store directory and cache lookup
- the delegation to the executor, including task arrays
- the task error strategy and retries
TaskProcessor keeps the dataflow interface -- the operator, the process
state, the output binding and the publishing -- as well as the process
scoped services shared by all task executions, such as the name, the
config, the executor and the task environment.
The processor creates a runner with `taskCompleted` as the completion
handler, so that the runner does not depend on the dataflow interface:
a task can be executed by invoking `TaskRunner.submit()` directly,
without the process being part of a dataflow network (see the test
'should execute a task without a dataflow network').
Note that `resumeOrDie` is now synchronized on the runner rather than on
the processor, so it no longer excludes `bindOutputs` and
`sendPoisonPill`. These methods do not share mutable state, the coarse
lock was incidental.
Assisted-by: Claude Opus 5 (1M context)
Signed-off-by: Ben Sherman <bentshermann@gmail.com>1 parent 2b2fce1 commit 790e1bd
9 files changed
Lines changed: 1361 additions & 1139 deletions
File tree
- docs/developer
- diagrams
- modules/nextflow/src
- main/groovy/nextflow/processor
- testFixtures/groovy/test
- test/groovy/nextflow/processor
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
16 | | - | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
17 | 22 | | |
18 | 23 | | |
19 | 24 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
24 | | - | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| |||
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
677 | 677 | | |
678 | 678 | | |
679 | 679 | | |
680 | | - | |
| 680 | + | |
681 | 681 | | |
682 | 682 | | |
683 | 683 | | |
| |||
743 | 743 | | |
744 | 744 | | |
745 | 745 | | |
746 | | - | |
| 746 | + | |
747 | 747 | | |
748 | 748 | | |
749 | 749 | | |
| |||
0 commit comments