Commit 3b56c37
committed
refactor: Restore parallelism and add backpressure to
In Obelisk 0.24 the join sets are closed on drop. Original version would
thus wait for child execution completion in every iteration of the `for` cycle.
This change introduces `join_set_batch` which collects join sets of the same
`list-stargazers` batch. Closing it on every `while` iteration adds backpressure.backfill_parallel
1 parent 000b315 commit 3b56c37
1 file changed
Lines changed: 9 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
93 | 93 | | |
94 | 94 | | |
95 | 95 | | |
| 96 | + | |
96 | 97 | | |
| 98 | + | |
97 | 99 | | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
106 | 105 | | |
107 | 106 | | |
| 107 | + | |
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
| 111 | + | |
111 | 112 | | |
112 | 113 | | |
113 | 114 | | |
| |||
0 commit comments