Fix issues with resuming async tasks awaiting a future (backport #1469) #1560
Conversation
…#1469) Signed-off-by: Błażej Sowa <[email protected]> Signed-off-by: Nadav Elkabets <[email protected]> Co-authored-by: Nadav Elkabets <[email protected]>
…atibility Signed-off-by: Błażej Sowa <[email protected]>
|
@nadavelkabets I modified the deque logic to preserve the LIFO order which is maintained for backward compatibility in Jazzy. Could you check it? |
| for _ in range(ready_tasks_count): | ||
| # We pop from the right to maintain LIFO order for backward compatibility | ||
| # From Kilted onwards, the order of task execution is FIFO | ||
| task = self._ready_tasks.pop() |
There was a problem hiding this comment.
I believe it's more complicated to mimic the old behavior precisely.
While iterating over ready tasks, new tasks may get added to the ready tasks list by either a user that created a new task or a finished task that triggered done callbacks.
In the existing implementation, these would only execute in the next _wait_for_ready_callbacks iteration (they iterate over a copy of the tasks list) while in our implementation they would execute right away (we iterate over the same list).
In addition, prior to our changes blocked tasks stayed in the tasks list while we remove and append them back. This also means that we change the execution order of resumed tasks.
The only perfect solution I can imagine is to add a creation_time timestamp to the TaskData class, and using a heapq for the ready_tasks list to keep it sorted from oldest task to newest task. Furthermore, we currently utilize done callbacks to execute resumed tasks. To maintain the same order, we would have to give the done callback the same timestamp as the original task.
I honestly find the idea of switching Jazzy to FIFO order more realistic and straightforward.
Is that unreasonable?
There was a problem hiding this comment.
If changing the order to FIFO is an option, I'm all for it.
|
We discussed this in our weekly project meeting. We've added your issue to the next Client Library Working Group Meeting on January 9th for discussion and you have been added to the agenda. |
3 participants
Backport of #1469 to Jazzy