Skip to content

Implement a controlled task scheduler #9

Open
@pior

Description

@pior

To run a high number of concurrent Lambda tasks we need a way to:

  1. limit concurrent running task
  2. limit max task scheduling per decision task

Naive implementation of 1.

def schedule_tasks(self, tasks, max_concurrent=1):
    running_tasks = 0
    for task in tasks:
        if running_task >= max_concurrent:
            self.context.decision_done()
        if not task.done:
            task.schedule()
            running_task += 1

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions