Skip to content

Clarify difference between scheduler flags #3

Open
@ms705

Description

@ms705

We currently support (at least) two flags that control how workflows are scheduled:

  1. --use_dynamic_scheduler, which is a bool indicating whether the scheduler invokes DynamicScheduleDAG or ScheduleDAG.
  2. --use_heuristic, which is a bool indicating whether to use exhaustive search or the dynamic programming algorithm to find the system assignments (this manifests itself by calling SchedulerDynamic::ComputeOptimal or SchedulerDynamic::ComputeHeuristic).

IIRC, the difference between a dynamic and a non-dynamic scheduling call is whether Musketeer re-assesses its decisions after each job completion (@ICGog -- correct?).

The naming of things is a bit misleading here, for two reasons:

  1. The scheduler class is called SchedulerDynamic, but the --use_dynamic_scheduler flag controls which method in this class gets called, rather than whether a SchedulerDynamic or something else is used.
  2. Since our heuristic scheduler is based on dynamic programming, it is easy to assume that --use_dynamic_scheduler refers to the heuristic, which it does not.

Proposal: let's rename the flags, so that they're more intuitive.

  • --use_dynamic_scheduler becomes --continuously_reschedule
  • --use_heuristic becomes a --scheduling_algorithm flag that can take three values: "automatic" (use heuristic if >18 operators, optimal otherwise), "heuristic" and "optimal".

This came up while investigating bugs reported by @n1v0lg.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions