Open
Description
We currently support (at least) two flags that control how workflows are scheduled:
--use_dynamic_scheduler
, which is abool
indicating whether the scheduler invokesDynamicScheduleDAG
orScheduleDAG
.--use_heuristic
, which is abool
indicating whether to use exhaustive search or the dynamic programming algorithm to find the system assignments (this manifests itself by callingSchedulerDynamic::ComputeOptimal
orSchedulerDynamic::ComputeHeuristic
).
IIRC, the difference between a dynamic and a non-dynamic scheduling call is whether Musketeer re-assesses its decisions after each job completion (@ICGog -- correct?).
The naming of things is a bit misleading here, for two reasons:
- The scheduler class is called
SchedulerDynamic
, but the--use_dynamic_scheduler
flag controls which method in this class gets called, rather than whether aSchedulerDynamic
or something else is used. - Since our heuristic scheduler is based on dynamic programming, it is easy to assume that
--use_dynamic_scheduler
refers to the heuristic, which it does not.
Proposal: let's rename the flags, so that they're more intuitive.
--use_dynamic_scheduler
becomes--continuously_reschedule
--use_heuristic
becomes a--scheduling_algorithm
flag that can take three values: "automatic" (use heuristic if >18 operators, optimal otherwise), "heuristic" and "optimal".
This came up while investigating bugs reported by @n1v0lg.