Unable to override runtime parameters due to use of WDL 1.0 instead of 1.1

[hexylena]

Describe the bug

I'm running on a cluster which requires setting a walltime, and otherwise defaults to 1 hour. This causes issues when running this pipeline as a number of steps take longer than an hour for us. Thus I need to add a time_minutes runtime parameter (available in miniwdl-slurm)

In WDL 1.1 the ability to override runtime parameters was added, which would enable me to set these via my inputs.json, preventing me from needing to modify the workflow itself.

Details

• Release Version: v2.1.3
• Backend: miniwdl-slurm on HPC
• Inputs: singleton sequenced to 30x on Revio
• Failing task(s)

Here is the current patch I'm using:
wdl.patch.txt

Expected behavior

It would be ideal if the tasks could use WDL 1.1 so I do not need to modify it by hand in order to execute. Example of the change in behaviour with 1.1.
Thanks for your consideration :)

[williamrowell]

Thanks for the suggestion. Unfortunately, Cromwell doesn't support v1.1, so we can't make this update. There is a workaround in miniwdl-slurm though. You can either add to your task runtime defaults:

[task_runtime]
# Setting a 'maxRetries' default allows jobs that fail due to intermittent
# errors on the cluster to be retried.
defaults = {
        "maxRetries": 2,
        "docker": "ubuntu:20.04",
        "time_minutes": 1440
    }

or to the slurm extra_args:

[slurm]
extra_args="--partition heavy_users,gpu --time 1440"

The first option is essentially the equivalent to modifying every task to include:

runtime {
  ...
  time_minutes: 1440
  ...
}

and the second is more like modifying every srun command directly.

The end result is fundamentally identical, but I'd recommend the task_runtime option.

[hexylena]

Ahh I'm sorry to hear that, but, completely understandable. (Please feel free to close this, not sure what y'all's issue policy is and whether you want to keep it around until cromwell someday supports it.)

Yes, that's definitely an option. I was hoping to avoid that since it'll make the rest of the jobs slower to schedule, but, that works indeed. I'll probably end up doing that, thanks for the suggestion :)