Skip to content

Running large jobs during training/workshops #5

Description

@natefoo

Right now, jobs during trainings can go to LS6 (specifically, the vm-small partition) where:

  1. only 4 jobs can run at a time
  2. only 16 jobs can be queued total before additional submissions are rejected with error
  3. there may be considerable queue wait.

On usegalaxy.org for trainings we use TIaaS and drastically decrease memory/core resources so that tools run on normal nodes, but this isn't always possible for some tools, like Kraken2 which load the entire selected database into memory.

We could use reservations for this on LS6, but:

  1. I don't know if the reservations would be whole nodes or consumable resources
  2. I don't know if slurm-drmaa supports --reservation (but could be added if it doesn't)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions