Delay in tasks that are cached with storeDir

[anoronh4]

Description of the bug

Certain tasks will delay even when the previous steps are cached in storeDir. After some digging it seems that nextflow spends time to stage remote reference files, even though the task is going to be skipped no matter what. this is a waste of time and disk space, and complicates our ability to run the pipeline in an offline environment. i submitted an issue to nextflow.

There are a few workarounds:

1. Download the reference files locally and change the reference configuration to point to local paths only.
2. Create two workflow modes -- one for reference only, that will download all remote references, and one for the sample processing, which will use a different reference configuration.

Command used and terminal output

No response

Relevant files

No response

System information

Nextflow version: 23.10.1
Hardware: HPC
Executor: LSF
Container engine: Singularity
Java version: 11
Operating system: Linux
Bash version: GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
Version of mskcc/forte: dev