Bump quant_tximport_summarizedexperiment and quantify_rsem for deterministic collect() ordering - #1885
Conversation
…inistic collect() ordering Pulls in nf-core/modules#12377: TXIMETA_TXIMPORT and CUSTOM_RSEMMERGECOUNTS each stage a multi-file input built by collecting per-sample process outputs on a queue channel, which emits in task-completion order. That order (and therefore the task hash) changed run to run even when nothing else did, breaking -resume for these processes and everything downstream. Both are now sorted by staged file name before collection. Neither script depends on input list order for correctness, so this only stabilizes the cache key.
toSortedList() always emits (even []) on an empty upstream, unlike the collect() it replaced which emits nothing at all. Without a guard, a channel that previously never triggered its downstream process now always does, running CUSTOM_RSEMMERGECOUNTS/TXIMETA_TXIMPORT once with zero staged files whenever no samples reach this point (e.g. all samples filtered out upstream). Filter to size() > 0 to restore the original no-op-on-empty behaviour. Applied here ahead of the modules.json bump; nf-core/modules#12378 carries the same fix upstream. Once that merges, only the pinned git_sha needs updating.
…/modules#12378 Pulls in the merged empty-list guard, matching the content already synced into these two subworkflow files.
|
Admin-merged with required-checks bypass. 3 in-flight workflow(s) cancelled: Rationale: the unverified diff since the last fully-green commit ( |
Description
Bumps
quant_tximport_summarizedexperimentandquantify_rsemto pull in nf-core/modules#12377.TXIMETA_TXIMPORTandCUSTOM_RSEMMERGECOUNTSeach stage a multi-file input built by collecting per-sample process outputs on a queue channel. Queue channels emit in task-completion order, which is non-deterministic across runs, so the staged input list order - and therefore the task hash - changed run to run even when nothing else did, breaking-resumefor these processes and everything downstream of them. Same pattern as theCUSTOM_TX2GENEfix already merged for this subworkflow.Both scripts already derive sample identity from the staged file/directory names rather than input list position (
TXIMETA_TXIMPORT's R template vialist.files(),CUSTOM_RSEMMERGECOUNTS's script via a directory glob), so sorting the collected list by name is a no-op on outputs - it only makes the cache key reproducible.Note the diff also carries forward the
CUSTOM_TX2GENEdeterminism fix from #1875, sincedevhasn't picked that up yet - both fixes live in the same subworkflow file.Verification
nf-core subworkflows lintclean for both subworkflows against the new pin.nf-core subworkflows test --profile dockerfor both subworkflows: all pass, snapshots byte-identical before/after (see Deterministic collect() ordering in tximport and RSEM merge counts modules#12377).Closes #1879
PR checklist
nf-core subworkflows lintpasses for both bumped subworkflows.