Skip to content

Bump quant_tximport_summarizedexperiment and quantify_rsem for deterministic collect() ordering - #1885

Merged
pinin4fjords merged 4 commits into
nf-core:devfrom
pinin4fjords:fix/collect-ordering-determinism
Jul 16, 2026
Merged

Bump quant_tximport_summarizedexperiment and quantify_rsem for deterministic collect() ordering#1885
pinin4fjords merged 4 commits into
nf-core:devfrom
pinin4fjords:fix/collect-ordering-determinism

Conversation

@pinin4fjords

Copy link
Copy Markdown
Member

Description

Bumps quant_tximport_summarizedexperiment and quantify_rsem to pull in nf-core/modules#12377.

TXIMETA_TXIMPORT and CUSTOM_RSEMMERGECOUNTS each stage a multi-file input built by collecting per-sample process outputs on a queue channel. Queue channels emit in task-completion order, which is non-deterministic across runs, so the staged input list order - and therefore the task hash - changed run to run even when nothing else did, breaking -resume for these processes and everything downstream of them. Same pattern as the CUSTOM_TX2GENE fix already merged for this subworkflow.

Both scripts already derive sample identity from the staged file/directory names rather than input list position (TXIMETA_TXIMPORT's R template via list.files(), CUSTOM_RSEMMERGECOUNTS's script via a directory glob), so sorting the collected list by name is a no-op on outputs - it only makes the cache key reproducible.

Note the diff also carries forward the CUSTOM_TX2GENE determinism fix from #1875, since dev hasn't picked that up yet - both fixes live in the same subworkflow file.

Verification

Closes #1879

PR checklist

  • This comment contains a description of changes (with reason).
  • nf-core subworkflows lint passes for both bumped subworkflows.
  • CHANGELOG.md is updated.

…inistic collect() ordering

Pulls in nf-core/modules#12377: TXIMETA_TXIMPORT and CUSTOM_RSEMMERGECOUNTS
each stage a multi-file input built by collecting per-sample process
outputs on a queue channel, which emits in task-completion order. That
order (and therefore the task hash) changed run to run even when nothing
else did, breaking -resume for these processes and everything downstream.

Both are now sorted by staged file name before collection. Neither script
depends on input list order for correctness, so this only stabilizes the
cache key.
toSortedList() always emits (even []) on an empty upstream, unlike the
collect() it replaced which emits nothing at all. Without a guard, a
channel that previously never triggered its downstream process now
always does, running CUSTOM_RSEMMERGECOUNTS/TXIMETA_TXIMPORT once with
zero staged files whenever no samples reach this point (e.g. all
samples filtered out upstream). Filter to size() > 0 to restore the
original no-op-on-empty behaviour.

Applied here ahead of the modules.json bump; nf-core/modules#12378
carries the same fix upstream. Once that merges, only the pinned
git_sha needs updating.
…/modules#12378

Pulls in the merged empty-list guard, matching the content already
synced into these two subworkflow files.
@pinin4fjords
pinin4fjords merged commit b6343cb into nf-core:dev Jul 16, 2026
8 of 80 checks passed
@pinin4fjords

Copy link
Copy Markdown
Member Author

Admin-merged with required-checks bypass. 3 in-flight workflow(s) cancelled: Run nf-test, Run nf-test on ARM, Run GPU nf-tests.

Rationale: the unverified diff since the last fully-green commit (1889d16b5) was just the two git_sha bumps in modules.json pulling in nf-core/modules#12378. Every other standard workflow (Run nf-test, Run nf-test on ARM, Run GPU nf-tests, Nextflow strict syntax lint, nf-core template version comment) was already green at that commit - the only failure was nf-core linting's check_local_copy check, which is exactly what this bump fixes. modules.json isn't read by Nextflow at runtime, so it can't affect the nf-test/GPU suites already verified green. Locally re-ran nf-core subworkflows lint for both bumped subworkflows against the new pin (clean), and by the time the merge went through, the nf-core linting run itself had actually finished on its own and come back green too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant