Skip to content

Fix module run crash with multi-output processes#6963

Closed
pditommaso wants to merge 1 commit into
masterfrom
fix/module-run-multi-output
Closed

Fix module run crash with multi-output processes#6963
pditommaso wants to merge 1 commit into
masterfrom
fix/module-run-multi-output

Conversation

@pditommaso

Copy link
Copy Markdown
Member

Summary

  • Fix GroovyCastException when running nextflow module run on processes with multiple output channels (e.g. nf-core/fastqc)
  • ProcessEntryHandler.printOutput() assumed all output channels were DataflowVariable (value channels), but processes with multiple outputs use DataflowBroadcast (queue channels)
  • Add readChannelValue() helper that detects the channel type via CH.isValue() and reads accordingly

Fixes #6958

Test plan

  • New test: readChannelValue correctly reads from DataflowVariable
  • New test: readChannelValue correctly reads from DataflowBroadcast
  • Existing tests pass
  • Manual: run nextflow module run nf-core/fastqc and verify no cast error

🤖 Generated with Claude Code

…es (#6958)

ProcessEntryHandler.printOutput() assumed all output channels were
DataflowVariable (value channels), but processes with multiple outputs
use DataflowBroadcast (queue channels). Add readChannelValue() helper
that handles both channel types via CH.isValue()/CH.getReadChannel().

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
@netlify

netlify Bot commented Mar 23, 2026

Copy link
Copy Markdown

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit ef35738
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/69c1b3a4ac442600089a7c7c

@pditommaso

Copy link
Copy Markdown
Member Author

Found a hang when running nextflow module run nf-core/salmon/quant with multiple inputs. The main thread is stuck waiting on a dataflow channel read in ProcessEntryHandler.printOutput:

"main" #1 prio=5 os_prio=31 cpu=974.16ms elapsed=92.20s
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park(java.base@21/Native Method)
	at java.util.concurrent.locks.LockSupport.park(java.base@21/LockSupport.java:371)
	at groovyx.gpars.dataflow.expression.DataflowExpression.getVal(DataflowExpression.java:261)
	at groovyx.gpars.dataflow.stream.StreamCore.getFirst(StreamCore.java:182)
	at groovyx.gpars.dataflow.stream.DataflowStreamReadAdapter.getVal(DataflowStreamReadAdapter.java:97)
	at nextflow.script.ProcessEntryHandler.readChannelValue(ProcessEntryHandler.groovy:159)
	at nextflow.script.ProcessEntryHandler.printOutput(ProcessEntryHandler.groovy:141)
	at nextflow.script.ProcessEntryHandler$_createProcessEntry_closure1$_closure6$_closure7.doCall(ProcessEntryHandler.groovy:95)
	at nextflow.script.ProcessEntryHandler$_createProcessEntry_closure1$_closure6$_closure7.call(ProcessEntryHandler.groovy)
	at nextflow.Session.callIgniters(Session.groovy:559)
	at nextflow.Session.fireDataflowNetwork(Session.groovy:551)
	at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:231)
	at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:128)
	at nextflow.cli.CmdRun.run(CmdRun.groovy:435)
	at nextflow.cli.module.CmdModuleRun.run(CmdModuleRun.groovy:85)
	at nextflow.cli.CmdModule.run(CmdModule.groovy:106)
	at nextflow.cli.Launcher.run(Launcher.groovy:524)
	at nextflow.cli.Launcher.main(Launcher.groovy:684)

Command used:

nextflow module run nf-core/salmon/quant \
    --meta.id test \
    --reads "reads_1.fastq.gz,reads_2.fastq.gz" \
    --index <salmon_index> \
    --gtf genes.gtf \
    --transcript_fasta transcriptome.fasta \
    --alignment_mode false \
    --lib_type A \
    --outdir results/salmon_quant

@pditommaso

Copy link
Copy Markdown
Member Author

I think this PR should be reverted. The proper fix should be in the channel construction logic underlying module run, not in DataflowBroadcast casting. The hang in ProcessEntryHandler.readChannelValue suggests the output channels are not being properly terminated/poisoned when the process completes, which points to a structural issue in how module run wires up the dataflow network for multi-output processes.

@jorgee jorgee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing some tests with this PR I sometimes get a PoisonPill object.
"versions_fastqc": "groovyx.gpars.dataflow.operator.PoisonPill@2c815fdc"
What should we print in this case null or an empty string?

@pditommaso pditommaso closed this Mar 24, 2026
@pditommaso

Copy link
Copy Markdown
Member Author

Indeed, the poison pill issue is another issue to be fixed. Opening a new PR for this

@jorgee

jorgee commented Mar 24, 2026

Copy link
Copy Markdown
Contributor

Also hangs when a container is not found or when conda env fails. I will have a look at this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nextflow module run fails with DataflowBroadcast cast error when process has multiple outputs

2 participants