Migrate file publishing from publishDir to centralized output {} block - I by ramprasadn · Pull Request #784 · nf-core/raredisease

ramprasadn · 2026-03-12T21:19:43Z

PR checklist

Replaces per-process publishDir config directives with a centralized publishing pattern. Each subworkflow now builds a ch_publish channel emitting [destination, value] tuples that bubble up to main.nf, where a single publish: + output {} block handles all file publishing.

The following published files are affected by this change (all verified to be present in the same location after the change)

results_subwfs/call_mobile_elements/justhusky_mobile_elements.vcf.gz
results_subwfs/call_mobile_elements/justhusky_mobile_elements.vcf.gz.tbi

results_subwfs/call_snv/genome/justhusky_snv.vcf.gz
results_subwfs/call_snv/genome/justhusky_snv.vcf.gz.tbi

results_subwfs/call_snv/mitochondria/justhusky_mitochondria.vcf.gz
results_subwfs/call_snv/mitochondria/justhusky_mitochondria.vcf.gz.tbi

results_subwfs/call_sv/genome/justhusky_sv.vcf.gz
results_subwfs/call_sv/genome/justhusky_sv.vcf.gz.tbi

results_subwfs/call_sv/mitochondria/earlycasualcaiman_LNUMBER1.breakpoint
results_subwfs/call_sv/mitochondria/earlycasualcaiman_mitochondria_deletions.txt
results_subwfs/call_sv/mitochondria/earlycasualcaiman_LNUMBER1.cluster

results_subwfs/call_sv/mitochondria/slowlycivilbuck_LNUMBER2.breakpoint
results_subwfs/call_sv/mitochondria/slowlycivilbuck_mitochondria_deletions.txt
results_subwfs/call_sv/mitochondria/hugelymodelbat_LNUMBER3.cluster

results_subwfs/call_sv/mitochondria/hugelymodelbat_LNUMBER3.breakpoint
results_subwfs/call_sv/mitochondria/hugelymodelbat_mitochondria_deletions.txt
results_subwfs/call_sv/mitochondria/slowlycivilbuck_LNUMBER2.cluster

results/alignment/earlycasualcaiman_mt_subsample.bam
results/alignment/earlycasualcaiman_mt_subsample.bam.bai
results/alignment/earlycasualcaiman_sorted_md.metrics.txt
results/alignment/earlycasualcaiman_sorted_md.bam
results/alignment/earlycasualcaiman_sorted_md.bam.bai

results/alignment/hugelymodelbat_mt_subsample.bam
results/alignment/hugelymodelbat_mt_subsample.bam.bai
results/alignment/hugelymodelbat_sorted_md.metrics.txt
results/alignment/hugelymodelbat_sorted_md.bam
results/alignment/hugelymodelbat_sorted_md.bam.bai

results/alignment/slowlycivilbuck_mt_subsample.bam
results/alignment/slowlycivilbuck_mt_subsample.bam.bai
results/alignment/slowlycivilbuck_sorted_md.metrics.txt
results/alignment/slowlycivilbuck_sorted_md.bam
results/alignment/slowlycivilbuck_sorted_md.bam.bai

And 159 files in qc_bam folder 😅

nf-core-bot · 2026-03-12T21:20:15Z

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.5.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

github-actions · 2026-03-12T21:23:20Z

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 57fa0af

+| ✅ 234 tests passed       |+
#| ❔   7 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗   8 tests had warnings |!

Details

❗ Test warnings:

pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ mq_metrics = SENTIEON_DATAMETRICS.out.mq_metrics.ifEmpty(null) // channel: [ val(meta), path(mq_metrics) ]
_
pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ qd_metrics = SENTIEON_DATAMETRICS.out.qd_metrics.ifEmpty(null) // channel: [ val(meta), path(qd_metrics) ]
_
pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ gc_metrics = SENTIEON_DATAMETRICS.out.gc_metrics.ifEmpty(null) // channel: [ val(meta), path(gc_metrics) ]
_
pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ gc_summary = SENTIEON_DATAMETRICS.out.gc_summary.ifEmpty(null) // channel: [ val(meta), path(gc_summary) ]
_
pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ aln_metrics = SENTIEON_DATAMETRICS.out.aln_metrics.ifEmpty(null) // channel: [ val(meta), path(aln_metrics) ]
_
pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ is_metrics = SENTIEON_DATAMETRICS.out.is_metrics.ifEmpty(null) // channel: [ val(meta), path(is_metrics) ]
_
schema_lint - Input mimetype is missing or empty
schema_description - No description provided in schema for parameter: hisat2

❔ Tests ignored:

files_exist - File is ignored: conf/modules.config
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: assets/nf-core-raredisease_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-raredisease_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-raredisease_logo_dark.png
modules_config - modules_config

❔ Tests fixed:

rocrate_readme_sync - Mismatch fixed: RO-Crate description updated from README.md.

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/nf-test.yml
files_exist - File found: .github/actions/get-shards/action.yml
files_exist - File found: .github/actions/nf-test/action.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-raredisease_logo_light.png
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-raredisease_logo_light.png
files_exist - File found: docs/images/nf-core-raredisease_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: nf-test.config
files_exist - File found: tests/default.nf.test
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: conf/igenomes_ignored.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File found: ro-crate-metadata.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-raredisease_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowRaredisease.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-schema plugin
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config variable (correctly) not found: params.max_cpus
nextflow_config - Config variable (correctly) not found: params.max_memory
nextflow_config - Config variable (correctly) not found: params.max_time
nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
nextflow_config - Config variable (correctly) not found: params.validationLenientMode
nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
nextflow_config - Config variable (correctly) not found: validation.failUnrecognisedParams
nextflow_config - Config variable (correctly) not found: validation.failUnrecognisedHeaders
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 2.7.0dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.bait_padding= 100
nextflow_config - Config default value correct: params.genome= GRCh38
nextflow_config - Config default value correct: params.igenomes_base= s3://ngi-igenomes/igenomes/
nextflow_config - Config default value correct: params.mito_name= chrM
nextflow_config - Config default value correct: params.analysis_type= wgs
nextflow_config - Config default value correct: params.homoplasmy_af_threshold= 1.0
nextflow_config - Config default value correct: params.platform= illumina
nextflow_config - Config default value correct: params.ngsbits_samplegender_method= xy
nextflow_config - Config default value correct: params.scatter_count= 20
nextflow_config - Config default value correct: params.aligner= bwamem2
nextflow_config - Config default value correct: params.mt_aligner= bwamem2
nextflow_config - Config default value correct: params.samtools_sort_threads= 4
nextflow_config - Config default value correct: params.mbuffer_mem= 3072
nextflow_config - Config default value correct: params.min_trimmed_length= 40
nextflow_config - Config default value correct: params.mt_subsample_approach= reads
nextflow_config - Config default value correct: params.mt_subsample_reads= 18000
nextflow_config - Config default value correct: params.mt_subsample_rd= 150
nextflow_config - Config default value correct: params.mt_subsample_seed= 30
nextflow_config - Config default value correct: params.cnvnator_binsize= 1000
nextflow_config - Config default value correct: params.sentieon_dnascope_pcr_indel_model= CONSERVATIVE
nextflow_config - Config default value correct: params.variant_caller= deepvariant
nextflow_config - Config default value correct: params.variant_type= snp,indel
nextflow_config - Config default value correct: params.vep_cache_version= 112
nextflow_config - Config default value correct: params.breakspan= 15
nextflow_config - Config default value correct: params.breakthreshold= 2
nextflow_config - Config default value correct: params.cluster_threshold= 2
nextflow_config - Config default value correct: params.deletion_threshold_max= 30000
nextflow_config - Config default value correct: params.deletion_threshold_min= 30
nextflow_config - Config default value correct: params.evalue_threshold= 1e-05
nextflow_config - Config default value correct: params.exclude= 5
nextflow_config - Config default value correct: params.flank= 15
nextflow_config - Config default value correct: params.hplimit= 0.01
nextflow_config - Config default value correct: params.mitosalt_depth= 10000000
nextflow_config - Config default value correct: params.paired_distance= 1000
nextflow_config - Config default value correct: params.score_threshold= 80
nextflow_config - Config default value correct: params.sizelimit= 10000
nextflow_config - Config default value correct: params.split_distance_threshold= 5
nextflow_config - Config default value correct: params.split_length= 15
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
nf_test_content - 'tests/default.nf.test' contains outdir parameter
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/test_singleton.nf.test' contains outdir parameter
nf_test_content - 'tests/test_singleton.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/test_singleton.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/test_bam.nf.test' contains outdir parameter
nf_test_content - 'tests/test_bam.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/test_bam.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/nextflow.config' contains pipelines_testdata_base_path
nf_test_content - 'tests/nextflow.config' contains modules_testdata_base_path
nf_test_content - 'nf-test.config' sets a testsDir
nf_test_content - 'nf-test.config' sets a workDir
nf_test_content - 'nf-test.config' sets a configFile
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_nf_test - '.github/workflows/nf-test.yml' is triggered on expected events
actions_nf_test - '.github/workflows/nf-test.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 25.10.4, Config: 25.10.4
readme - README nf-core template version badge found.
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix_linting.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: template-version-comment.yml
actions_schema_validation - Workflow validation passed: nf-test.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
local_component_structure - local subworkflows directory structure is correct 'subworkflows/local/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
base_config - NFCORE_RAREDISEASE found in conf/base.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.5.1
rocrate_readme_sync - RO-Crate description matches the README.md.

Run details

nf-core/tools version 3.5.1
Run at 2026-03-16 18:06:46

…putblock

maxulysse

Looking good to me, only changes in the snapshots are in the local subworkflows

ramprasadn added 2 commits March 12, 2026 21:12

migrate to output for align*, qc_bam, and call*

d3431c1

updaate changelog

cbfbb34

ramprasadn added 18 commits March 13, 2026 10:09

update tests and snaps

9178577

update snapshot

aa5f876

update snapshots

5788ace

Merge branch 'outputblock' of github.com:nf-core/raredisease into out…

5f9ce3c

…putblock

update sentieon tests

88ad048

add sort [skip ci]

53557f3

update sentieon snapshots [skip ci]

221eee3

Update other snaps

78bfd88

update nxf version

736592d

Merge branch 'outputblock' of github.com:nf-core/raredisease into out…

7313f09

…putblock

update snap for postprocess MT [skip ci]

746ba25

update sentieon snapshot

0b74fb3

Update ro-crate-metadata.json

ef5ec40

Update README.md

21dc695

Update nf-test.yml

4a3734c

Merge branch 'outputblock' of github.com:nf-core/raredisease into out…

2535e92

…putblock

update sentieon snapshot

6c7ba9b

Merge branch 'outputblock' of github.com:nf-core/raredisease into out…

e456257

…putblock

ramprasadn marked this pull request as ready for review March 13, 2026 23:32

maxulysse reviewed Mar 16, 2026

View reviewed changes

Comment thread nextflow.config

ramprasadn added 5 commits March 16, 2026 14:42

Migrate strange and subsampling workflows to output and update snapshot

1de736c

Update nextflow.config

2659d97

Merge branch 'outputblock' of github.com:nf-core/raredisease into out…

5212f79

…putblock

update snapshot

6b5625a

Merge branch 'dev' into outputblock

57fa0af

ramprasadn requested a review from maxulysse March 16, 2026 19:29

maxulysse approved these changes Mar 17, 2026

View reviewed changes

ramprasadn merged commit 875a6f0 into dev Mar 17, 2026
125 of 131 checks passed

ramprasadn deleted the outputblock branch March 30, 2026 18:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate file publishing from publishDir to centralized output {} block - I #784

Migrate file publishing from publishDir to centralized output {} block - I #784
ramprasadn merged 25 commits into
devfrom
outputblock

ramprasadn commented Mar 12, 2026

Uh oh!

nf-core-bot commented Mar 12, 2026

Uh oh!

github-actions Bot commented Mar 12, 2026 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

❔ Tests fixed:

✅ Tests passed:

Run details

Uh oh!

Uh oh!

maxulysse left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ramprasadn commented Mar 12, 2026

PR checklist

Uh oh!

nf-core-bot commented Mar 12, 2026

Uh oh!

github-actions Bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

nf-core pipelines lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

❔ Tests fixed:

✅ Tests passed:

Run details

Uh oh!

Uh oh!

maxulysse left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Mar 12, 2026 •

edited

Loading

`nf-core pipelines lint` overall result: Passed ✅ ⚠️