Skip to content

Update parabricks minimap2 for scnanoseq #10788

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gburnett-nvidia wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Update parabricks minimap2 for scnanoseq #10788

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ed5cf65
feat: update minimap2 inputs and outputs for scnanoseq pipeline
gburnett-nvidia Mar 12, 2026
c5ceb35
fix: adding back compatible_versions to play nicely with other packag…
gburnett-nvidia Mar 12, 2026
213668f
chore: linting
gburnett-nvidia Mar 12, 2026
f7c508b
Merge branch 'master' into update_parabricks_minimap2_for_scnanoseq
gburnett-nvidia Mar 13, 2026
db98d56
Merge branch 'master' into update_parabricks_minimap2_for_scnanoseq
gburnett-nvidia Mar 13, 2026
4de0f17
Merge branch 'master' into update_parabricks_minimap2_for_scnanoseq
gburnett-nvidia Mar 13, 2026
b170df6
Merge branch 'master' into update_parabricks_minimap2_for_scnanoseq
gburnett-nvidia Mar 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 20 additions & 7 deletions modules/nf-core/parabricks/minimap2/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,14 @@ process PARABRICKS_MINIMAP2 {
input:
tuple val(meta), path(reads)
tuple val(meta2), path(fasta)
tuple val(meta3), path(intervals)
tuple val(meta4), path(known_sites)
tuple val(meta3), path(index)
tuple val(meta4), path(intervals)
tuple val(meta5), path(known_sites)
val output_fmt

output:
tuple val(meta), path("*.bam"), emit: bam, optional: true
tuple val(meta), path("*.bai"), emit: bai, optional: true
tuple val(meta), path("*.cram"), emit: cram, optional: true
tuple val(meta), path("*.crai"), emit: crai, optional: true
tuple val(meta), path("*.{bam,cram}"), emit: bam, optional: true
tuple val(meta), path("*.{bai,crai}"), emit: index, optional: true
Copy link
Contributor

@SPPearce SPPearce Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tuple val(meta), path("*.{bai,crai}"), emit: index, optional: true
tuple val(meta), path("*.{bai,csi,crai}"), emit: index, optional: true

tuple val(meta), path("*.table"), emit: bqsr_table, optional: true
tuple val(meta), path("*_qc_metrics"), emit: qc_metrics, optional: true
tuple val(meta), path("*.duplicate-metrics.txt"), emit: duplicate_metrics, optional: true
Expand Down Expand Up @@ -49,6 +48,7 @@ process PARABRICKS_MINIMAP2 {
}
def extension = "${output_fmt}"

def index_command = index ? "--index ${index}" : ""
def known_sites_command = known_sites ? (known_sites instanceof List ? known_sites.collect { knownSite -> "--knownSites ${knownSite}" }.join(' ') : "--knownSites ${known_sites}") : ""
def known_sites_output_cmd = known_sites ? "--out-recal-file ${prefix}.table" : ""
def intervals_command = intervals ? (intervals instanceof List ? intervals.collect { interval -> "--interval-file ${interval}" }.join(' ') : "--interval-file ${intervals}") : ""
Expand All @@ -60,11 +60,25 @@ process PARABRICKS_MINIMAP2 {
--ref ${fasta} \\
${in_command} \\
--out-bam ${prefix}.${extension} \\
${index_command} \\
${known_sites_command} \\
${known_sites_output_cmd} \\
${intervals_command} \\
${num_gpus} \\
${args}

# Capture the full version output once and store it in a variable
pbrun_version_output=\$(pbrun minimap2 --version 2>&1)
Copy link
Contributor

@SPPearce SPPearce Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Does the tool take a long time to give the version information?


# We handle this different to the other modules because minimap does not begin with an Uppercase letter

# Generate compatible_versions.yml
cat <<EOF > compatible_versions.yml
Comment on lines +73 to +76
Copy link
Contributor

@SPPearce SPPearce Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you swap to a topic channel please, which may make this unnecessary.

"${task.process}":
pbrun_version: \$(echo "\$pbrun_version_output" | grep "pbrun:" | awk '{print \$2}')
compatible_with:
\$(echo "\$pbrun_version_output" | tr '\\t' ' ' | awk -F':' '/Compatible With:/,/^---/ { if (\$0 !~ /Compatible With:/ && \$0 !~ /^---\$/ && index(\$0,":")>0) { key=\$1; val=\$2; gsub(/^[ ]+|[ ]+\$/, "", key); gsub(/^[ ]+|[ ]+\$/, "", val); printf " %s: %s\\n", key, val } }')
Comment on lines +79 to +80
Copy link
Contributor

@SPPearce SPPearce Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this really really long expression doing?

Copy link
Contributor

@famosab famosab Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This extracts the compatible version of the non-gpu tool so that you can directly see that in the snapshot. There is no reason to have this as a topic because this is specific for parabricks modules.

EOF
"""

stub:
Expand All @@ -85,7 +99,6 @@ process PARABRICKS_MINIMAP2 {
${known_sites_output}
${qc_metrics_output}
${duplicate_metrics_output}

# Capture the full version output once and store it in a variable
pbrun_version_output=\$(pbrun minimap2 --version 2>&1)

Expand Down
48 changes: 18 additions & 30 deletions modules/nf-core/parabricks/minimap2/meta.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,22 @@ input:
type: map
description: |
Groovy Map containing index information
- index:
type: file
description: (optional) minimap2 index file for the reference
pattern: "*.mmi"
ontologies: []
- - meta4:
type: map
description: |
Groovy Map containing intervals information
- intervals:
type: file
description: (optional) file(s) containing genomic intervals for use in base
quality score recalibration (BQSR)
pattern: "*.{bed,interval_list,picard,list,intervals}"
ontologies: []
- - meta4:
- - meta5:
type: map
description: |
Groovy Map containing known sites information
Expand All @@ -72,44 +81,22 @@ output:
description: |
Groovy Map containing sample information
e.g. [ id:'test', single_end:false ]
- "*.bam":
- "*.{bam,cram}":
type: file
description: Sorted BAM file
pattern: "*.bam"
description: Sorted BAM or CRAM file
pattern: "*.{bam,cram}"
ontologies:
- edam: "http://edamontology.org/format_2572" # BAM
bai:
- - meta:
type: map
description: |
Groovy Map containing sample information
e.g. [ id:'test', single_end:false ]
- "*.bai":
type: file
description: index corresponding to sorted BAM file
pattern: "*.bai"
ontologies: []
cram:
- - meta:
type: map
description: |
Groovy Map containing sample information
e.g. [ id:'test', single_end:false ]
- "*.cram":
type: file
description: Sorted CRAM file
pattern: "*.cram"
ontologies: []
crai:
index:
- - meta:
type: map
description: |
Groovy Map containing sample information
e.g. [ id:'test', single_end:false ]
- "*.crai":
- "*.{bai,crai}":
type: file
description: index corresponding to sorted CRAM file
pattern: "*.crai"
description: Index corresponding to sorted BAM or CRAM file
pattern: "*.{bai,crai}"
Copy link
Contributor

@SPPearce SPPearce Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pattern: "*.{bai,crai}"
pattern: "*.{bai,csi,crai}"

ontologies: []
Copy link
Contributor

@SPPearce SPPearce Mar 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should have some ontologies, as should several of the others.

bqsr_table:
- - meta:
Expand Down Expand Up @@ -178,3 +165,4 @@ authors:
- "@haidyi"
maintainers:
- "@haidyi"
- "@gburnett-nvidia"
80 changes: 47 additions & 33 deletions modules/nf-core/parabricks/minimap2/tests/main.nf.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ nextflow_process {
]
input[2] = [ [], [] ]
input[3] = [ [], [] ]
input[4] = 'bam'
input[4] = [ [], [] ]
input[5] = 'bam'
"""
}
}
Expand All @@ -41,7 +42,7 @@ nextflow_process {
{ assert process.success },
{ assert snapshot(
bam(process.out.bam[0][1]).getReadsMD5(),
file(process.out.bai[0][1]).name,
file(process.out.index[0][1]).name,
process.out.findAll { key, val -> key.startsWith('versions') }
).match() }
)
Expand All @@ -68,7 +69,8 @@ nextflow_process {
]
input[2] = [ [], [] ]
input[3] = [ [], [] ]
input[4] = 'bam'
input[4] = [ [], [] ]
input[5] = 'bam'
"""
}
}
Expand Down Expand Up @@ -101,12 +103,13 @@ nextflow_process {
[ id:'test' ],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
input[2] = [
input[2] = [ [], [] ]
input[3] = [
[:],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true)
]
input[3] = [ [], [] ]
input[4] = 'bam'
input[4] = [ [], [] ]
input[5] = 'bam'
"""
}
}
Expand All @@ -116,7 +119,7 @@ nextflow_process {
{ assert process.success },
{ assert snapshot(
bam(process.out.bam[0][1]).getReadsMD5(),
file(process.out.bai[0][1]).name,
file(process.out.index[0][1]).name,
).match() }
)
}
Expand All @@ -141,12 +144,13 @@ nextflow_process {
[ id:'test' ],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
input[2] = [
input[2] = [ [], [] ]
input[3] = [
[:],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true)
]
input[3] = [ [], [] ]
input[4] = 'bam'
input[4] = [ [], [] ]
input[5] = 'bam'
"""
}
}
Expand Down Expand Up @@ -178,11 +182,12 @@ nextflow_process {
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
input[2] = [ [], [] ]
input[3] = [
input[3] = [ [], [] ]
input[4] = [
[:],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/ngscheckmate/test1.vcf.gz', checkIfExists: true)
]
input[4] = 'bam'
input[5] = 'bam'
"""
}
}
Expand All @@ -192,7 +197,7 @@ nextflow_process {
{ assert process.success },
{ assert snapshot(
bam(process.out.bam[0][1]).getReadsMD5(),
file(process.out.bai[0][1]).name,
file(process.out.index[0][1]).name,
process.out.findAll { key, val -> key.startsWith('versions') }
).match() }
)
Expand All @@ -219,11 +224,12 @@ nextflow_process {
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
input[2] = [ [], [] ]
input[3] = [
input[3] = [ [], [] ]
input[4] = [
[:],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/ngscheckmate/test1.vcf.gz', checkIfExists: true)
]
input[4] = 'bam'
input[5] = 'bam'
"""
}
}
Expand Down Expand Up @@ -254,15 +260,16 @@ nextflow_process {
[ id:'test' ],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
input[2] = [
input[2] = [ [], [] ]
input[3] = [
[:],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true)
]
input[3] = [
input[4] = [
[:],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/ngscheckmate/test1.vcf.gz', checkIfExists: true)
]
input[4] = 'bam'
input[5] = 'bam'
"""
}
}
Expand All @@ -272,7 +279,7 @@ nextflow_process {
{ assert process.success },
{ assert snapshot(
bam(process.out.bam[0][1]).getReadsMD5(),
file(process.out.bai[0][1]).name,
file(process.out.index[0][1]).name,
process.out.findAll { key, val -> key.startsWith('versions') }
).match() }
)
Expand All @@ -298,15 +305,16 @@ nextflow_process {
[ id:'test' ],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/genome.fasta', checkIfExists: true)
]
input[2] = [
input[2] = [ [], [] ]
input[3] = [
[:],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/bed/test.bed', checkIfExists: true)
]
input[3] = [
input[4] = [
[:],
file(params.modules_testdata_base_path + 'genomics/sarscov2/genome/ngscheckmate/test1.vcf.gz', checkIfExists: true)
]
input[4] = 'bam'
input[5] = 'bam'
"""
}
}
Expand Down Expand Up @@ -339,7 +347,8 @@ nextflow_process {
]
input[2] = [ [], [] ]
input[3] = [ [], [] ]
input[4] = 'bam'
input[4] = [ [], [] ]
input[5] = 'bam'
"""
}
}
Expand All @@ -349,7 +358,7 @@ nextflow_process {
{ assert process.success },
{ assert snapshot(
bam(process.out.bam[0][1]).getReadsMD5(),
file(process.out.bai[0][1]).name,
file(process.out.index[0][1]).name,
process.out.findAll { key, val -> key.startsWith('versions') }
).match() }
)
Expand All @@ -376,7 +385,8 @@ nextflow_process {
]
input[2] = [ [], [] ]
input[3] = [ [], [] ]
input[4] = 'bam'
input[4] = [ [], [] ]
input[5] = 'bam'
"""
}
}
Expand Down Expand Up @@ -408,7 +418,8 @@ nextflow_process {
]
input[2] = [ [], [] ]
input[3] = [ [], [] ]
input[4] = 'cram'
input[4] = [ [], [] ]
input[5] = 'cram'
"""
}
}
Expand All @@ -419,10 +430,10 @@ nextflow_process {
{ assert process.success },
{ assert snapshot(
cram(
process.out.cram[0][1],
process.out.bam[0][1],
fasta,
).getReadsMD5(),
file(process.out.crai[0][1]).name,
file(process.out.index[0][1]).name,
process.out.findAll { key, val -> key.startsWith('versions') }
).match() }
)
Expand All @@ -449,7 +460,8 @@ nextflow_process {
]
input[2] = [ [], [] ]
input[3] = [ [], [] ]
input[4] = 'cram'
input[4] = [ [], [] ]
input[5] = 'cram'
"""
}
}
Expand Down Expand Up @@ -481,7 +493,8 @@ nextflow_process {
]
input[2] = [ [], [] ]
input[3] = [ [], [] ]
input[4] = 'cram'
input[4] = [ [], [] ]
input[5] = 'cram'
"""
}
}
Expand All @@ -492,10 +505,10 @@ nextflow_process {
{ assert process.success },
{ assert snapshot(
cram(
process.out.cram[0][1],
process.out.bam[0][1],
fasta,
).getReadsMD5(),
file(process.out.crai[0][1]).name,
file(process.out.index[0][1]).name,
process.out.findAll { key, val -> key.startsWith('versions') }
).match() }
)
Expand All @@ -522,7 +535,8 @@ nextflow_process {
]
input[2] = [ [], [] ]
input[3] = [ [], [] ]
input[4] = 'cram'
input[4] = [ [], [] ]
input[5] = 'cram'
"""
}
}
Expand Down
Loading
Loading