What happens
--save_bam defaults to false, but every run still publishes a sorted BAM and its index to results/samtools/.
On the full test that is ~700 MB per sample across four samples, i.e. several GB of alignments and indexes in the published output of a run whose results are linked from the documentation.
Why
There are two BAM publish paths, and only one of them is gated.
BBMAP_ALIGN (conf/modules.config#L129-L144) publishes its unsorted *.bam with enabled: params.save_bam, so with the default nothing lands in bbmap/bbmap/.
The .*SAMTOOLS.* block (conf/modules.config#L150-L171) then publishes *.bam and *.bam.bai with no enabled: guard at all:
withName: ".*SAMTOOLS.*" {
publishDir = [
[
pattern: "*.bam",
path: { "${params.outdir}/samtools/" },
mode: params.publish_dir_mode
],
[
pattern: "*.bam.bai",
path: { "${params.outdir}/samtools/" },
mode: params.publish_dir_mode
],
[
pattern: "*.{flagstat,idxstats}",
...
enabled: params.save_samtools,
]
]
}
Only the {flagstat,idxstats} rule in that block is gated, on params.save_samtools (default true).
So suppressing the unsorted BBMap BAM just means a sorted copy of the same alignments gets published instead, which is not what the param's own description ("Save the bam files from mapping") says.
This is long-standing rather than a recent regression — git log -L 150,171:conf/modules.config puts the block at ecd5f42 ("Merge 3.2.1 template plus some more").
Documentation gap
docs/output.md#L242 documents bbmap/bbmap/*.bam as being written only "if enabled with --save_bam", which is the misleading half of the story.
There is no samtools/ section in docs/output.md at all, so the directory that actually receives the BAMs and BAIs is undocumented.
Suggested fix
- Add
enabled: params.save_bam to the *.bam and *.bam.bai rules in the .*SAMTOOLS.* block.
- Add a
samtools/ section to docs/output.md covering the BAM/BAI (gated on --save_bam) and the flagstat/idxstats files (gated on --save_samtools).
Leave save_samtools alone; it governs small text files and defaulting it to on is fine.
Worth deciding while in there: SAMTOOLS_STATS emits *.stats, which the *.{flagstat,idxstats} pattern does not match, so those are never published even with --save_samtools.
MultiQC still picks them up from the work directory, so this may well be deliberate — but it is inconsistent with the param's description.
Note on cost
This changes published output for every profile, so all twelve pipeline snapshots need regenerating, two of which (diamond and kofamscan) need a machine with the databases.
Worth keeping out of any branch that is mid-testing.
Drafted with help from Claude Code.
What happens
--save_bamdefaults tofalse, but every run still publishes a sorted BAM and its index toresults/samtools/.On the full test that is ~700 MB per sample across four samples, i.e. several GB of alignments and indexes in the published output of a run whose results are linked from the documentation.
Why
There are two BAM publish paths, and only one of them is gated.
BBMAP_ALIGN(conf/modules.config#L129-L144) publishes its unsorted*.bamwithenabled: params.save_bam, so with the default nothing lands inbbmap/bbmap/.The
.*SAMTOOLS.*block (conf/modules.config#L150-L171) then publishes*.bamand*.bam.baiwith noenabled:guard at all:Only the
{flagstat,idxstats}rule in that block is gated, onparams.save_samtools(defaulttrue).So suppressing the unsorted BBMap BAM just means a sorted copy of the same alignments gets published instead, which is not what the param's own description ("Save the bam files from mapping") says.
This is long-standing rather than a recent regression —
git log -L 150,171:conf/modules.configputs the block at ecd5f42 ("Merge 3.2.1 template plus some more").Documentation gap
docs/output.md#L242documentsbbmap/bbmap/*.bamas being written only "if enabled with--save_bam", which is the misleading half of the story.There is no
samtools/section indocs/output.mdat all, so the directory that actually receives the BAMs and BAIs is undocumented.Suggested fix
enabled: params.save_bamto the*.bamand*.bam.bairules in the.*SAMTOOLS.*block.samtools/section todocs/output.mdcovering the BAM/BAI (gated on--save_bam) and the flagstat/idxstats files (gated on--save_samtools).Leave
save_samtoolsalone; it governs small text files and defaulting it to on is fine.Worth deciding while in there:
SAMTOOLS_STATSemits*.stats, which the*.{flagstat,idxstats}pattern does not match, so those are never published even with--save_samtools.MultiQC still picks them up from the work directory, so this may well be deliberate — but it is inconsistent with the param's description.
Note on cost
This changes published output for every profile, so all twelve pipeline snapshots need regenerating, two of which (
diamondandkofamscan) need a machine with the databases.Worth keeping out of any branch that is mid-testing.
Drafted with help from Claude Code.