Skip to content

Update saltshaker reports#856

Open
ieduba wants to merge 33 commits into
devfrom
saltshaker_reports
Open

Update saltshaker reports#856
ieduba wants to merge 33 commits into
devfrom
saltshaker_reports

Conversation

@ieduba
Copy link
Copy Markdown
Contributor

@ieduba ieduba commented May 25, 2026

  • add customer ID to samples' saltshaker classification reports (previously they had no identifier at all, and customer ID is preferred by customers over sample ID)
  • display individual reports as tabs in the final html (previously were just printed one after the other)
  • remove find/concatenate module, previously used to concatenate sample reports

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/raredisease branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Ensure the test suite passes (nextflow run . -profile test_singleton,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 25, 2026

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 1f64587

+| ✅ 251 tests passed       |+
#| ❔   6 tests were ignored |#
!| ❗  10 tests had warnings |!
Details

❗ Test warnings:

  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
  • pipeline_todos - TODO string in CONTRIBUTING.md: Add any pipeline specific contribution guidelines here, such as coding styles, procedures, checklists etc.
  • pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ mq_metrics = SENTIEON_DATAMETRICS.out.mq_metrics.ifEmpty(null) // channel: [ val(meta), path(mq_metrics) ]
    _
  • pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ qd_metrics = SENTIEON_DATAMETRICS.out.qd_metrics.ifEmpty(null) // channel: [ val(meta), path(qd_metrics) ]
    _
  • pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ gc_metrics = SENTIEON_DATAMETRICS.out.gc_metrics.ifEmpty(null) // channel: [ val(meta), path(gc_metrics) ]
    _
  • pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ gc_summary = SENTIEON_DATAMETRICS.out.gc_summary.ifEmpty(null) // channel: [ val(meta), path(gc_summary) ]
    _
  • pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ aln_metrics = SENTIEON_DATAMETRICS.out.aln_metrics.ifEmpty(null) // channel: [ val(meta), path(aln_metrics) ]
    _
  • pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/raredisease/raredisease/subworkflows/local/align_sentieon/main.nf: _ is_metrics = SENTIEON_DATAMETRICS.out.is_metrics.ifEmpty(null) // channel: [ val(meta), path(is_metrics) ]
    _
  • schema_lint - Input mimetype is missing or empty
  • schema_description - No description provided in schema for parameter: hisat2

❔ Tests ignored:

  • files_exist - File is ignored: conf/modules.config
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/nf-core-raredisease_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-raredisease_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-raredisease_logo_dark.png
  • modules_config - modules_config

✅ Tests passed:

Run details

  • nf-core/tools version 4.0.2
  • Run at 2026-06-01 13:25:51

@ieduba ieduba marked this pull request as ready for review May 26, 2026 13:25
Copy link
Copy Markdown
Contributor

@fellen31 fellen31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! And I think good to have all the HTML generation in one process.

Since you are writing code that groups together one or more samples into one process, I would really like to encourage you to add a test beforehand (preferably in a separate PR) to CALL_SV_MT with multiple samples so you can be confident that your changes work not only with one sample. If not a test with real data, then at least a stub test.


input:
tuple val(meta), path(classify)
tuple val(meta), path(files_in), val(ids)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps?

Suggested change
tuple val(meta), path(files_in), val(ids)
tuple val(meta), path(saltshaker_classify_reports), val(sample_ids)

$args \
--input ${files_in.join(' --input ')} \
--sample ${ids.join(' --sample ')} \
--output ${prefix}.saltshaker_classify.html
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would always strive for rather than less flexibility on naming within the module. You could change the prefix to be ${meta.id}.saltshaker_classify in the config if you want.

Suggested change
--output ${prefix}.saltshaker_classify.html
--output ${prefix}.html


output:
tuple val(meta), path("*.html"), emit: classify_html
tuple val(meta), path("*.saltshaker_classify.html"), emit: classify_html
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://github.com/nf-core/raredisease/pull/856/changes#r3304124293

Suggested change
tuple val(meta), path("*.saltshaker_classify.html"), emit: classify_html
tuple val(meta), path("*.html"), emit: classify_html

ch_ploidy_model // channel: [optional; used by mandatory for GATK's cnvcaller][ path(ploidy_model) ]
ch_readcount_intervals // channel: [optional; used by mandatory for GATK's cnvcaller][ path(intervals) ]
ch_reads // channel: [mandatory] [ val(meta), [path(reads)] ]
ch_sample_id_map // channel: [optional] [val(id), val(id)]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What ids are expected in ch_sample_id_map? sample ids? case ids? customer ids?

input[19] = 'chrM'
input[20] = 15
input[21] = 0.01
input[10] = channel.empty()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you need to test ch_sample_id_map with real inputs? One test where you have customer ids and one where you don't have them would be good I think.

Comment thread subworkflows/local/call_sv_MT/main.nf Outdated
Comment on lines +133 to +143
.branch { sample_meta, caseid, txt, samplemap ->
id: samplemap.equals(null)
return [['id':caseid], txt, sample_meta.id]
custid: !(samplemap.equals(null))
return [['id':caseid], txt, samplemap]
}
.set { ch_for_mix }

FIND_CONCATENATE(
ch_saltshaker_txts
)
ch_saltshaker_txt = FIND_CONCATENATE.out.file_out
channel.empty()
.mix(ch_for_mix.id, ch_for_mix.custid)
.groupTuple()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is samplemap here? Is it customer_ids?

Suggested change
.branch { sample_meta, caseid, txt, samplemap ->
id: samplemap.equals(null)
return [['id':caseid], txt, sample_meta.id]
custid: !(samplemap.equals(null))
return [['id':caseid], txt, samplemap]
}
.set { ch_for_mix }
FIND_CONCATENATE(
ch_saltshaker_txts
)
ch_saltshaker_txt = FIND_CONCATENATE.out.file_out
channel.empty()
.mix(ch_for_mix.id, ch_for_mix.custid)
.groupTuple()
.map { sample_meta, case_id, txt, customer_id ->
def report_identifier = customer_id ?: sample_meta.id
return [['id': case_id], txt, report_identifier]
}
.groupTuple()

Comment thread subworkflows/local/call_sv_MT/main.nf Outdated
Comment on lines +144 to +148
.map { meta, paths, ids ->
def sorted_paths = paths.sort { a, b -> a.getName() <=> b.getName() }
def sorted_ids = ids.sort()
return [meta, sorted_paths, sorted_ids]
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really need to do this? Will they not be sorted automatically with groupTuple?

Comment thread subworkflows/local/call_sv_MT/main.nf Outdated
def sorted_ids = ids.sort()
return [meta, sorted_paths, sorted_ids]
}
.dump('tag': 'saltshaker_html_input')
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.dump('tag': 'saltshaker_html_input')

Comment thread bin/saltshaker_to_html.py
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will leave this for @peterpru :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants