Skip to content

Commit df95006

Browse files
fxwiegandFelixMoelderjohanneskoester
authored
feat!: Pin wrapper versions in meta-wrappers; add alignoth_report meta-wrapper (#4678)
<!-- Ensure that the PR title follows conventional commit style (<type>: <description>)--> <!-- Possible types are here: https://github.com/commitizen/conventional-commit-types/blob/master/index.json --> <!-- Add a description of your PR here--> ### QC <!-- Make sure that you can tick the boxes below. --> * [ ] I confirm that I have followed the [documentation for contributing to `snakemake-wrappers`](https://snakemake-wrappers.readthedocs.io/en/stable/contributing.html). While the contributions guidelines are more extensive, please particularly ensure that: * [ ] `test.py` was updated to call any added or updated example rules in a `Snakefile` * [ ] `input:` and `output:` file paths in the rules can be chosen arbitrarily * [ ] wherever possible, command line arguments are inferred and set automatically (e.g. based on file extensions in `input:` or `output:`) * [ ] temporary files are either written to a unique hidden folder in the working directory, or (better) stored where the Python function `tempfile.gettempdir()` points to * [ ] the `meta.yaml` contains a link to the documentation of the respective tool or command under `url:` * [ ] conda environments use a minimal amount of channels and packages, in recommended ordering <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an alignoth_report workflow to export variants, generate per-sample alignment/coverage plots, and render an interactive datavzrd HTML report. * **Improvements** * Pinned many tool wrappers to explicit versions for more reproducible runs. * Added a reusable datavzrd visualization template supporting per-index plots and dynamic plot datasets. * **Tests** * Added tests and test resources for the new workflow; updated the test harness to support meta-wrappers. * **Chores** * Narrowed QA lint scope and added a CI note for future automation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Felix Mölder <[email protected]> Co-authored-by: Johannes Koester <[email protected]>
1 parent cc3bf0e commit df95006

File tree

32 files changed

+253
-75
lines changed

32 files changed

+253
-75
lines changed

‎.github/workflows/autobump.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,4 +33,5 @@ jobs:
3333
with:
3434
subcommand: update-conda-envs
3535
args: "*/${{ matrix.prefix }}*/environment.yaml */${{ matrix.prefix }}*/*/environment.yaml --create-prs --warn-on-error --entity-regex '(?P<entity>.+)/environment.yaml' --pr-add-label --pin-envs"
36-
36+
37+
# TODO add snakedeploy update-wrappers command applied to all meta-wrappers once it has the --create-prs option

‎.github/workflows/qc.yml‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -63,11 +63,23 @@ jobs:
6363
run: |
6464
declare -i ERRORS=0
6565
# get all modified and added files, not those that are deleted
66-
for f in $(git diff origin/master --name-only --diff-filter=d | grep Snakefile)
66+
for f in $(git diff origin/master --name-only --diff-filter=d | grep -P "(test/Snakefile|meta_wrapper.smk)")
6767
do
68-
echo "Linting $f"
69-
(cd $(dirname $f) && snakemake --lint)
70-
ERRORS+=$?
68+
# lint meta_wrapper.smk and test/Snakefile of normal wrappers
69+
if [[ $f == meta/**/meta_wrapper.smk ]]
70+
then
71+
echo "Linting $f"
72+
# ignore errors in meta_wrapper.smk files for now
73+
# TODO We have to rather lint test/Snakefile because it defines the right pathvars
74+
# or we disable the pathvar checking when linting
75+
(cd $(dirname $f) && (snakemake --lint --snakefile $(basename $f)) || true)
76+
ERRORS+=$?
77+
elif [[ $f != meta/**/test/Snakefile ]]
78+
then
79+
echo "Linting $f"
80+
(cd $(dirname $f) && snakemake --lint --snakefile $(basename $f))
81+
ERRORS+=$?
82+
fi
7183
done
7284
if (( $ERRORS > 0 ))
7385
then

‎meta/bio/alignoth_report/meta.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
name: alignoth_report
2+
description: Export variants via vembrane and create alignment plots with alignoth for a Datavzrd report.
3+
authors:
4+
- Felix Wiegand
5+
- Felix Mölder

‎meta/bio/alignoth_report/meta_wrapper.smk‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
def get_alignoth_tables(wildcards, results_dir):
2+
count = count_variants(wildcards)
3+
return [f"{results_dir}/alignoth/{{sample}}/{i}/" for i in range(count)]
4+
5+
6+
def count_variants(wildcards):
7+
return sum(1 for _ in open(checkpoints.vembrane_table.get(sample=wildcards.sample).output[0], "r")) - 1
8+
9+
10+
checkpoint vembrane_table:
11+
input:
12+
"<results>/{sample}.bcf",
13+
output:
14+
"<results>/tables/{sample}.tsv"
15+
log:
16+
"<logs>/vembrane_table/{sample}.log"
17+
params:
18+
expression="INDEX, CHROM, POS, REF, ALT",
19+
extra=""
20+
wrapper:
21+
"v7.6.1/bio/vembrane/table"
22+
23+
24+
rule alignoth:
25+
input:
26+
bam="<results>/mapped/{sample}.bam",
27+
bam_idx="<results>/mapped/{sample}.bam.bai",
28+
reference="resources/genome.fa",
29+
reference_idx="resources/genome.fa.fai",
30+
vcf="<results>/{sample}.bcf",
31+
vcf_idx="<results>/{sample}.bcf.csi",
32+
overview="<results>/tables/{sample}.tsv"
33+
output:
34+
directory("<results>/alignoth/{sample}/{index}/")
35+
params:
36+
extra=lambda wc, input: f"--around-vcf-record {wc.index} -f tsv"
37+
log:
38+
"<logs>/alignoth/{sample}_{index}.log"
39+
wrapper:
40+
"v7.9.1/bio/alignoth"
41+
42+
43+
rule datavzrd:
44+
input:
45+
config=workflow.source_path("resources/template.datavzrd.yaml"),
46+
overview="<results>/tables/{sample}.tsv",
47+
plot_tables=lambda wc: get_alignoth_tables(wc, "<results>")
48+
output:
49+
report(
50+
directory("<results>/datavzrd-report/{sample}"),
51+
htmlindex="index.html",
52+
),
53+
# config = "resources/datavzrd/{sample}.rendered_config.yaml"
54+
params:
55+
max_index=lambda wc: count_variants(wc)
56+
log:
57+
"<logs>/datavzrd/{sample}.log",
58+
wrapper:
59+
"v7.9.1/utils/datavzrd"

‎meta/bio/alignoth_report/resources/template.datavzrd.yaml‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
__use_yte__: true
2+
3+
default-view: variants
4+
5+
datasets:
6+
variants:
7+
path: ?input.overview
8+
separator: "\t"
9+
links:
10+
alignment plot:
11+
column: INDEX
12+
view: "{value}"
13+
?for i in range(params.max_index):
14+
?f"{i}.coverage":
15+
path: ?f"{input.plot_tables[i]}/{wildcards.sample}.coverage.tsv"
16+
separator: "\t"
17+
?f"{i}.reference":
18+
path: ?f"{input.plot_tables[i]}/{wildcards.sample}.reference.tsv"
19+
separator: "\t"
20+
?f"{i}.reads":
21+
path: ?f"{input.plot_tables[i]}/{wildcards.sample}.reads.tsv"
22+
separator: "\t"
23+
?f"{i}.highlight":
24+
path: ?f"{input.plot_tables[i]}/{wildcards.sample}.highlight.tsv"
25+
separator: "\t"
26+
views:
27+
variants:
28+
dataset: variants
29+
render-table:
30+
columns:
31+
SAMPLE:
32+
label: sample
33+
CHROM:
34+
label: chromosome
35+
POS:
36+
label: position
37+
INDEX:
38+
display-mode: hidden
39+
REF:
40+
label: reference allele
41+
spell:
42+
url: v1.5.0/med/alleles
43+
ALT:
44+
label: alternative allele
45+
spell:
46+
url: v1.5.0/med/alleles
47+
?for i in range(params.max_index):
48+
?i:
49+
hidden: true
50+
datasets:
51+
reads: ?f"{i}.reads"
52+
reference: ?f"{i}.reference"
53+
coverage: ?f"{i}.coverage"
54+
highlight: ?f"{i}.highlight"
55+
render-plot:
56+
spec-path: ?f"{input.plot_tables[i]}/{wildcards.sample}.vl.json"

‎meta/bio/alignoth_report/test/Snakefile‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
from snakemake.utils import min_version
2+
3+
min_version("9.13.1")
4+
5+
module alignoth_report:
6+
meta_wrapper: "file:.."
7+
8+
use rule * from alignoth_report

‎meta/bio/alignoth_report/test/resources/genome.fa‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
>1
2+
TGTGCACCCAGGCTAATTAATGTTCATGGGAACTAATTAGAGCTGACATGTACAGCCAGCCTTTTGCTTACTCTGCGGGATCTTGTTCTGTGTGATTTCTCTAGGGGGAGCCTGCTCACTTTGAGTATTTGATGTATCCATTGCACTCAGCACCCATCACCGGTCTAGCTACCTGCATCCGCAAACCCCTTATAGCCACCTGTTCTCTGGATCGATCCATCCGCCTTTGGAATTATGAAACAAAGTAAGGAATGAAAGGCTTGCCTACTTTATTATGCAAGACCCCAGTTGTGAATTTATGTGAATTATTTTAATTACAGTGTTCCAAACTTTAATTATTTAGAAGCTTTCAGTTGGAAAAGATTTTTAAAAAATCAAGTCAGTCAACCTACCTGTTTCTCACTCTTGCGTGGGCTAGAAAACTACTTTGTGGTTTATTTAATGCTTACCACATGCTAGACACTGGGGACATGAAGAGATGAAGACAACACCTCAGTCTCTCACCTTCTTACATAAACTCCTGGGTCCCTCCAGCTCTCCTATCCCCTTGTTTCCCACAGGAACTGCTCTTTGCTTCAGGGAGCTCTCCAGACCCTTTCTTCCTCATCTTTCTCCCAGGACATCACTATCTCAACCACCCCCAACTCCCTCCTGACTTTACCTTCCAGCACAGCTTCCCTGGGAAGCTCCAACCCTAGGTGGACCCCCACCCCCTACTGTTTTCTTTGTTCTTATGTACAACTTCTGGGGCTTCTGGGGAAAAAATACTTGGACAATACTCATGAATTTTGGAATATAAACAGAGCTATGTCTTCAGCATCGCTAGACCACACCTTGACCAGTCTCCCTATCTACCATGCTCCACGCTGGACATTATCCCCTAATAACCAAGATCTAGAAAAGAGAGGAACCAACAATTGCAAACAGTAGTGTGTCTCAAATACGATGATAACAATATATATCAGATACAGTAGATGACTGGAAGGATTTAGTGAGCTAAGCTTACAATTCAGTAATTGATCTTTTTCACAGCACCCTGGAACTATTTAAGGAATACCAAGAAGAGGCATATTCCATCAGCCTTCATCCATCTGGACACTTCATTGTAGTAGGGTTTGCTGACAAACTACGCCTCATGAATCTACTCATTGATGATATACGTTCTTTCAAAGAATACTCTGTTAGAGGATGCGGAGAGGTAAAAAAAAAACTGCTGAAGACAAAAGTCCAGTTTCTTAGAAAGCCACGGCTGAAATATCAGGAACTAACATGATCTGGGGAGGTGGGACCAAAATCTGTCTTGGAGAACTACAGTGGCTTAAAAAAAGGGGGAAGGAGGAAATCCAGTTATTTTCCCAGCAACCTTATTGAGTATATGGTTCCATCCCCATATTCTTATGTCCACCAGACTATTTCATTGAACCTAAAACCTTGTGAGATCCCCGCTATGATAAGTCCTGTCTGTAATTGTGGAATATTAACAAATATTTTAAAAGAAAAGTAATCTCAGAATCAGAGTCACAGCTAAGAAGGTGAGGTTGGAGACGGAGGAGCCAGGGAAAAATAAGCATCAATGGTGAGAGGTAATAAAGAGCGCATCAGCTTTCCCTGAGGTGTCTTGCAGAAACAAGGCAATGATTTCAGGAATTACTTCCCCGGCTCTGAAACATGCTTCTACAGGCAAAATGATTGCTTGTGGTCCCAACGGCTTAATGATTTGTGGGCCTGACAGTGTCAGCATCTGGCCACTATGAGTTACAGGAATAATGAAAGTGTCTGAAAATGTCAGCCACTCCTCAGTCTCATTTGCACCTTCCCCCCACTCCCACCCTCACACGTAGTTTCAGTCTGAACTCGTTGGGAAAAGCACCTCTTGACTGTAACCTCTAATTAGACTGCTCACTTCCTGCTTGCTTGCTCTCTTGCTGTCTTTCCCTATAGTGTTCCTTTAGCAATGGAGGTCACCTGTTTGCTGCAGTCAATGGAAATGTGATTCACGTTTACACCACCACGAGCCTAGAGAACATCTCAAGCCTGAAAGGACACACAGGGAAGGTAAGTGAGTGAACAGTCTCTGGGGAAACAAGGGGCACGGAGCCAAGTATGCCGCCAGCCAGTGGGATACAGGTGAACAAAAGAGAGATGGTTCCTTTCCTCATGGGTTCACTGTCTACTTGGGGAAAGAGACAATAAAACAAATGCACAGACAAATGTAGTTACTAATTGCAGTTCCACTGAAAGAAAAACATGGTGTTGTTGTGAGAGAAATAATGAAGGGGGCGGCATAATTTAGGTGGCTGTATTTGCAAAGGCTTGGTCCTATAAGAGTGAGTAGGAGTTAATCAGACACAACTGGTGGGGATAATCACTGCAGATAGGGATCCAAAGGCCAGAGATGGGAGACTCATAGG

‎meta/bio/alignoth_report/test/resources/genome.fa.fai‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
1 2411 3 2411 2412

‎meta/bio/alignoth_report/test/results/NA12878.bcf‎

370 Bytes
Binary file not shown.

‎meta/bio/alignoth_report/test/results/NA12878.bcf.csi‎

89 Bytes
Binary file not shown.

0 commit comments

Comments
 (0)