Docker - #90

Merged

stefinfection merged 420 commits into

mainfrom

Jul 29, 2026

This pull request is big! We're only showing the most recent 250 commits

Commits on Nov 2, 2025

Look for resources locally first, then fetch from S3 if necessary
stefinfection
committed
67f88f8
Update singularity container image variable to docker image one
stefinfection
committed
a3c49dc
Make env file argument mandatory; use docker resources script via S3 to fetch resources if needed (don't require S3 locally); add download_hash script in resource_helpers
stefinfection
committed
d1ffa12
split region worker from launch script because parallel doesn't inherit env vars
stefinfection
committed
3e3f76e
launch script delete whooops
stefinfection
committed
57b90d3
lowercase docker container name
stefinfection
committed
7b1ee2c
lowercase rufus in docker image pull; add no sign request option to s3 sync calls for hash fetching
stefinfection
committed
9857a0c
use aws cp instead of sync for hash files
stefinfection
committed
923d3af
Update variable name in download script
stefinfection
committed
598308e
missing var marker
stefinfection
committed
3c79f09

Commits on Nov 3, 2025

Commits on Nov 4, 2025

Commits on Nov 5, 2025

Commits on Nov 6, 2025

Commits on Nov 11, 2025

Sync branch
stefinfection
committed
e519214
Merge branch 'docker' of github.com:stefinfection/RUFUS into docker
stefinfection
committed
6bed639

Commits on Nov 13, 2025

Commits on Nov 14, 2025

add index checks for bams and crams; set kmer length to 25 in temp env file
stefinfection
committed
7d19463
fix paths in region worker
stefinfection
committed
59dcc94
match working version
stefinfection
committed
4a0a52f
spacing mistake for run mounts; keep ref clause even if we have to build bwa indexes; update any realpaths for ref
stefinfection
committed
4c42cb0
use correct ref path
stefinfection
committed
cc28867
debugging progress
root
committed
6c85f3a
working post p
Ubuntu
committed
09b9076
switch to supplementals instead of resources; correct post p cleanup
stefinfection
committed
40698e4
Compartmentalize cleanup within container; more robust checking for hash downloads
stefinfection
committed
57736e6

Commits on Nov 15, 2025

Clean outputs; working for both local and remote hashes
stefinfection
committed
c7803cc
Don't need container cleanup
stefinfection
committed
6b68605
remove dev mount
stefinfection
committed
849315d
Remove redundant launch file so we don't get confused; update readme
stefinfection
committed
b774c08
Copy over readme
stefinfection
committed
a21c0c4
Add processing output per region
stefinfection
committed
7b1e164

Commits on Nov 30, 2025

samtools pointing to container location
stefinfection
committed
f82b995
remove gcc check
stefinfection
committed
bcf5eb1
erroneous ;
stefinfection
committed
6acb30c
ignore sifs
stefinfection
committed
0173658

Commits on Dec 1, 2025

get rid of repeat if statement
stefinfection
committed
9b5c199
vcf header fixes
stefinfection
committed
2954e8e
vcf rufus command fix
stefinfection
committed
c1321f7

Commits on Dec 11, 2025

update vcf header out
stefinfection
committed
2b4e698

Commits on Dec 12, 2025

strip off region postfix from vcf sample names
stefinfection
committed
35611da
brackets
stefinfection
committed
4d21af1
remove redundant version echo
stefinfection
committed
05cc9b0
spacing
stefinfection
committed
9285701

Commits on Dec 13, 2025

Update trim and combine to work with multiple controls; remove prefiltered vcf processing
stefinfection
committed
36ca118

Commits on Dec 15, 2025

fix rufus_cmd file deletion bug that was bricking cleanup
stefinfection
committed
61d18c4
update arg parser to work with multiple controls
stefinfection
committed
e18397b

Commits on Dec 16, 2025

Commits on Dec 18, 2025

Remove all absolute paths for tools; update docker def to put general tools in path
stefinfection
committed
6eacb58
Fix hard coded overlap depth of 5
stefinfection
committed
288c15c
debug exit
stefinfection
committed
5b6f61b
Merge branch 'singularity_delta' of github.com:stefinfection/RUFUS into singularity_delta
stefinfection
committed
b83792b
fix done keyword on newline in rufus_call slurm script
stefinfection
committed
bcd6f84
Add mem_per_job and cpus_per_job parameters in setup_slurm script for rufus calling stage; also add srun memory cap to the same agrument/default
stefinfection
committed
99ad14c
left off notes + RUFUS_ROOT echo
stefinfection
committed
96df3af

Commits on Dec 19, 2025

trap file cleanup instead of explicitly calling
stefinfection
committed
d1a1d7a
Merge branch 'singularity_delta' of github.com:stefinfection/RUFUS into singularity_delta
stefinfection
committed
d993eaf

Commits on Dec 20, 2025

Added post-process cleanup trap; made remove_coinheriteds compatible with multiple controls; cleaning all intermediate files w/ post finishing
stefinfection
committed
eb9831c
set defaults for cpu/mem variables if empty; fix for loop bug in setup for rufus_call script
stefinfection
committed
1778253
move static defaults to arg_parser instead of setup_slurm so don't have to use literals
stefinfection
committed
98cbada
switch file removal to find
stefinfection
committed
e6d5b80
edit finds so we don't delete the temp final vcf per region
stefinfection
committed
ee871db
line break at end of loop in setup
stefinfection
committed
17eb621
for loop setup new line fix; fix command txt name
stefinfection
committed
0cd1913
fix default setting for cpus/mem/threads
stefinfection
committed
0b62340
Update empty assembly result to non-error exit code
stefinfection
committed
334bab5
switch from tbi to csi; clean up post process with find for temp region vcfs; delete indexes along with vcfs;
stefinfection
committed
1d25255

Commits on Dec 21, 2025

update arg checking; correct file deletion retaining final vcf; safeguard finds so we don't send in wildcard only
stefinfection
committed
7748400

Commits on Dec 22, 2025

Safeguard against premature clean up exit if no final vcf (e.g. when assembly fails or hashes are empty); force removal of Intermediates/TempOverlap directories with find even if have files in them
stefinfection
committed
add363e

Commits on Dec 29, 2025

make mnt dir in def file - singularity does not make NE directories like docker does on binding; don't make a prefinal vcf if we exit 0 from empty assembly step
stefinfection
committed
8c28cd5

Commits on Jan 16, 2026

fix post interpret vcf
stefinfection
committed
65a4785
temp final vcf post calling stage correctly reporting variants; correct header information
stefinfection
committed
e898f72

Commits on Jan 17, 2026

Remove debugs; move /mnt dir creation to container def file
stefinfection
committed
809f607
remove change wd to def file
stefinfection
committed
339a063
Have to go to /mnt in main script
stefinfection
committed
4e7b903
add specific tag to def file
stefinfection
committed
b79a05d
Merge branch 'singularity' of github.com:stefinfection/RUFUS into singularity
stefinfection
committed
079d15b

Commits on Jan 18, 2026

Update zenodo link
stefinfection
committed
815d96f

Commits on Jan 23, 2026

removing cmd file
stefinfection
committed
2ba025c

Commits on Jan 24, 2026

Commits on Jan 26, 2026

Commits on Jan 27, 2026

Commits on Jan 28, 2026

Commits on Jan 29, 2026

Commits on Jan 30, 2026

Update RUFUS interpret to include contigs in header; add -h parameter when viewing bams in overlap
stefinfection
committed
598df6b
Update sam header parsing in interpret
stefinfection
committed
818bdd8
build update script
stefinfection
committed
78f6a3f
Remove rufus_temp region specific files in cleanup
stefinfection
committed
db79c23
Remove bam output check in overlap
stefinfection
committed
19da09d
Clean up old, broken header/contig parsing
stefinfection
committed
7a3b4db

Commits on Jan 31, 2026

Test suite fully functional, added in vcf results
stefinfection
committed
2a4cb4b
remove intermediates
stefinfection
committed
acc7a7c

Commits on Feb 4, 2026

Commits on Mar 2, 2026

Update slurm setup to not require all inputs in single directory; add local hash validation and s3 fetching to slurm setup - replace process_region_workers.sh functionality with this new shared scr…
stefinfection
committed
ab37059
trap
stefinfection
committed
18a0870
Shift hash fetching to internal script; update readme with hash examples; update whole genome hash resolution to setup time
stefinfection
committed
3603b1e

Commits on Mar 5, 2026

Look for full parent paths when checking indexes; remove old debug out
stefinfection
committed
860ed7f
Fix breaking, redundant file path pass to remove_coinheriteds
stefinfection
committed
bad5632
If non-zero exit code, keep intermediate directory for debugging (in trap)
stefinfection
committed
349cbc0
Fix broken remove-coinheriteds script
stefinfection
committed
1c59228
Merge branch 'docker' of github.com:stefinfection/RUFUS into docker
stefinfection
committed
3454f99

Commits on Mar 6, 2026

Fix bcftools sort commands; exits with 1 if no temp vcfs found; removes passed working dir argument
stefinfection
committed
8fc9341
Update setup script to comply with new post-process script
stefinfection
committed
e469186
Remove region substring from parent column in vcf to allow for concatenation; replace fragile parent string parsing with find ops instead of hardcoded lengths
stefinfection
committed
f71ccfe
remove rufus.cmd storing; add exit reasons + logging; combine in post_process; fix parent file path in vcf header
stefinfection
committed
667f678

Commits on Mar 7, 2026

Add exit code logging for region mode; add cleanup for intermediate post-process files
stefinfection
committed
d3ad71d

Commits on Mar 9, 2026

Fix silent failure of removing region dir in cleanup trap
stefinfection
committed
24e7861
Fix header types throwing updated bcftools warnings
stefinfection
committed
d829820
Fix bcftools segfault crash when trying to intersect control pileup and rufus vcf in remove_coinheriteds
stefinfection
committed
bf37c65
update singularity readme
stefinfection
committed
a163a68
Update download link for RUFUS singularity image
stefinfection
authored
7b4412d
Allow find ops for hashes to follow symlinks
stefinfection
committed
c1a89e9
Merge branch 'docker' of github.com:stefinfection/RUFUS into docker
stefinfection
committed
64b41e3
Detect input file type and update reference flag to -cr for crams when creating rufus_call slurm script
stefinfection
committed
3ede7c5
Ensure only a single file type put in for subjects+controls - i.e. no mixtures of bams & crams, crams & fastqs, etc
stefinfection
committed
4b2fd5c

Commits on Mar 16, 2026

Commenting + clearer exit messages
stefinfection
committed
4b69685
Enable multiple subject input files
stefinfection
committed
848ebb3
Add dev bind to setup scripts for testing
stefinfection
committed
6e01580
version bump setup slurm
stefinfection
committed
41c0cab

Commits on Mar 31, 2026

Change hash check to only first and last 10 to speed up; fix files, region, and count check bugs in remove coinherited logic
stefinfection
committed
c430448

Commits on Apr 7, 2026

Address bcftools merge and call bugs for multi-allelic records bricking remove_coinheriteds; bump post_process version to 1.1.9
stefinfection
committed
0719594

Commits on Apr 30, 2026

Clean up legacy files; preserved in legacy/full-codebase branch
stefinfection
committed
9f83c0f

Commits on Jun 4, 2026

Fix slurm time argument pickup in setup script
stefinfection
committed
744ff5b
Add to ignore list
stefinfection
committed
fb050cc

Commits on Jun 20, 2026

Add parent low coverage argument to accommodate DSA controls; add hash size argument override; set default whole genome hash size to 64G to accommodate high depth samples
stefinfection
committed
15f147e

Commits on Jun 22, 2026

Change default wg size to 16G for hash build; multi-thread samtools view command in generators; remove PassThroughSamCheck (paired and single end) and replace with samtools collate which has multit…
stefinfection
committed
5f237af
Add Filter single end source back in and update cmake to build in container
stefinfection
committed
c4102fb
version bump
stefinfection
committed
cb9377c

Commits on Jun 30, 2026

Consolidate container build and add CI/CD pipeline

stefinfection
and
claude
committed
7c29ee1
Fix samtools/bcftools build: point at /opt/htslib + ldconfig

stefinfection
and
claude
committed
173118f
Drop vestigial ubuntu-toolchain-r PPA that broke the apt layer

stefinfection
and
claude
committed
db5d5db
Regenerate modifiedJellyfish.tar.gz before cmake

stefinfection
and
claude
committed
420b666
Restore tracked modifiedJellyfish.tar.gz; build from it directly

stefinfection
and
claude
committed
c942646
Add release-tag guard: tag must match RUFUS_VERSION in globals.txt

stefinfection
and
claude
committed
bb8b844

Commits on Jul 1, 2026

Vendor modified jellyfish as a pinned submodule

stefinfection
and
claude
committed
a2803cc
Fix submodule jellyfish build: autotools mtimes + add file pkg

stefinfection
and
claude
committed
878ca14
Bump CI actions to Node 24 runtimes

stefinfection
and
claude
committed
2e75334
Wire build job to GitHub Environments (dev/staging/production)

stefinfection
and
claude
committed
e97c9b6

Commits on Jul 21, 2026

Fix broken variant calling and add functional test suite

authored and
stefinfection
committed
025e3b2
Validate BWA indexes at slurm setup instead of failing mid-run

stefinfection
and
claude
committed
786e6a6
Validate BWA indexes in the AWS launch path

stefinfection
and
claude
committed
ff3e750
Drop ENVARGS overlay from F1 now that the image sets BCFTOOLS_PLUGINS
stefinfection
committed
d33d4d9
Add F2 somatic case; drop stale ENVARGS overlay from F1
stefinfection
committed
0a2584d
Merge pull request #89 from stefinfection/fix/bwa-index-validation
stefinfection
authored
34a7dee

Commits on Jul 22, 2026

Guard against 255+ chars in filenames
stefinfection
committed
7b58878
Support paired-end FASTQ input (whole-genome)

stefinfection
and
claude
committed
9dceeaf
Fix single-end FASTQ/filter hang (FIFO held open by exec fd)

stefinfection
and
claude
committed
246685c
Add FASTQ functional test cases (paired + single-end)

stefinfection
and
claude
committed
c1c9180
Add F3 input-format concordance case (BAM vs CRAM)

stefinfection
and
claude
committed
14a70ea

Commits on Jul 28, 2026

Validate BWA indexes in runRufus.sh itself

stefinfection
and
claude
committed
b76bd46
Add RUFUS.interpret functional documentation and defect audit

stefinfection
and
claude
committed
463adb8
Add RUFUS.interpret replay harness and chr20_m5 baseline

stefinfection
and
claude
committed
73ae1bf
Merge master README update into docker
stefinfection
committed
80ada23
Merge interpret replay harness and audit

stefinfection
and
claude
committed
378af22
Point the staging trigger at main, not master

stefinfection
and
claude
committed
c5fc3cf
Distinguish a failed k-mer count from a region with no coverage

stefinfection
and
claude
committed
836d134
Document that the coverage model is never built, and correct the help text

stefinfection
and
claude
committed
8b434b6
Make VCF-finalization resume-safe: bgzip -f / index -f

stefinfection
and
claude
committed
8f43b26
Fix control-hash query: pass full path, not stripped basename

stefinfection
and
claude
committed
9e07891
docs(interpret audit): verify FILTER=fail and correct 2.1 scope from a WG run

stefinfection
and
claude
committed
d8e1f03
Bump version and branch in globals to 1.2.0
stefinfection
committed
b8bbbde
Correct audit 2.1: the model phase is unreachable in all modes, not just -min/exome

stefinfection
and
claude
committed
175c909
Skip remove_coinheriteds when no BAM/CRAM control is provided

stefinfection
and
claude
committed
5130bb3
Stop doc version strings drifting from globals.txt

stefinfection
and
claude
committed
c29cd64
post_process: find the whole-genome temp VCF (add .wg postfix)

stefinfection
and
claude
committed
c8b960e
README: document both Docker Hub and Zenodo download routes

stefinfection
and
claude
committed
1187116
README: correct the input-data description; note the container-runtime TODO

stefinfection
and
claude
committed
da005ff
README: document single-sample mode for the SLURM helper; note getopts bugs

stefinfection
and
claude
committed
a5fa8ea

Commits on Jul 29, 2026

Make -C cpus_per_call actually take its argument

stefinfection
and
claude
committed
04ccab0
README: align the argument blocks to consistent columns

stefinfection
and
claude
committed
1195f66
Fix help cosmetic fix in arg_parser slurm setup script
stefinfection
committed
83d43d6