v4.1.3

- Fix: when the paired end files are compressed, read chunks did not resize, which led to excessive copy, and copy number might accumulate round by round.

Author: cihga39871

CHANGELOG.md

@@ -5,6 +5,10 @@
 - Feature: multiple primer trimming.
 - Feature: UMI trimming.
 
+## v4.1.3
+
+- Fix: when the paired end files are compressed, read chunks did not resize, which led to excessive copy, and copy number might accumulate round by round.
+
 ## v4.1.2
 
 - Fix: do not throw error if input paired end files are empty when doing `--detect-adapter`.
@@ -84,13 +88,11 @@
 ## v3.1.0
 
 - New feature: `--detect-adapter` for adapter determination.
-
 - Fix: when input is an empty compressed fastq, atria exits with error because `read_chunks!(::IO, ...)` should return 4 elements, but returned 2.
 
 ## v3.0.3
 
 - Fix v3.0.2: `will_eof` should be true when unknown.
-
 - Do not resize chunk sizes before cycle 1 when inputs are compressed and cannot determine uncompressed sizes. Just assume data are not trimmed before.
 
 ## v3.0.2
@@ -100,9 +102,7 @@
 ## v3.0.1
 
 - Avoid to lock `IOStream` when write fastq in thread_output.jl: replace `write(::IOStream, ...)` with `write_no_lock(::IOStream, ...)`. It is slightly faster.
-
 - Speed optimization for consensus calling: overwrite `BioSequences.complement(::DNA)` (1.40X), and define `iscomplement(::DNA, ::DNA)` (1.79X).
-
 - Other minor parallel implementations.
 
 ## v3.0.0
@@ -112,7 +112,6 @@
 ## v2.1.2
 
 - Parameter optimization using `atria simulate`: --trim-score-pe 19->10, --tail-length 8->12.
-
 - Development of Atria simulation methods.
 
 ## v2.1.1
@@ -126,17 +125,11 @@
 ## v2.0.0
 
 - Supporting low-complexity filtration.
-
 - Supporting polyX tail trimming.
-
 - Supporting single-end fastq.
-
 - Supporting bzip2 compression/decompression.
-
 - Supporting non standardized gzip compression files.
-
 - Optimizing default parameters. (r1-r2-diff 0->0, trim-score-pe 8->10, score-diff removed, kmer-n-match 8->9)
-
 - Robustness optimization: the lower bound of match probability is set to 0.75 because match probability lower than 0.75 is outlier and affect trim score strongly.
 
 ## v1.1.1
@@ -146,19 +139,14 @@
 ## v1.1.0
 
 - Performance optimization: adapter and PE trimming: if no adapters were matched, the number of errors of PE match is loosen.
-
 - Performance optimization: consensus calling: new arg `--kmer-tolerance-consensus 2->10`; optimized arg `--min-ratio-mismatch 0.2->0.28`.
-
 - Speed optimization: check `overlap_score > 0` before computing score (`pe_consensus!`).
 
 ## v1.0.3
 
 - More detailed error output when encoding a non-nucleotide character (`throw_encode_error(...)`).
-
 - Following symbolic link before checking file size for non-Windows platforms (`check_filesize(::String)`).
-
 - When run in multi-file parallel mode, write stdout and stderr to a 'stdlog' file (`julia_wrapper_atria(...)`).
-
 - Add option `--check-identifier` to check whether the identifiers of r1 and r2 are the same.
 
 ## v1.0.2

Project.toml

@@ -1,7 +1,7 @@
 name = "Atria"
 uuid = "226cbef3-b485-431c-85c2-d8bd8da14025"
 authors = ["Jiacheng Chuan <jiacheng_chuan@outlook.com>"]
-version = "4.1.2"
+version = "4.1.3"
 
 [deps]
 ArgParse = "c7e460c6-2fb9-53a9-8c5b-16f535851c63"

docs/1.2.Install_from_source.md

@@ -2,7 +2,7 @@
 
 ## Install from source
 
-Atria is tested in [Julia Language](https://julialang.org/) v1.8 and v1.9. 
+Atria is tested in [Julia Language](https://julialang.org/) v1.8 and v1.9.
 
 It is recommended to build Atria using Julia v1.8.5 because it is 3-20% faster than v1.9.
 
@@ -36,8 +36,7 @@ brew install pigz
 brew install pbzip2
 ```
 
-> If you do not use Homebrew, you can also download them from [pigz's official site](https://zlib.net/pigz/) and [pbzip2](https://pkgs.org/download/pbzip2).
-
+> If you do not use Homebrew, you can also download them from [pigz's official site](https://zlib.net/pigz/) and [pbzip2](https://pkgs.org/download/pbzip2).
 
 #### Atria
 
@@ -81,10 +80,10 @@ juliaup default 1.8
 
 Then, download `pigz` and `pbzip2` (a compression/decompression software used in Atria).
 
-If you use `apt` package manager (Ubuntu/Debian), try `sudo apt install pigz pbzip2`.   
-If you use `yum` package manager (CentOS), try `sudo yum install pigz pbzip2`.   
+If you use `apt` package manager (Ubuntu/Debian), try `sudo apt install pigz pbzip2`.
+If you use `yum` package manager (CentOS), try `sudo yum install pigz pbzip2`.
 
-You can also download them from [pigz's official site](https://zlib.net/pigz/) and [pbzip2](https://pkgs.org/download/pbzip2).
+You can also download them from [pigz's official site](https://zlib.net/pigz/) and [pbzip2](https://pkgs.org/download/pbzip2).
 
 #### Atria
 

src/Trimmer/wrapper_pe.jl

@@ -582,14 +582,11 @@ function julia_wrapper_atria_pe(ARGS::Vector{String}; exit_after_help = true)
             n_r1_before = length(r1s) - n_reads
             n_r2_before = length(r2s) - n_reads
 
-            if typeof(io1) <: IOStream  # not compressed
+            if typeof(io1) <: IOStream  # not compressed                
                 length(in1bytes) == chunk_size1 || resize!(in1bytes, chunk_size1)
                 length(in2bytes) == chunk_size2 || resize!(in2bytes, chunk_size2)
                 (n_r1, n_r2, r1s, r2s, ncopied) = load_fqs_threads!(io1, io2, in1bytes, in2bytes, vr1s, vr2s, r1s, r2s, task_r1s_unbox, task_r2s_unbox; remove_first_n = n_reads, njobs = njobs, quality_offset = quality_offset)
 
-                # it only get the sizes, did not change the sizes. Size changing is done in the "Read" part.
-                chunk_size1, chunk_size2 = get_ideal_inbyte_sizes(in1bytes, in2bytes, n_r1, n_r2, n_r1_before, n_r2_before, max_chunk_size, chunk_size1, chunk_size2)
-
             else  # gziped
                 total_n_bytes_read1 += length(in1bytes)  # will read INT in this batch
                 total_n_bytes_read2 += length(in2bytes)  # will read INT in this batch
@@ -606,6 +603,8 @@ function julia_wrapper_atria_pe(ARGS::Vector{String}; exit_after_help = true)
                     njobs = njobs
                 );
             end
+            # it only get the sizes, did not change the sizes. Size changing is done in the "Read" part.
+            chunk_size1, chunk_size2 = get_ideal_inbyte_sizes(in1bytes, in2bytes, n_r1, n_r2, n_r1_before, n_r2_before, max_chunk_size, chunk_size1, chunk_size2)
 
             n_reads = min(n_r1, n_r2)
             total_read_copied_in_loading += ncopied