feat: minor speedups #56
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nh13
commented
Feb 26, 2025
src/bin/commands/demux.rs
Outdated
| @@ -93,6 +93,7 @@ impl ReadSet { | |||
| const SPACE: u8 = b' '; | |||
| const COLON: u8 = b':'; | |||
| const PLUS: u8 = b'+'; | |||
| const READ_NUMBERS: &[u8] = &[b'1', b'2', b'3', b'4', b'5', b'6', b'7', b'8']; | |||
There was a problem hiding this comment.
profiling showed that converting the read number (usize) to a string was costing a non-trivial amount of time. Here we have a fast path for read numbers up to 8.
| output_dir.join(format!("{}.{}{}.fq.gz", prefix, file_type_code, idx)), | ||
| )?)); | ||
| output_type_writers.push(BufWriter::with_capacity( | ||
| 65_536usize, |
There was a problem hiding this comment.
I was seeing write_all_cold in the flame graph, which means that our buffer is improperly set (default 8_192usize). Increasing this by 4x sped things up and the write_all_cold call no longer shows up.
src/lib/barcode_matching.rs
Outdated
| expected_bases.len(), | ||
| sample.sample_id | ||
| ); | ||
| if sample.barcode_bytes.len() != observed_bases.len() { |
There was a problem hiding this comment.
two things:
- The creation of
observed_stringwas causing this function to be slower than it needed to be, sinceobserved_stringonly is needed if we are going to panic. - Keeping a copy of the sample barcode as a vector of bytes is faster than calling
as_bytesevery time we compared the observed barcode to a sample (which is a lot)
src/lib/barcode_matching.rs
Outdated
| for (&expected_base, &observed_base) in expected_bases.iter().zip(observed_bases.iter()) { | ||
| if !byte_is_nocall(expected_base) && expected_base != observed_base { | ||
| for (&expected_base, &observed_base) in sample.barcode_bytes.iter().zip(observed_bases.iter()) { | ||
| if expected_base != observed_base && !byte_is_nocall(expected_base) { |
There was a problem hiding this comment.
swapped the order of the conditionals since it is much more likely that we have a mismatch than a no call in the sample barcode.
nh13
force-pushed
the
feat/minor-speedups
branch
from
f76d08b to
2e7e419
Compare
nh13
force-pushed
the
feat/minor-speedups
branch
2 times, most recently
from
10776b3 to
803f453
Compare
* replace ahash with rustc-hash. rustc-hash was faster than ahash and fxhash empirically. * optimize how the read name, specifically the read number, is written. * increase the buffer size by 4xfor the output writers to 65K. * delay creating the read bases from bytes for the display in an panic until it is needed. This occurred during every comparison of the read sample barcode with the expected sample barcode, so while not individually expenseive, expensive enough in aggregate. * create the necessary bytes for each sample barcode only once, since this was being done every time we compared barcodes. Miscellaneous changes: * fix the usage for threads, where it incorrectly said the minimum number of threads was three, and should be five. This also changes the README. * update rust toolchain to 1.85 (from 1.65.0). This requires minor code changes throughout based on clippy requirements and formatting.
nh13
force-pushed
the
feat/minor-speedups
branch
from
803f453 to
4c6a22d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This speeds up fqtk ~10% in my hands.