Problem
We use MD5 for file hashing in the processing pipeline:
src/file-processor/file-queue-processor.provider.ts (line 121)
src/file-processor/helper/drive-helper.ts (line 29)
src/fileCleanup/file-cleanup-queue-processor.provider.ts (line 126)
MD5 collisions are practical, two different files can produce the same hash. If we rely on these hashes for dedup or integrity checks, that's a problem.
The concern: speed
These hashes run on large files (multi-GB). MD5 is roughly 30-40% faster than SHA-256 in software, which adds up on a 15 GB download.
That said:
- Modern CPUs (Intel Ice Lake+, AMD Zen+) have hardware SHA-256 instructions. Node's OpenSSL bindings pick these up automatically, such that the gap shrinks to near-zero on supported hardware.
- The hash probably isn't the bottleneck anyway. Network and disk I/O on a 15 GB file will dominate wall-clock time regardless of algorithm.
Before switching
- Run
openssl speed md5 sha256 on the deployment target to see actual throughput
- Figure out what these hashes are actually used for, if it's just cache keys, urgency is lower
- Check if the hash is even visible in profiling during large file processing
Fix
If perf is fine: createHash('md5') -> createHash('sha256'). Same streaming API, same hex output, just 64 chars instead of 32.
Problem
We use MD5 for file hashing in the processing pipeline:
src/file-processor/file-queue-processor.provider.ts(line 121)src/file-processor/helper/drive-helper.ts(line 29)src/fileCleanup/file-cleanup-queue-processor.provider.ts(line 126)MD5 collisions are practical, two different files can produce the same hash. If we rely on these hashes for dedup or integrity checks, that's a problem.
The concern: speed
These hashes run on large files (multi-GB). MD5 is roughly 30-40% faster than SHA-256 in software, which adds up on a 15 GB download.
That said:
Before switching
openssl speed md5 sha256on the deployment target to see actual throughputFix
If perf is fine:
createHash('md5')->createHash('sha256'). Same streaming API, same hex output, just 64 chars instead of 32.