v0.2.0 - Update of Performance

Release Notes for v0.2.0

With this release I am aiming to improve the performance of the wordlist_sort tool.

• Parallel Power for Multiple Files
  I've supercharged how wordlist_sort handles multiple input files! It now processes them concurrently using std::async. This means if you're working with several files, the tool will leverage your multi core processor to get the job done much faster. Each file gets its own thread, and the results are then efficiently combined.

• Smarter --dup-sense Logic
  The feature that filters out words with too many repeated characters (--dup-sense) got a major speed boost. I've moved from a slower, quadratic check (O(N²)) for each word to a linear one (O(N)) using a frequency array. This makes the --dup-sense option much more practical, especially with longer words.

• Efficient In Place Trimming
  Operations like --digit-trim and --special-trim now modify strings directly. This cuts down on creating temporary strings and copying data, leading to quicker processing.

• Accurate --noutf8 Behavior
  The --noutf8 option now correctly removes non ASCII characters (those with byte values greater than 127). This ensures that if you're using it (often alongside --dewebify), the output will be plain ASCII text, which was my original intention.

• Clearer Error Messages
  When things go wrong with file operations, the error messages are now more helpful. They'll include the specific file path that caused a problem, making troubleshooting easier. I've also added checks for issues like negative file sizes.

• Deduplication and Sorting Note
  I've clarified that if you use --deduplicate without also specifying --sort, the words will be sorted automatically first. This is because the deduplication process needs a sorted list to effectively remove all duplicates. Sound logical?

• More Descriptive Class Names
  Some internal classes for file handling have been renamed to better describe what they do. For example, MemoryMapping is now FileBuffer (since it buffers the file in memory rather than using true OS memory mapping), and CompressedMemoryMappedFile is now BufferedFile (as no compression was actually happening).

• Streamlined Codebase
  I've removed the FileDescriptor class as it wasn't really being used. Some internal helper functions for checking character types have also been given slightly more explicit names (e.g., is_digit became is_digit_char).

• Improved CLI Help
  The descriptions for several command line options have been updated to be clearer and more informative.

• Status of --email-split
  A note has been added to acknowledge that while the --email-split option can parse its parameters, the functionality to write to separate user and domain output files isn't fully implemented yet.

These changes work together to make wordlist_sort a faster, more reliable, and more developer friendly tool. I am very confident these improvements will provide a much better experience, especially when you're tackling large or numerous wordlists!