v0.11.1
- Expose the number of available CPUs for concurrent processing (#244). The library now provides a method to determine the number of logical CPUs available for the current process, considering factors such as containerized environments where the available CPU quota may differ from the total number of CPUs present. This method checks for the availability of the
process_cpu_countattribute, and if not available, attempts to use thesched_getaffinityfunction on Linux or falls back to the total number of CPUs in the system, defaulting to 1 if unknown. Thegathermethod has been updated to utilize this new method, allowing for more accurate determination of the available CPU count and improved concurrency. Additionally, several test cases have been added to verify the correct behavior of the method, including scenarios where the count is retrieved from different sources, ensuring a reliable way to determine the available CPU count for configuring concurrent processing in downstream applications. - Improve support for reading text files that contain a Unicode BOM at the start (#243). The library now provides enhanced support for reading text files that contain a Unicode Byte Order Mark (BOM) at the start, allowing for accurate detection and handling of the file's encoding. New methods have been introduced to detect the BOM and decode the file accordingly, including handling of decoding errors and newline characters. The
read_textfunction has been added, enabling the reading of text files with a BOM prefix, and is designed to work with both seekable and non-seekable files, although specifying a read size for non-seekable files will raise an error. Additionally, the existing code for handling Workspace files has been refactored to utilize the same implementation, and improvements have been made to support non-seekable files, ensuring a more robust and reliable reading experience for text files with Unicode BOM markers.
Contributors: @asnare