dwarfs-0.12.1

A quick update to v0.12.0 that addresses a few issues and improves the performance of the release binaries while mostly making them even smaller. The universal x86_64 binary is slightly bigger, but that's a different story [1].

	v0.11.3	v0.12.0	v0.12.1
Linux x86_64 universal binary	5,319,916	2,833,280	2,903,624
Linux aarch64 universal binary	4,637,312	2,725,864	2,588,924
Linux x86_64 fuse-extract binary	-	1,183,752	1,075,536
Linux aarch64 fuse-extract binary	-	1,188,760	1,059,588
Linux x86_64 binary tarball	7,736,712	3,888,104	3,698,356
Linux aarch64 binary tarball	6,791,424	3,497,140	3,271,584

Bugfixes

• Attempt to fix a linking issue in the Homebrew build.

Features

• Added --memory-limit=auto to mkdwarfs to use a more reasonably (hopefully) default for the block queue. The old default of 1 GiB was quite arbitrary and definitely not suitable for low-end systems. The new auto default will determine the limit based on the number of workers (which in turn is based on the number of CPUs), the block size, and the amount of physical memory of the system.

• Replaced vector_byte_buffer with malloc_byte_buffer, which is internally based around a simple buffer that doesn't incur the cost of initializing each element like std::vector. Especially for large blocks which are known to be overwritten immediately, this can save a few CPU cycles.

• The x86_64 release binaries now use an optimized memcpy implementation (if supported by the CPU) instead of the rather slow musl memcpy implementation. This makes mkdwarfs a few percent faster and dwarfsextract up to 20% faster.

Build

• Switched the release binaries to use mimalloc instead of jemalloc. The primary reason for this change is a reduction in binary size.

• Updated the xz library to the latest 5.8.1 release.

Full Changelog: v0.12.0...v0.12.1

SHA-256 Checksums

f61d49436ad6b02e7e496f746ce09d69a3f050592286b5ff3d6e3edba765b82a  dwarfs-0.12.1-Linux-aarch64.tar.xz
6685622e6bf1edea138023dfb5a84ec241c6a57619ec5d7ee86344057d89296b  dwarfs-0.12.1-Linux-x86_64.tar.xz
5523a5c3aea244cbfbccfe64f1df6053b3901e6af8916fac1530faf0f7a5f07f  dwarfs-0.12.1.tar.xz
d51c2e5ed021a7322928aeb8f09cc3c392362c8a1ea6217e2ba177f241f8a809  dwarfs-0.12.1-Windows-AMD64.7z
8850a0002d7008791c2629fd3a5bc718c50606dfc391bf5b1259d9f6d79c8401  dwarfs-fuse-extract-0.12.1-Linux-aarch64
d01dd82068e0d2020fd35ba5a2ddf416aeba32a9f6f7ac4544a173eefc6c743f  dwarfs-fuse-extract-0.12.1-Linux-x86_64
3dfc3d8d2152f4d9d29c543d0a30b2ba1bc0bba470742e08924ad75dc8c23967  dwarfs-universal-0.12.1-Linux-aarch64
518faa4f5a476dcc4ec75d8ed4ac31d076990bb515cd32a752a8165b3ad04885  dwarfs-universal-0.12.1-Linux-x86_64
1ad50c0b6127e56dc026de7d2f7a3df28fa3983bdd628584a652d05f97e28b88  dwarfs-universal-0.12.1-Windows-AMD64.exe

[1] I've noticed that the universal binary, and sometimes the other binaries as well, fluctuate dramatically in size after being UPX compressed. In the last CI build before tagging the new DwarFS release, the UPX-compressed universal binary had a size of 2,713,752 bytes. The only change after tagging the release was the version info (although, with LTO, you never know what the compiler makes of that). Interestingly, when unpacked, both binaries have exactly the same size. But the packed size differs by ~190 KiB.

dwarfs-0.12.0

The main features of this release are new licensing conditions and significantly smaller binaries.

New Licensing Conditions

Instead of being all GPL-3.0 like all the previous releases, this release changes the license of a large fraction of the DwarFS code to MIT. All tools and libraries that only read DwarFS images are now MIT-licensed. Everything else (e.g. mkdwarfs) is still GPL-3.0 for the time being.

Significantly reduced binary size

By moving the build pipeline to Alpine Linux and through some major refactoring, the size of the release binaries has been reduced significantly.

	v0.11.3	v0.12.0
Linux x86_64 universal binary	5,319,916	2,833,280
Linux aarch64 universal binary	4,637,312	2,725,864
Linux x86_64 binary tarball	7,736,712	3,888,104
Linux aarch64 binary tarball	6,791,424	3,497,140

There's now also an additional binary called dwarfs-fuse-extract that combines the functionality of the FUSE driver dwarfs and dwarfsextract in a single, extremely small binary:

	v0.12.0
Linux x86_64 fuse-extract binary	1,183,752
Linux aarch64 fuse-extract binary	1,188,760

The main use case for this binary is single-file application image runtimes (e.g. uruntime). This binary doesn't have built-in manual pages or support for the performance monitor. It also only supports zstd and lzma compression.

Bugfixes

• Build release binaries against an up-to-date libfuse. Fixes github #252.

• Changes for compatibility with Boost.Process v2.

Features

• Re-licensed all libraries required for reading DwarFS images under the MIT license. The source of all tools that just read DwarFS images (i.e. everything except for mkdwarfs) are also under the MIT license now. Everything else is still GPL-3.0. Addresses github #255.

• Significantly reduced binary size in the static release builds. This is the result of refactoring code that unconditionally pulled in code-heavy dependencies such as libcrypto, as well as optimizing the build pipeline (e.g. building dependencies with only the necessary set of features) and turning on link time optimization.

• A new kind of "universal" binary dwarfs-fuse-extract is part of the release now. This combines the FUSE driver (dwarfs) and dwarfsextract into a single binary, but does not include the mkdwarfs and dwarfsck tools that are also part of the regular universal binary. dwarfs-fuse-extract is much smaller than the regular universal binary and especially suitable to AppImage-like applications.

• New hotness categorizer in mkdwarfs that allows a list of "hot" files to be stored in distinct file system blocks.

• New explicit ordering mode in mkdwarfs that allows files to be ordered accoring to the order in a given list file.

• dwarfs now shows the version of the FUSE library used.

• New dwarfs options preload_all and preload_category to populate the block cache immediately after mounting.

• New dwarfs option analysis_file that can be used for profiling and as input to mkdwarfs new hotness categorizer and explicit ordering mode.

• New dwarfs option block_allocator that allows the user to switch from a malloc-based block allocator to an mmap-based one. This can help with returning memory back to the system if the blocks are evicted from the cache.

Full Changelog: v0.11.3...v0.12.0

SHA-256 Checksums

15867a3e1b5a310ea27700806ce7d504c912de8741dddbb40430eee419c4532f  dwarfs-0.12.0-Linux-aarch64.tar.xz
4fd1e23a97d871d0536b818c11a58c58859627e347d10c1e211522c8dd56b328  dwarfs-0.12.0-Linux-x86_64.tar.xz
91d5a22e5cf125a9871bcbdb4875bdd661557757b9f50e88553da4b47f8351d2  dwarfs-0.12.0.tar.xz
e9a4d513085bb91ab26f86277cfdfbf7930adf4bcb3c44be9f67f205d1db29fb  dwarfs-0.12.0-Windows-AMD64.7z
f7e7909b779a96dccced8e6749729ccf73ac5782ed5721cc27199498b84a27c1  dwarfs-fuse-extract-0.12.0-Linux-aarch64
264e6cbac5ade98275241082770e7aefb6e931612c543deb87322be03fc65bb8  dwarfs-fuse-extract-0.12.0-Linux-x86_64
41981a4f7a068f6c3bc32dad5307aa4466d0d74bf13083a495151d0ece34b17c  dwarfs-universal-0.12.0-Linux-aarch64
1425e102fd3b8251629025fc3b8aab0f74b7079466261f6fcef6c0e0531249ac  dwarfs-universal-0.12.0-Linux-x86_64
fe5904d64f1bd08f8f2f7dd271587b415bef1237562da4a77ee30982b48e24c4  dwarfs-universal-0.12.0-Windows-AMD64.exe

dwarfs-0.11.3

Bugfixes

• Handle absolute paths in --input-list. Fixes github #259.
• Don't prefetch blocks that are already in the active list within the block cache.
• Ensure that statistics for block tidying are correctly updated in the block cache.
• A few build fixes, mainly to simplify building on Alpine.

New Contributors

• @hexahigh made their first contribution in #257

Full Changelog: v0.11.2...v0.11.3

SHA-256 Checksums

de7f6609a4ddd6f2feff4cb4e43c4481515c5da178bbe12db24de9e7fec48bac  dwarfs-0.11.3-Linux-aarch64-clang-reldbg-stacktrace.tar.xz
7c0835b89871e48025b2e30577fb1b3c39927f9b86940dd3c5d7c41871e12533  dwarfs-0.11.3-Linux-aarch64-clang.tar.xz
2e771b53ebee66278b3a2e6e18fd04e20abc0f6defccb5a347dbfc2b7436b729  dwarfs-0.11.3-Linux-x86_64-clang-reldbg-stacktrace.tar.xz
adc3fc58d36848a312e846f0e737056b7e406894e24fa20d80fcc476ca7f401f  dwarfs-0.11.3-Linux-x86_64-clang.tar.xz
5ccfc293d74e0509a848d10416b9682cf7318c8fa9291ba9e92e967b9a6bb994  dwarfs-0.11.3.tar.xz
33b3488bc1097b1b2b54194eaa5fb169dfded9a6046de6c4fee693d9a97ece32  dwarfs-0.11.3-Windows-AMD64.7z
e14c0caa38a8d10273a84e57f532e513b2cbc50bb8df707b57c01d575f040a43  dwarfs-universal-0.11.3-Linux-aarch64-clang
7d4857ee18ffae705a41f164a0a810f173bf8d69bc8bef8dcbd1018fa8287f6e  dwarfs-universal-0.11.3-Linux-aarch64-clang-reldbg-stacktrace
64b349aec059b9d460211af6c517f6edd89e79c5e9581381229af745ebf3cc87  dwarfs-universal-0.11.3-Linux-x86_64-clang
07a9ef68256e76e7bda552b24955a3db12c3b34312d7d664e47639995ccdabf1  dwarfs-universal-0.11.3-Linux-x86_64-clang-reldbg-stacktrace
772f00d5d02fdebca4cbd74f4b1b37ee7b57fb3004a078c053b4f58e97a794ed  dwarfs-universal-0.11.3-Windows-AMD64.exe

dwarfs-0.11.2

Bugfixes

• macOS Ventura's version of clang appears to be missing an implementation of std::hash<std::filesystem::path, making it hard to define an unordered_map<filesystem::path>. Work around by simply using an unordered_map<string> instead.
• Installing the binaries using cmake did not honor the CMAKE_INSTALL_BINDIR or CMAKE_INSTALL_SBINDIR variables. Fixes github #253.

Full Changelog: v0.11.1...v0.11.2

SHA-256 Checksums

61fce8eaa6bbdf10917a5a12331e192748a54ab1aa175ed6f55cb26825ab3177  dwarfs-0.11.2-Linux-aarch64-clang-reldbg-stacktrace.tar.xz
06fc4ed91ee5c348dbfc70771fe3e3ea6834277e4a58f1f99e0bc98cb16ed3d4  dwarfs-0.11.2-Linux-aarch64-clang.tar.xz
15905007cff432bb9be0bdabed93473764c1706796e0da6f3af083f0a142db6d  dwarfs-0.11.2-Linux-x86_64-clang-reldbg-stacktrace.tar.xz
3c82708e00af9d1622e78047efd216e4e29213a60aff3afa8326bade8353ea38  dwarfs-0.11.2-Linux-x86_64-clang.tar.xz
1b38faf399a6d01cd0e5f919b176e1cab76e4a8507088d060a91b92c174d912b  dwarfs-0.11.2.tar.xz
8a028693ce0a7ab083b25dc491b100f41fbf98f28413a38f6773fe1cf27574fb  dwarfs-0.11.2-Windows-AMD64.7z
600134267dd0ad51dd9d8bd1b58fa614b0a0da9a7a3d57f5fce4dbda9bb80460  dwarfs-universal-0.11.2-Linux-aarch64-clang
a9f5f79afeff4eba5cc23893de46e4c8eaa3b51b8f5938ed7f9e6cb92560fa4f  dwarfs-universal-0.11.2-Linux-aarch64-clang-reldbg-stacktrace
1bee828de84c1a3a1c2134bc866f28bdf93a62927cb7e8c416813f389f7745ad  dwarfs-universal-0.11.2-Linux-x86_64-clang
ddbd62d3bf0bf420a1720af6c03bea21ce6a77a73cedc46f28c8d79e4ac26827  dwarfs-universal-0.11.2-Linux-x86_64-clang-reldbg-stacktrace
d95dab93a7e9d8349d4a4393213a401d9ded79040b1c48e661df7dfe118b72a7  dwarfs-universal-0.11.2-Windows-AMD64.exe

dwarfs-0.11.1

Bugfixes

• macOS Ventura's version of clang appears to be missing the <source_location> header, despite Apple claiming otherwise. Fix this by shipping a wrapper and providing a fallback implementation.

Full Changelog: v0.11.0...v0.11.1

SHA-256 Checksums

3e1b6331cf2f589d7058700aa2c5dc41f1825f3954f3828eb709034ba57a7c97  dwarfs-0.11.1-Linux-aarch64-clang-reldbg-stacktrace.tar.xz
23b1e0b18a7c3ffeb6c5fcc97ab032a7c1c651454d0fa5cb9741918d97a14ab3  dwarfs-0.11.1-Linux-aarch64-clang.tar.xz
4ec6614e87064ac96dfdb9b7957620bd889f5f1e95416409369d00606cfff0a1  dwarfs-0.11.1-Linux-x86_64-clang-reldbg-stacktrace.tar.xz
1eebf6e66eb5d6dc7cfb9c9b3c7c6e67084acc5ced7a018c15a511e929598f99  dwarfs-0.11.1-Linux-x86_64-clang.tar.xz
7a0cccb1ec3c2a18e9a014893c1d3e1f8f2c44ade6936c9f6d3bab5ec14b2052  dwarfs-0.11.1.tar.xz
c43fd9f2089b94ddd2819c7853dd6d8c34951e2d42cbfdc2e4470cde9c3e18fb  dwarfs-0.11.1-Windows-AMD64.7z
e9bf1f8bcccf363be25396d2f60d9f4e7765eba5bd647f071aa4d0ba5cb3785b  dwarfs-universal-0.11.1-Linux-aarch64-clang
22966f1dba98697db0cad127d1d8c50ef5952b5c9816cc2564b9410a37cdbaa3  dwarfs-universal-0.11.1-Linux-aarch64-clang-reldbg-stacktrace
0a025dc0f854ad9f3a5f9ca89ac43ada5305de14fe4b2e03088ce9ee5a23dbf4  dwarfs-universal-0.11.1-Linux-x86_64-clang
a38de846f48c7979c204699af6a1fda0d8d634caac68223b2b46fcb1c52c0e56  dwarfs-universal-0.11.1-Linux-x86_64-clang-reldbg-stacktrace
b8299e5f2102283c52c1cbd8ad14a0c3b71244937327e895e0aee809d92e4474  dwarfs-universal-0.11.1-Windows-AMD64.exe

dwarfs-0.11.0

Bugfixes

• Remove the access implementation from the FUSE driver. There's no point here trying to be more clever than FUSE's default. This makes sure DwarFS will behave more like other FUSE file systems. See github discussion #244 for details.
• Limit the number of chunks returned in inodeinfo xattr. Highly fragmented files would have megabytes in inodeinfo, which not only breaks the xattr interface, but can also dramatically slow down tools like eza who like to read xattrs for no apparent reason.
• Avoid nested indentation in manpages to work around ronn-ng bug. Fixes github #249.
• Don't link library against jemalloc. This fixes both issues with pydwarfs and issues building with jemalloc support on macOS. Only the binaries are now linked against jemalloc, which should be sufficient.

Features

• Support case-insensitive lookups. Fixes github #232.
• Allow setting image size in FUSE driver. Fixes github #239.
• Support extracting a subset of files with dwarfsextract using the new --pattern option. The same glob patterns can be used as for the filter rules in mkdwarfs. Fixes github #243.
• Allow overriding UID / GID for the whole file system when using the FUSE driver on non-Windows platforms. See github discussion #244.
• Expose more LZMA options (mode, mf, nice, depth).
• Improve filter patterns, which now support ranges and complementation.
• Improve speed of filesystem walk / walk_data_order calls by 80% / 40%. The impact of this will largely depend on what the code is being run for each inode, but, for example, the speed of listing more than 14 million files with dwarfsck will take about 16 seconds compared to 17 seconds with the previous release.
• Added an inode size cache to the metadata to speed up file size computation for large, highly fragmented files. The configuration is currently fixed using a conservative default. Only files with at least 128 chunks will be added to the cache, so in a lot of cases this cache may be completely empty and not contribute to the size of the file system image at all.
• Use bit-packing for hardlink, shared files, and chunk tables. This will consume less memory when loading a DwarFS image.
• Show total hardlink size in dwarfsck output.
• Library: return a dir_entry_view from readdir and find. This is more consistent, but was previously not easily possible due to the lack of a "self" dir entry in the metadata. The "self" entry has been added and will only impact the size of the metadata if directories metadata is not packed.
• Library: prefer std::string_view over char const*.
• Library: add directory iterator to directory_view.
• Library: support for maxiov parameter in readv call.

Other

• Lots of internal refactoring to improve overall code quality.

Full Changelog: v0.10.2...v0.11.0

SHA-256 Checksums

2040a951697ddb78a4b6ee887e06be4295f9d2e3708a311ac72ad9ad7bd28aa3  dwarfs-0.11.0-Linux-aarch64-clang-reldbg-stacktrace.tar.xz
0db0d6bc823d26f86d47f1edf8b4ddbcf85fab24e118be7de9ee091234b5623e  dwarfs-0.11.0-Linux-aarch64-clang.tar.xz
a7214b10902653c582aa4c21e05e2476518ed1d15e4640cc3eb2bbe53a297120  dwarfs-0.11.0-Linux-x86_64-clang-reldbg-stacktrace.tar.xz
35e851bce5ba6a17b6b53081d1305ebcee5698d8bc770b8b1a875d2986fd6d7c  dwarfs-0.11.0-Linux-x86_64-clang.tar.xz
852c96133444493eff6f03324bc2700e31859d75410a937f0714eae9f75d2dd4  dwarfs-0.11.0.tar.xz
15591223010400488c5066a864bcee3ad71c045e2aa4bf60b7c05e9d45909b9f  dwarfs-0.11.0-Windows-AMD64.7z
da197d19b3eadfea5180034765d70c050ae9b85ade58dd0aa91b65283a079236  dwarfs-universal-0.11.0-Linux-aarch64-clang
d58ad14583345d4e7efb4ddb0278ec39c836646a39868422ca1358fa22a990b7  dwarfs-universal-0.11.0-Linux-aarch64-clang-reldbg-stacktrace
72fe171dd9d9abd0bba46e52a983934affbcc9a7349d07854eda91d788ea686b  dwarfs-universal-0.11.0-Linux-x86_64-clang
1c5b19c21aca4dc6df8cff3e06358c96fb4e3bb1e969ed3ceef0eb381d84f98b  dwarfs-universal-0.11.0-Linux-x86_64-clang-reldbg-stacktrace
f2451ed0832c13157f869a3d7ba3596fcb4bb0c5c55741fc054ce6b1bdc977c8  dwarfs-universal-0.11.0-Windows-AMD64.exe

dwarfs-0.10.2

Bugfixes

• Gracefully handle localized error message on Windows. These error messages can contain characters from a Windows (non-UTF-8) code page, which could cause a fatal error in fmt::print in the logging code. Call sites that log such error messages now try to convert these from the code page to UTF-8 or, if that fails, simply replace all characters that are invalid from a UTF-8 point-of-view. Partial fix for #241.
• Handle invalid wide chars in file names on Windows. For some reason, Windows allows invalid UTF-16 characters in file names. Try to handle these gracefully when converting to UTF-8. Partial fix for #241.
• Workaround for new boost versions which have a process component.
• Workaround for a deprecated boost header.
• Support for upcoming Boost 1.87.0. io_service was deprecated and replaced by io_context in 1.66.0. The upcoming Boost 1.87.0 will remove the deprecated API. (Thanks to Michael Cho for the fix.)
• Disable extended output algorithms (shake(128|256)).
• Install libraries to CMAKE_INSTALL_LIBDIR. Fixes #240.
• mode/uid/gid checks were expecting 16-bit types.
• stricter metadata checks and improved error messages.
• Various fixes for filesystem_extractor to prevent memory leaks, correctly handle errors during extraction, and prevent creation of invalid archive outputs due to padding.
• Various minor fixes: non-virtual dtors, missing includes, std::move vs. std::forward, unused code removal.

Other

• More test cases for stricter metadata checks. Also enable the strict checks in in unit tests by default.
• Fix typos in README.md. (Thanks to Christian Clauss for the fix.)
• Fix typos in man pages.

New Contributors

• @cclauss made their first contribution in #235
• @cho-m made their first contribution in #245

Full Changelog: v0.10.1...v0.10.2

SHA-256 Checksums

2f4d275d006228acb2280c3bf5327f02098c2ba110d81fe3854a80f5fd848058  dwarfs-0.10.2-Linux-aarch64-clang-reldbg-stacktrace.tar.xz
75878252ef0bfc490e5bd6ad5870bc5a02531650ceacf1258807e09606069561  dwarfs-0.10.2-Linux-aarch64-clang.tar.xz
74b52460ebd2d8e752ad7fbe976c683be542a8a581fdf25ac59ba1dea5bc5d0c  dwarfs-0.10.2-Linux-x86_64-clang-reldbg-stacktrace.tar.xz
a018bfe2531763a273a2d78bc507b1c89fe58a44f7955c980c854a55f9adbaea  dwarfs-0.10.2-Linux-x86_64-clang.tar.xz
36767290a39f92782e41daaa3eb45e39550ad1a4294a6d8365bc0f456f75f00c  dwarfs-0.10.2.tar.xz
c15280d920b67b51b42117612bd8a959eb5ca9ed0202fd765e19743aad61a728  dwarfs-0.10.2-Windows-AMD64.7z
36f72f1ff049a1d955e68547540b932539beab44b0cba23efbdb7a1b0bfd32d4  dwarfs-universal-0.10.2-Linux-aarch64-clang
4d55e783e352a5aafc321f7ac36964b0493601320d3d93d021634e78e743505d  dwarfs-universal-0.10.2-Linux-aarch64-clang-reldbg-stacktrace
b565399a0a671d06be3e078376e02b388ee14133680b8d19483fc93c294b12d2  dwarfs-universal-0.10.2-Linux-x86_64-clang
cb374fc2d64bbf3bd4dd4714f1be37e3d6fc6ecffc7afd93714b6897e9d3751a  dwarfs-universal-0.10.2-Linux-x86_64-clang-reldbg-stacktrace
eb69b1bf4703d28bd3d5f477dca1ab3460dda4250c7ce1899eb4192c2c1bef69  dwarfs-universal-0.10.2-Windows-AMD64.exe

dwarfs-0.10.1

Bugfixes

• Allow building utils_test against a non-compatible, system-installed version of gtest. This is a common issue when trying to integrate dwarfs into a package manager, as these generally disallow fetching external dependencies at build time.
• dwarfsck was always reporting a block size of 1 byte rather than the actual block size of the image.
• DWARFS_HAVE_LIBBROTLI was not set correctly in the config file, causing build errors if the library was built without brotli.
• Several small fixes for building with Homebrew.

Full Changelog: v0.10.0...v0.10.1

SHA-256 Checksums

53bb766f3a22f019c4bac7cbf995376cb4f3f0ad69e4793471af11c954185227  dwarfs-0.10.1-Linux-aarch64-clang-stacktrace.tar.xz
f272f667649d71ec7d29d6822ad4198e13a33e997d722f74f2bca23b239de72f  dwarfs-0.10.1-Linux-aarch64-clang.tar.xz
671ce264938ab4cacc8af0aabcacb1ecfffa01284b4959441e921264ae19b47e  dwarfs-0.10.1-Linux-x86_64-clang-stacktrace.tar.xz
84894bf6a26cac2eb2c8d43d6fccf1ece7665c4c15050cec494d09199bd8310e  dwarfs-0.10.1-Linux-x86_64-clang.tar.xz
4041ed9aa19e03f44dbe69b470f31423a3c358bcd07e78230311b859629785b6  dwarfs-0.10.1-Windows-AMD64.7z
db785e0e0f257fa4363d90153db34127add4552791a72998b30ded787840d039  dwarfs-0.10.1.tar.xz
3e003c9a5fbf31b75548c11a2c2c1958f606ce2c2022db4baa6d62b80201c76d  dwarfs-universal-0.10.1-Linux-aarch64-clang
44ad0a3f2d89e373b0279d1db7c19aeca46879972a2db226e31ec7ebe8ff103e  dwarfs-universal-0.10.1-Linux-aarch64-clang-stacktrace
18f99297c7425bd1bea87d47a2046bfc7e00fe9cc566f024e631ed39a6bb1913  dwarfs-universal-0.10.1-Linux-x86_64-clang
c60821be4a248be2feb54905b5bb6c5cd323014bcb7107f0d586ba7f70deb488  dwarfs-universal-0.10.1-Linux-x86_64-clang-stacktrace
768d013d55cd030c1fbabd35ad522648581c79435da4994cc39de75b3a7eda30  dwarfs-universal-0.10.1-Windows-AMD64.exe

dwarfs-0.10.0

This release doesn't bring a lot of new features or bugfixes to the command-line tools. However, large parts of the code base have been refactored (in more than 300 commits since the last release) and a couple of long-standing issues have been resolved:

• It is finally possible to properly install the libraries that implement most of DwarFS. These libraries come with all necessary headers and a CMake config script so you can start building your own tools.

• Shared library builds are now explicitly supported. These have caused some trouble in the past, but the problematic code has been refactored to avoid issue like missing compression / categorizer code.

• The code now builds about twice as fast. This is due to shipping generated code in the source tarball and removing a lot of dependencies into the folly library.

• It is now possible to do modular builds, i.e. the library, tools and FUSE driver can be built independently.

Bugfixes

• Fixed a race condition identified by ThreadSanitizer in the root node name processing.

• The terminal abstraction code did not check any errors when trying to determine the terminal width, leading to a random terminal width value. This caused the manual page tests to occasionally crash.

• Fixed some flaky tests, e.g. unmounting the FUSE driver on macOS.

Features

• Two sets of universal binaries and binary tarballs are provided for Linux platforms: one without any debug symbols, the other with minimal debug symbols and support for stack traces. For the universal binary, only the version without debug symbols will be UPX-compressed, as the stack trace functionality doesn't work with a compressed binary.

• Symbolic links to the universal binary may now be suffixed with a version (i.e. any part of the name starting with - and followed by a digit will be ignored, e.g. the symlink could be mkdwarfs-0.10 and it would be treated as mkdwarfs).

• Introduced support for extended attributes on Windows, including a new utility for cross-platform xattr manipulation (pxattr, for portable xattr).

• Enhanced file system API, adding error-code based and exception-safe versions for getattr, access, and similar functions.

• Filter rules now consistently use Unix path separators, even for the root path component. Addresses a comment in github discussion #228.

Improvements

• Extensive refactoring to improve code modularity, maintainability and to provide proper libraries. The library code has been moved to different namespaces to make it easier to understand the role of different components (e.g. reader, writer, extractor).

• Replaced all folly library dependencies in the public DwarFS library interface with alternatives from libraries like e.g. boost or nlohmann::json which are more broadly available. folly and fbthrift are still used as implementation details, but no longer leak into the public library interfaces.

• A much smaller subset of folly is now used in DwarFS and only the necessary components are built, significantly reducing the number of compilation units when building DwarFS.

• It is now possible to do modular builds in addition to the default monolithic build, i.e. you can build and install just the DwarFS libraries and later build/install the tools (mkdwarfs, ...) and/or the FUSE driver against these libraries. This is particularly useful for packaging (e.g. in Homebrew, which has removed all FUSE support from the core formulae).

• Shared library builds are now explicitly supported. This fixes issues such as github #184.

• The source tarball now contains all auto-generated code, e.g. manual pages or generated thrift code. This reduces the number of build-time dependencies (e.g. ronn or mistletoe are no longer required) and significantly reduces the build steps (it is no longer necessary to build the thrift compiler). The build is now roughly twice as fast as in the 0.9.x releases.

• The parallel-hashmap, xxHash and zstd submodules have been removed from the git repo and are no longer added to the source tarball. Both xxHash and zstd are now widely available. If a suitable version of parallel-hashmap is found on the system, this will be used, otherwise it will be fetched during the build. Being a header-only library and only used internally, there's no need for it to be installed.

• A lot of GCC warnings have been fixed and upstreamed to folly / fbthrift.

New Contributors

• @mindfocus made their first contribution in #222

Full Changelog: v0.9.10...v0.10.0

SHA-256 Checksums

52045506839249823a9b05b711f89e3fd18a729b264149fbe74f557bd8f16b81  dwarfs-0.10.0-Linux-aarch64-clang-stacktrace.tar.xz
b12a33650694baa6f546ac18768b8a7b2e1dd90c1c674d26bec7f5149a285537  dwarfs-0.10.0-Linux-aarch64-clang.tar.xz
8593eaecc1bed4f570626ff07b0e56a7f6a025c3a5849f4b0056006ab9801393  dwarfs-0.10.0-Linux-x86_64-clang-stacktrace.tar.xz
53f02da7ff7fe484b3bb91e6532987c3fb59db28131309ccbb46d2c2c9cec9ce  dwarfs-0.10.0-Linux-x86_64-clang.tar.xz
f24ee132cf0c77b0c94e49b3d8108fdf7e0ff965031bbc34aa51e3c2e066fa43  dwarfs-0.10.0-Windows-AMD64.7z
c01ae59d4662e4f027a7c8a5934b7aebe6edc4f7affd836fa73e6d861d18bf35  dwarfs-0.10.0.tar.xz
573656e33a171017046a32500b9ebe50ebd117df16212e6ef8f8d4b034618210  dwarfs-system-gtest.patch
149c9518d5229bf6dd07b5c39df02ed5cbc4bddbf9bf54f6fa5617d76a594b1c  dwarfs-universal-0.10.0-Linux-aarch64-clang
bfaad42a15c687fdaf502283792322692e479c563c56e8734efd9a298956d716  dwarfs-universal-0.10.0-Linux-aarch64-clang-stacktrace
5f3401cdd7267b6eec6f758f902d7d874534cc77b5be7233944832992e2c0081  dwarfs-universal-0.10.0-Linux-x86_64-clang
307f973fd7da525c5f06f6090b4a4f7a0bea041d785dcb718fb226f6b15f942f  dwarfs-universal-0.10.0-Linux-x86_64-clang-stacktrace
99cf71972156ec6f5d88ab0331709ecdc1e80f571582fce95c89b694d3be258b  dwarfs-universal-0.10.0-Windows-AMD64.exe

dwarfs-0.9.10

Bugfixes

• When cloning LZMA compressor objects, the LZMA filter options of the cloned instance would still point to an options object in the original instance. This could lead to LZMA errors when initializing a new compressor. Fixes github #224.
• Fetch range-v3 if no suitable version is found. Fixes github #221.
• Filter rules did not work correctly when input is root dir
• duf reports odd sizes due to using bsize instead of frsize