dwarfs-0.15.3

Bug fixes

• The 0.15.2 release did not include the legacy FUSE v2 driver binaries (dwarfs2) in the static release tarballs.

• The op_readlink implementation is now guaranteed to terminate the returned string, as required by the FUSE specification.

Build

• Warning supressions for (newer) GCC warnings triggered warnings on older GCC versions. These have been fixed.

Other

• Some automatic code refactoring from clang-tidy was applied and some minor issues were fixed in the process.

Full Changelog: v0.15.2...v0.15.3

SHA-256 Checksums

70edc89104233aa8afe7548c4b83c97625baf44c34f625fa99deff4b4b01f89f  dwarfs-0.15.3-Linux-aarch64.tar.xz
40c97466c7f8bd6c031ddc3adf3b5a4c90efbccbf1b734e24bf00c8cbf7f8413  dwarfs-0.15.3-Linux-arm.tar.xz
07eb788cfc9cee51e80f319a95591d92510211ac12fcc08e58231903f5e9d835  dwarfs-0.15.3-Linux-i386.tar.xz
10dd05063a8ae4732f49dbb61abb9dbd1ff9d759a9f8ff88a39ecd4251b12660  dwarfs-0.15.3-Linux-loongarch64.tar.xz
b975fa2e0388ccd81b586b766cf30d4c7550a740013394f1a3eb60b66d754036  dwarfs-0.15.3-Linux-ppc64le.tar.xz
e90b19f10fcb7b9c3f0951032f878ed1501d853b393f0cbf2c47f310c9221442  dwarfs-0.15.3-Linux-ppc64.tar.xz
bd53abaf33983401635fc1d1e27565b9dd7aa3f21aa1dd8168115608658fa951  dwarfs-0.15.3-Linux-riscv64.tar.xz
6cc901e63f7f7c7f31f8b11e31a9390f0f03a2199efebefdee4d05c7bd45935c  dwarfs-0.15.3-Linux-s390x.tar.xz
2dba61946786ec563c96783f719316199eeb55a4cbde1436e1b5796f28494fa8  dwarfs-0.15.3-Linux-x86_64.tar.xz
2a9c6b7cb2841f3c7b75839da9326724a2817e4467b20e79e3e24c3eefc13eca  dwarfs-0.15.3.tar.xz
24b3654c6353c4253605d13361c3187b08d38c5844ea3e67145ad0f694a4bdc8  dwarfs-0.15.3-Windows-AMD64.7z
e51b8658ffdfacf8fbe232c4425c58a3cc22e4ad8802494d1e6e940106758222  dwarfs-fuse-extract-0.15.3-Linux-aarch64
9b1f82d9b9fcf93909ea0fc97ad0439f9521120d145d6a43b2ef759511ae259d  dwarfs-fuse-extract-0.15.3-Linux-aarch64.upx
a5c9e9ab2f1b8446f3162569dd5b6738643f117391a729456cc244cc91dae6ce  dwarfs-fuse-extract-0.15.3-Linux-arm
8fbc03a7aa43b3e7a5bce862e41407feb349018a22941db4819cf6fd21e53571  dwarfs-fuse-extract-0.15.3-Linux-arm.upx
7a733919ad311d9c2418e4da08905d61a78d55518998531273b4375ec213a82f  dwarfs-fuse-extract-0.15.3-Linux-i386
e8d79bb1e9f56b7811ed9288449f724700b8bf7fc49063a6e1e42e07cd272849  dwarfs-fuse-extract-0.15.3-Linux-i386.upx
f1ea878bb3e01bb61ad998d3a08f15d4ac43a43a03a877a383697612f50189c9  dwarfs-fuse-extract-0.15.3-Linux-loongarch64
2594e3d37a3cf3cf278e74a04e08a343c3c939f7545e01d66e4054e2af916447  dwarfs-fuse-extract-0.15.3-Linux-ppc64
87a9fafe6b926668acaa63bf4a6542375c5c8e330426d2992b37785a21c6642e  dwarfs-fuse-extract-0.15.3-Linux-ppc64le
a4ef6a576e27486123388c5a4426d84621f911890ebe1d0039c22b4910f3097b  dwarfs-fuse-extract-0.15.3-Linux-riscv64
4f095dbc965f875645ac99c09b69254a88dabb6f767581c4463799879ee1046d  dwarfs-fuse-extract-0.15.3-Linux-riscv64.upx
374e2a4153800818017534bbb4ed28266ebef4b7912e18af0a6084b6afaf9cc3  dwarfs-fuse-extract-0.15.3-Linux-s390x
6b46fcef644fbc88a8e8235f9aece0be1940d568daa631ff0f71c6c2787f0dfe  dwarfs-fuse-extract-0.15.3-Linux-x86_64
9a17a645c121340b687f3e1ea1f1d350ac4b4de390ee0e75ec82ccd2f95cc963  dwarfs-fuse-extract-0.15.3-Linux-x86_64.upx
87a514821c762371a50dac7a271a6c78af3a23cc0c7cc47571713f8fb6707684  dwarfs-universal-0.15.3-Linux-aarch64
daa98361d4671f2afb644c59065632e8e1bc8f033b0692478e5039852d5d39db  dwarfs-universal-0.15.3-Linux-aarch64.upx
88692a6670bec935eaee69d61a9066f3d0e13591229c784b45e056760bc7c9ca  dwarfs-universal-0.15.3-Linux-arm
c62d47af8becf42bae630839d3d04736b56322f1a6d4fc341ad33ea6cd8a28f2  dwarfs-universal-0.15.3-Linux-arm.upx
abf7897855c5d2d84beb11f845630ac14365de5793ebc605679314c7c375a9ad  dwarfs-universal-0.15.3-Linux-i386
42eabcba8ff64c56931d63a7824d59dd58ad86ee603bbad0944fbcb501f0a78d  dwarfs-universal-0.15.3-Linux-i386.upx
ca2723e432f6b751157c62ef5f6bb4e86e9465f327a1b786ff3f6edbf7bf01c5  dwarfs-universal-0.15.3-Linux-loongarch64
691915391b8e77dd0bd3993f70d0198b2abdef69350d850781d0679a9612d8ce  dwarfs-universal-0.15.3-Linux-ppc64
a37f7f25f0acb898971f0911f655cbe432673a26c340d3c5b3cbfd43e569ebcf  dwarfs-universal-0.15.3-Linux-ppc64le
ad30fdecbc301ae10c14ee1141806d16dabd14400c32f980bda4c5643b78049c  dwarfs-universal-0.15.3-Linux-riscv64
404b334707b865cd5ee6bc8749af335301afe94da2fb2524e46ab6613b783f97  dwarfs-universal-0.15.3-Linux-riscv64.upx
393257fb56de88d219e79496c24f50f57a4dece910a255ed421b894b00c93804  dwarfs-universal-0.15.3-Linux-s390x
30c868737fdf7b4167b35650839bcbce0fd73d203f83702bc0afa7eaa2e1bded  dwarfs-universal-0.15.3-Linux-x86_64
f2d2c33c7f22995abd45601dbd61fdee673727d67069c443828fecd6704c7e7c  dwarfs-universal-0.15.3-Linux-x86_64.upx
0ce305dc4c593f9f30098fdc3aec6367159b8f2f25627cda19c794a9c1378b34  dwarfs-universal-0.15.3-Windows-AMD64.exe
4c0c1ac1bb786e62658bf06502c710047ae2c7a670c707abbb92918caef13de4  dwarfs-universal-small-0.15.3-Linux-aarch64
11cecac8b833dc15c1cf18f148400a0608ff1d56d608f1f8a8466ffc584ee66e  dwarfs-universal-small-0.15.3-Linux-aarch64.upx
064a3f389fe7c815449406f1beefea914ec7ce554171fbc27110ceed87518026  dwarfs-universal-small-0.15.3-Linux-arm
a5f000b76db896cc1cdcd4bf0b588602c6da3238ac5a96581671bce18db525dc  dwarfs-universal-small-0.15.3-Linux-arm.upx
c34ceb2ea5e662d46a3fc8220cec23a2b8f8dee5ac000125e00c30f69a235644  dwarfs-universal-small-0.15.3-Linux-i386
242176ab5768625dc59154445125d41b95cbf8459db890da77c8923e4544199a  dwarfs-universal-small-0.15.3-Linux-i386.upx
98c3d497e030cd5d8bad7525650875417cbc00a911d2b813e754cc440f09676b  dwarfs-universal-small-0.15.3-Linux-loongarch64
ad942bbac4235d3a2ac1e1f54e0b3e8e97630726595279df0687bffaa64300f0  dwarfs-universal-small-0.15.3-Linux-ppc64
b1341c2d3469a96c5c8f4404145bb9af986b8f6a0d8b5f168d0ec528c872c249  dwarfs-universal-small-0.15.3-Linux-ppc64le
e7d05a966ad8a9e469f01b8281ba9680c9fa85912bb5881d50567f935f001602  dwarfs-universal-small-0.15.3-Linux-riscv64
9f3afa8633708a3dbcf6c06b34dd75a6b06c46ac703669c8926154b6eb60a9f2  dwarfs-universal-small-0.15.3-Linux-riscv64.upx
4de99aed5e01bd7b85aac72414e61848917e878103a28d76a7f8caabf013d7d0  dwarfs-universal-small-0.15.3-Linux-s390x
16fc1f7bc8d6034a2101323e31b3e18aebf91ed0b8eb41a10c1d4886968aa25c  dwarfs-universal-small-0.15.3-Linux-x86_64
1b7b64d88a361a4f21679366c802513a927081be263bd87aa805c65c1a20d52b  dwarfs-universal-small-0.15.3-Linux-x86_64.upx

dwarfs-0.15.2

32-bit glibc Build Fixes and FUSE Driver Cleanup

32-bit native build fixes (#354)

This release fixes a set of issues that showed up in native 32-bit builds, most notably on openSUSE Tumbleweed with glibc. The most important one was related to sparse file tests that create files larger than 4 GiB: glibc requires _FILE_OFFSET_BITS=64 for 64-bit file operations in that environment, and without it those tests would fail. This was not just a test bug, all the binaries simply could not deal with files larger than 4 GiB in this scenario.

However, this was only an issue for glibc-based 32-bit builds. It did not affect musl-libc-based builds, which is why it went unnoticed for some time. This did not affect the statically linked release binaries, which are using musl-libc.

A few additional 32-bit related issues were also fixed: the formatting code for times and ratios relied too heavily on floating-point arithmetic and turned out not to be deterministic across platforms and compilers. That code has now been rewritten to use integer arithmetic instead, avoiding platform-specific behavior. A bug in the test code itself, which surfaced only with GCC, has been fixed as well.

FUSE driver refactoring

The FUSE drivers (dwarfs, dwarfs2) have gone through a substantial internal cleanup and refactoring pass in this release. This fixes a number of subtle startup and option-handling issues and makes the behavior of the different driver variants much more consistent.

One visible improvement is error reporting for invalid options. For example, if -o image_size=1234 was used instead of the correct -o imagesize=1234, older versions would end up reporting a misleading filesystem loading failure rather than flagging the unknown option directly. That kind of behavior has now been fixed.

Internally, this also removes a great deal of preprocessor-heavy startup logic that had diverged over time across the various FUSE implementations and modes (high-level, low-level, Windows, FUSE v2, and FUSE v3). The resulting code is much easier to follow, more consistent across platforms, and better covered by tests.

---

Bug fixes

• The image size was not passed correctly to one instance of the filesystem parser, which caused errors when loading a DwarFS image embedded inside a larger file (for example, a multi-layer file). Thanks to Ruan Formigoni for the pull request fixing this issue.

• The performance monitor timer for op_lseek in the FUSE driver was not initialized correctly. This could lead to segmentation faults or bus errors due to an uninitialized index into a std::deque. The issue has been fixed, and an additional check has been added to catch similar errors in the future.

• Native 32-bit glibc builds could fail in tests involving sparse files larger than 4 GiB because _FILE_OFFSET_BITS=64 was required for 64-bit file operations. Additional non-deterministic test failures related to time and ratio formatting were also fixed by rewriting that code to use integer arithmetic instead of floating-point arithmetic. A separate test bug that surfaced only with GCC was fixed as well. Addresses #354.

• The FUSE drivers (dwarfs, dwarfs2) were refactored to eliminate a number of subtle startup and option-handling issues and to make behavior much more consistent across the various supported FUSE variants.

• The manual pages shown with --man for all tools unintentionally included the license header from XML comments at the top of the source files. The renderer now ignores those comments.

Build

• After benchmarking the latest mimalloc allocator, it turns out to perform mostly on par with jemalloc. It is still less configurable, but it is clearly usable when that extra configurability is not needed.

• The small universal release binaries are now built with mimalloc instead of jemalloc, reducing their size by about 10%.

• Older Clang versions, such as those shipped with Ubuntu 22.04, are no longer supported because they cannot use libstdc++'s std::expected implementation. DwarFS can still be built on Ubuntu 22.04 with GCC.

Test

• A few particularly slow tests were identified through profiling and have been reworked to run faster while still covering most of the same code paths.

New Contributors

• @ruanformigoni made their first contribution in #355

Full Changelog: v0.15.1...v0.15.2

SHA-256 Checksums

f81290d5c4890a274a60a82b801e62110975ca466799e4408ad046bc70695aef  dwarfs-0.15.2-Linux-aarch64.tar.xz
2466659a4e783d1bc794e17215dcdf093fae8e9493ffaa03091a85c8c11369f2  dwarfs-0.15.2-Linux-arm.tar.xz
fd6ed59b7805539c6fc6e6e508344d10479f6587abe1040a1e86997f19b52fa0  dwarfs-0.15.2-Linux-i386.tar.xz
62f5f2bac65232184cb14e7bb7921a38b34440dcd688080eebdcdc512915d3ba  dwarfs-0.15.2-Linux-loongarch64.tar.xz
08e556adeb248a20aec04bf70886146f5e56bd0efc0bbe225aae69d20b25d33e  dwarfs-0.15.2-Linux-ppc64le.tar.xz
1668eca05412c2bd848ad4aa606a61fb2bf48f3cf92aac556149778a713885f2  dwarfs-0.15.2-Linux-ppc64.tar.xz
3110c8f630a4ae65fe16e4e3280f74d34b2c7a8224630e03361ed799819eeb9f  dwarfs-0.15.2-Linux-riscv64.tar.xz
5879d6d1e406a5ddf4dc68d8179540bd797af5d58abe7def9e62c6841221eb95  dwarfs-0.15.2-Linux-s390x.tar.xz
ad9ef4edf2d124b5c8d4540b2298a9b520e15841c68b921192d9bcb83eb69455  dwarfs-0.15.2-Linux-x86_64.tar.xz
6b7edcb2121347e273753d949f72913f006ec12477248384b24c48989ec34995  dwarfs-0.15.2.tar.xz
c34c05e7f65edcfd5f21bc3fed2de90d3f4b8c5b311d3b5ae302ae3ec6fc7161  dwarfs-0.15.2-Windows-AMD64.7z
46d690ecf546539d3229df9e3ad2412a806e4dee95e22c9ea843d934003f21bb  dwarfs-fuse-extract-0.15.2-Linux-aarch64
3d49d59416bcb989504b70bc63ebcbbb5be5c9a049bd432cefeb4620aa0fc3ae  dwarfs-fuse-extract-0.15.2-Linux-aarch64.upx
d6b4f4fc498e48031f7e7bb5ed3297a1272a68465f1803ad599ee486b0d49799  dwarfs-fuse-extract-0.15.2-Linux-arm
c5214353ad9de53a7fbff6aa5adfa65a9b830b694be839d5de1dbec9ff6a12f0  dwarfs-fuse-extract-0.15.2-Linux-arm.upx
5056562a6e12bcb20e7e07aeff0fa31113bae52f6feefd1a79f87d412c8bcd6d  dwarfs-fuse-extract-0.15.2-Linux-i386
92ac91f59581663985250cc0bcad02742c5cf1a5b009925212f97c1a424f69c3  dwarfs-fuse-extract-0.15.2-Linux-i386.upx
ede37a302b6e038f25dc4a35f25d8c9378fb2ae35600ec445a1eec0bf8b9d64c  dwarfs-fuse-extract-0.15.2-Linux-loongarch64
eec1b96f2d19c04aeda2099914e0b905cec6dec98be439f1bfc4f8f31c447554  dwarfs-fuse-extract-0.15.2-Linux-ppc64
983ee30f114bbafdeefc7ca8a79eecffa33c6314b9c516e7183424ec1d75802c  dwarfs-fuse-extract-0.15.2-Linux-ppc64le
fba9d984f3258f5c35e441b2eb468e25baa9f339e6100013c5a2fbb1c6bfcc1e  dwarfs-fuse-extract-0.15.2-Linux-riscv64
a3210163095b118407f6b8bd174d60bdb92a9d3fa70b9cdcb761d39570bd2bbb  dwarfs-fuse-extract-0.15.2-Linux-riscv64.upx
28f9ca8b2c9ccfc3c7e5949ed89b33a24aade539b3e0de40d5d364e0d06b0065  dwarfs-fuse-extract-0.15.2-Linux-s390x
85084aae43406fa8892df1cc24bd6df96948d99e8a35ca4c2873518fef0fa41c  dwarfs-fuse-extract-0.15.2-Linux-x86_64
40d027c79c7be4a5f7701433477f832390c3aaab437ca3bb2e3f0aa0a007d557  dwarfs-fuse-extract-0.15.2-Linux-x86_64.upx
b9c25399e1e4672601c0afbcb44e522af71698ea41691384555abe09776c113e  dwarfs-universal-0.15.2-Linux-aarch64
432a460080765f52b2b6e51ef99ca87bc4ba1cdf8fa9257155cd89d1b20edbc8  dwarfs-universal-0.15.2-Linux-aarch64.upx
114496a88f27af058481d09e4c75a5079ff9567317745fd50ca4ae79eec30070  dwarfs-universal-0.15.2-Linux-arm
23703b2ce7de3a3ed52f1441759f91a10ab654243678a050e98e5fda0d821c05  dwarfs-universal-0.15.2-Linux-arm.upx
35771934f6dc34038624cf99179bcfb4fe707c533e294d4a75cebdc56785cf08  dwarfs-universal-0.15.2-Linux-i386
24d6ae52dc7166d667cabb47b77233f3a2e87848fb7ffc71a1d230aeb54dd5f7  dwarfs-universal-0.15.2-Linux-i386.upx
a3d97d40a2cc1baf70efe5e5509f9537826b13ee27bdf3bf95a2914361d31181  dwarfs-universal-0.15.2-Linux-loongarch64
547650e46dde48abdd5824115e3c39cb66d551e4647383716591ea5bbcfb7f2e  dwarfs-universal-0.15.2-Linux-ppc64
9712e02402f2a2094a565d39ec2bad2a3a43dbc042a18c93035726de4b133c24  dwarfs-universal-0.15.2-Linux-ppc64le
24f53482d4992797b178e9d63156519569168d2ad469c998451670c44362c418  dwarfs-universal-0.15.2-Linux-riscv64
912b7c0205df9c3a3af61e9b5816fd7b83219259db3ecbe9d0761be97335d0c1  dwarfs-universal-0.15.2-Linux-riscv64.upx
13719648b094adf2c86df6dcc1c34f6b414d01f2032ab1c035bcc95e97f7d135  dwarfs-universal-0.15.2-Linux-s390x
7e4d7ed1b4c8d3f1db01d6914e5b8bcba1d1eb69dfaafe06d32214797fb0dd55  dwarfs-universal-0.15.2-Linux-x86_64
3f073cf225f975415da316aacee1825497a981ff7dad654bf50adb9b2592f7bb  dwarfs-universal-0.15.2-Linux-x86_64.upx
af985a17f080dd708f4fe8c6993adfa0a9f2545f96322e86dd4464ec2bcc0838  dwarfs-universal-0.15.2-Windows-AMD64.exe
feb6e82cd37eba74540c20ab4b73f82c5698f37bada15c7869ab976e04792c46  dwarfs-universal-small-0.15.2-Linux-aarch64
622c917795af04b045d070846ef8492f1c033f5722aa2c02549e218bcaaeace3  dwarfs-universal-small-0.15.2-Linux-aarch64.upx
759c66628b384769e9a3bcd5f98feadec3faa5334910e865c57551a44258f817  dwarfs-universal-small-0.15.2-Linux-arm
2b2fb1d586d79eff8024f69ffc805a341323358ef96ea7cbaf06b41fb76e9628  dwarfs-universal-small-0.15.2-Linux-arm.upx
041cd1200eeac5cf6bdd0c8dc922d119390959a4881b70eddf82d7ac79ae8d89  dwarfs-universal-small-0.15.2-Linux-i386
1dee9fc1846b902f544d8fcfaf7299b0381fa5d8345c2e9eb5d581ce73350b86  dwarfs-universal-small-0.15.2-Linux-i386.upx
c41cc39c690bba884077f995233eeeef968e6de20b2814fc4c1836281892ce70  dwarfs-universal-small-0.15.2-Linux-loongarch64
9f45a53288ac79192454aa5c96d90c92d47206a6162516b5d8f7efa6b85128a4  dwarfs-universal-small-0.15.2-Linux-ppc64
8a6d16cc922ba48b17c78964388cd04f5fa39593340a1721453ed3ebc766862c  dwarfs-universal-small-0.15.2-Linux-ppc64le
5f05442e1426ce962a0f319f0e27167943024758f5dd508053683a8cefa261a7  dwarfs-universal-small-0.15.2-Linux-riscv64
b84b860adee190cadd0d54a643d9eb787366320371d67944e3e0aff035a4acb5  dwarfs-universal-small-0.15.2-L...

dwarfs-0.15.1

Serious Bug in All Previous Releases

In #350, cipriancraciun started a discussion that got me thinking about the file scanner class. At some point, I realized that there might be a bug, and, after looking at the code, it turned out that there was one: when collecting hard-linked files, the class did not take the device these files were on into account.

When --file-hash is set to anything except none, the issue is triggered if and only if all of the following are true:

1. The input to mkdwarfs spans more than one device (i.e. mount point).
2. There are regular files with the same inode number on more than one of these devices, and these files are part of the input to mkdwarfs (e.g. not filtered out).
3. At least two files in such a set of files with identical inode numbers also have a hard link count greater than 1 on their respective devices.

For any set of files with identical inode numbers and hard link counts greater than 1 for which all of the above conditions are true, only one inode will be chosen to represent all files. That means the data for all other inodes in the set is lost and will not be present in the resulting DwarFS image.

When --file-hash is set to none, the issue is triggered regardless of condition (3) above. In this case, however, mkdwarfs is guaranteed to crash with an assertion if it runs into the issue:

$ mkdir data && echo "hello" >data/x
$ mkdwarfs -i data -o data.dwarfs
$ mkdir -p mnt/a mnt/b
$ dwarfs data.dwarfs mnt/a && dwarfs data.dwarfs mnt/b
$ mkdwarfs --file-hash=none -i mnt -o /dev/null --force
[...]
Assertion `!files_.empty()` failed in /home/mhx/dwarfs/src/writer/internal/inode_manager.cpp(212): inode has no file (any)

So the default, hash-based deduplication mode is much more dangerous because it fails silently, but I hope that condition (3) is rarely true in practice.

The fix is 3c15ab2, along with a dedicated test. I strongly recommend upgrading to this new release if your input to mkdwarfs spans multiple devices.

---

Bug fixes

• mkdwarfs did not correctly handle inputs where hardlinks had the same inode number on different devices. To run into this issue, you would have to make mkdwarfs scan files from multiple devices (e.g. the root of a directory tree with multiple mounted filesystems) and have files with the same inode number on different devices and have at least two of those files also have a link count greater than 1. While this is hopefully rare in practice, it is a serious bug that can lead to crashes (in the best case) or even data loss (in the worst case), as only the data of one of these files would be stored in the image. This has been fixed and a test has been added to cover this case.

• A missing dependency was causing linker errors with shared library builds on macOS. This has been fixed.

Build

• The static release binaries are now all built using Clang and link-time optimization. This was previously not the case for some architectures due to bugs in the toolchain. As a result, the binaries are now significantly smaller.

• There is now a new set of binaries (dwarfs-universal-small) that are built without brotli support and without support for the performance monitor. The performance monitor is rarely used and brotli compression comes with a huge dictionary that bloats the binary size without offering much benefit over lzma or zstd in most cases. If you care about binary size, these new binaries are a good default choice.

---

Full Changelog: v0.15.0...v0.15.1

SHA-256 Checksums

ff5ef1716dec13082356a23ca8b9a349d00e8af71712cd659d95195202838e5d  dwarfs-0.15.1-Linux-aarch64.tar.xz
1d017f5da0a92f61d8620c45670cd799e0bd452a8c1f31080cea554bce880dda  dwarfs-0.15.1-Linux-arm.tar.xz
c91bfe1eb348a8a34581d6377a49c937f18a96a8d1460241fde7f79fe7d3cd47  dwarfs-0.15.1-Linux-i386.tar.xz
eaead2ac3c61c6765b80fe57e6c870c7e9f14b83c4947a533aeda0d3720aac7a  dwarfs-0.15.1-Linux-loongarch64.tar.xz
c70c58a47a81b51bb61fff6f878d9b9e984ac944b704c713b126aaabae9429f6  dwarfs-0.15.1-Linux-ppc64le.tar.xz
0a11aff785ae7ebed0aa2f9a0f12e85cb14336901937387e45c4ecc94399b7b6  dwarfs-0.15.1-Linux-ppc64.tar.xz
7d7af8685ac2527760fa4ca848ded3780a6fc10760242ac789094cd9ee612153  dwarfs-0.15.1-Linux-riscv64.tar.xz
f5b9c87e4471fe658690951861949b8fe9bf8ddae0fde36a04dc672cc4926568  dwarfs-0.15.1-Linux-s390x.tar.xz
24453ca3f18e08cde0e323fae4447d2e9f47c65508d8da223d22553e73cafc36  dwarfs-0.15.1-Linux-x86_64.tar.xz
a180086f9a898b4b52a5217e336c0134d63a10b395b493f2e19f231d575a87ec  dwarfs-0.15.1.tar.xz
b3d3ff5608766f05a37b9a9d1f3cc3cac7ca01959ac388d43140136c858d54b7  dwarfs-0.15.1-Windows-AMD64.7z
1258d788d9950de4db55e22b5ec510665eb3049f8c199386fce16a132ab6f846  dwarfs-fuse-extract-0.15.1-Linux-aarch64
8f8f958217180f1d49fd775a84fb3f2df4a6c7d42b9537f7258c155c805f5ece  dwarfs-fuse-extract-0.15.1-Linux-aarch64.upx
814204960bdd739da73c965288f4d86dab91620241a05cc99ec84be8c0377ffd  dwarfs-fuse-extract-0.15.1-Linux-arm
5ea3eb07bc38d5a9cad9a296b83a6049df80c9508888341fb74b9cbb1695a750  dwarfs-fuse-extract-0.15.1-Linux-arm.upx
b7974ad380b1dac65516cd06fa1f0918708e212cb89fb6e79a79ca133e227097  dwarfs-fuse-extract-0.15.1-Linux-i386
60e448f27084f30727ae64346d5ae2c2e13cff28445e0c5b5f85a7666cfcc325  dwarfs-fuse-extract-0.15.1-Linux-i386.upx
cfd53bd6d61474cf4adf32cfbdf937717c9c329f5fd833ee997a8cd50afff74d  dwarfs-fuse-extract-0.15.1-Linux-loongarch64
925433bac0999babff000767e1a3bc5f33cfc93bc4d4f5344e9c160e0fe04e0a  dwarfs-fuse-extract-0.15.1-Linux-ppc64
7574dd2f519cc76c39a67d7a62fdb700a264be56f379df893872f7c864c7b5da  dwarfs-fuse-extract-0.15.1-Linux-ppc64le
ccec93d4277e82ac44b78116e27c3df68b376e1ed3588c60d57a94ba5234d8ff  dwarfs-fuse-extract-0.15.1-Linux-riscv64
7cae9c8f7b0ea283f7ac1f70fa8fdcd6fd50f34597536211ca854c78c6088d5a  dwarfs-fuse-extract-0.15.1-Linux-riscv64.upx
afe1cfcc82314af7593054b81cee5191aa35a40a0b06bf6aab036e9d97e3da79  dwarfs-fuse-extract-0.15.1-Linux-s390x
0d0a771c7849f6f6e56cfba879ad03cf4194b57cb4fa94b4d34b797ebdd34a99  dwarfs-fuse-extract-0.15.1-Linux-x86_64
dce7fb26462f7ef0ba4c5aef5563e5e74ba23b880195ce37b4a0bd7ca46ff431  dwarfs-fuse-extract-0.15.1-Linux-x86_64.upx
36fd1012426446d85ab37503b1f8780b9c2e94e6368dac5cd3de1fd58c224cca  dwarfs-universal-0.15.1-Linux-aarch64
7927c4ba101efe64fc9015de0f4b6b0c4898b3e854e037670b9998556dc6406f  dwarfs-universal-0.15.1-Linux-aarch64.upx
29fcfc2bd09abe023ae8f17072b1a5967fc4536867b19fcbe79818f9f3dd9920  dwarfs-universal-0.15.1-Linux-arm
00f84a9a3e93102a880e641fcf0bda77216c8ef65b84e6239bbcdc4aa70271fa  dwarfs-universal-0.15.1-Linux-arm.upx
f59359c67e5e43e67a086ad020c66f08ceb87ded0ddd641e21d35fa3ffb323b5  dwarfs-universal-0.15.1-Linux-i386
dd8116704ea22628699a0500c1d4b1fb7ff81acd3f4fe2c836604c6f654ff1ef  dwarfs-universal-0.15.1-Linux-i386.upx
48624d95428286e8cac2801f23b62c000fcabedbc01bc91db734341fa621facb  dwarfs-universal-0.15.1-Linux-loongarch64
428565e0033be5fe47b97c4cb1ec60da4550ec76e388326250148ecefb04ae21  dwarfs-universal-0.15.1-Linux-ppc64
a9671a135f7362d42dd8479e02593a63ac7e34430c94b9257e1fb6f28bf506a2  dwarfs-universal-0.15.1-Linux-ppc64le
c8c2ef012b80e554b0052bf85fd2271b374abd4a82337777d75200ebf40ffc0d  dwarfs-universal-0.15.1-Linux-riscv64
459aa0cac669f9794f114dc868a628f8bebb8ca538d4213b7d25448e424db3ae  dwarfs-universal-0.15.1-Linux-riscv64.upx
ff7b663e44f1f6d5e5b1b022fce85a6c993991f9c99e3d0e1d89c45a42d25284  dwarfs-universal-0.15.1-Linux-s390x
69750c543ea2272d96397a14627f5bb98a42385026df118cb3a4363312892fa4  dwarfs-universal-0.15.1-Linux-x86_64
83dd7dad048a86347a53cfa62d1b2dc3765a0803681a3c701a32029b464f4873  dwarfs-universal-0.15.1-Linux-x86_64.upx
5a4da367840829874f89a80112e1f49b25393969121130dc489ed6b6e8e6b782  dwarfs-universal-0.15.1-Windows-AMD64.exe
fb9baa894d32d7182d07ce54c4623d27b2b0ce6a0c6b4f1f0101462877d4ab0e  dwarfs-universal-small-0.15.1-Linux-aarch64
accef8fb32026084953e9e2e0174950f9fa789e9d1ece3263bccc1db58dde6a6  dwarfs-universal-small-0.15.1-Linux-aarch64.upx
0499ffc9b7093bee499dd56845659590fdcbd32aaf3e0a5f4bb9a5db92dac60c  dwarfs-universal-small-0.15.1-Linux-arm
fcd1422cd2315a6dbab0526ebe0275db9713623bd64f8f5b7bf11437f2e74f19  dwarfs-universal-small-0.15.1-Linux-arm.upx
1f3faf769fefe6517e0086288e17a0e60780cfab63b527c8e9c0cc6f695ea1df  dwarfs-universal-small-0.15.1-Linux-i386
a0bad3f10125a08ac51b765371f1d82c14ff0738ddda76aa40597e64e8faf9c9  dwarfs-universal-small-0.15.1-Linux-i386.upx
eac51d831d84a89df62718ba7c3c55f4994d0c5d9387199e060d9c2242572eed  dwarfs-universal-small-0.15.1-Linux-loongarch64
588de2941e5353daa164afbb55f5b4245ef27d7b0a19a2302a76093ba0c68005  dwarfs-universal-small-0.15.1-Linux-ppc64
4db6b7fec01bbaef8d51c8850eaccccbb72723dea02d5966fc8cd5198cc9f316  dwarfs-universal-small-0.15.1-Linux-ppc64le
58cda2c96c82bae3fed2d461be37786093e8f68bba7573c57ac7a1fa45cedd33  dwarfs-universal-small-0.15.1-Linux-riscv64
18b2d1c203b07a5bee39453f200d79d2069ca607e8913e060befdd5456b9aa7a  dwarfs-universal-small-0.15.1-Linux-riscv64.upx
391e3e75743899a23fefe3b816cd2dda86f2048deedbc1490870a55e49cb6309  dwarfs-universal-small-0.15.1-Linux-s390x
0d84291b3e7e26f3791d50169097a7b0901542440ce934fb1ec263ae0c4a256e  dwarfs-universal-small-0.15.1-Linux-x86_64
fc607ba7af3485feb8b90fd4f4678bd586c587ce8dd982e57f4fd9afb1faf391  dwarfs-universal-small-0.15.1-Linux-x86_64.upx

dwarfs-0.15.0

Leaner Core, Hollow Filesystems and REUSE compliance

Major dependency reduction

This release removes the project's long-standing dependencies on folly and fbthrift, along with a number of transitive dependencies that came with them. DwarFS now uses a much lighter internal stack built around standard C++23 facilities, Boost, and a small set of in-repo components tailored specifically to what the project actually needs.

At the center of this change is a new internal thrift_lite library, which replaces fbthrift for the subset of functionality DwarFS uses: the thrift compiler, compact protocol support, JSON serialization, debug output, and frozen-layout support. The frozen library from fbthrift has been forked, cleaned up, and integrated as an internal component, with all folly dependencies removed. The frozen fork also opens up new possibilities for adding safety features that would have been hard to upstream.

All this significantly simplifies the build, reduces the dependency footprint, and removes several indirect dependencies as well, including gflags, glog, double-conversion, fast_float, and libevent. On macOS, it also eliminates the libsodium dependency. In addition to making the project easier to build and package, the new implementation also improves performance in some places: the refactored frozen code, while being fully compatible, is measurably faster than the original thanks to a new abstraction for accessing bit-packed data.

Compatibility: thrift_lite is only intended to be "compatible enough" with fbthrift for DwarFS' purposes. The compact protocol remains fully compatible, ensuring forwards- and backwards-compatibility of filesystem schema metadata. However, the debug and JSON output formats are no longer identical to the old fbthrift output. The debug output should look very familiar, but not byte-for-byte the same. The JSON output now follows a simpler style and represents maps as lists of pairs rather than JSON objects.

Hollow filesystem images

mkdwarfs now supports a new --hollow option for building "hollow" filesystem images. In this mode, DwarFS preserves the full directory structure, metadata, and file sizes of the input, but does not store the actual file contents. Instead, files are represented as empty sparse files of the same size, and reads from those files return zeroes.

This is useful for test scenarios where a realistic filesystem layout is needed, but the actual contents are irrelevant or would be wasteful to include.

New binary file categorizer

This release adds a new binary file categorizer, enabled with --categorize=binary. It can detect ELF, PE, and Mach-O executables and shared libraries, and group them by type and architecture.

This can significantly improve compression ratios when the input data contains binaries from different platforms and architectures. Even in less extreme cases, it can still help by separating binary and non-binary content into different streams.

REUSE compliance

The project is now compliant with the REUSE specification. All source files have been annotated with SPDX license identifiers, a REUSE.toml file has been added, and the full license texts are now included in the LICENSES directory.

This makes the licensing status of the repository much clearer and easier to verify automatically.

---

Bug fixes

• Commas in the filesystem image path were not escaped when passed to the FUSE driver as the fsname option. Because commas are used as FUSE argument separators, this could cause mounting to fail for paths containing commas. Fixes #323.

• Progress reporting in dwarfsextract was broken when extracting a subset of files using patterns, because it was computed relative to the total filesystem size rather than the total size of the selected files. Several subtle edge cases could also cause progress percentages to fail to reach 100% or even exceed it. These issues have been fixed. Fixes #316.

• Fixed FUSE argument vector initialization in dwarfs_main, which could trigger an assertion inside libfuse when extra arguments were added after an uninitialized vector was passed to FUSE_ARGS_INIT.

• The Windows build of mkdwarfs no longer aborts with a fatal error when it encounters an empty file during scanning.

• Fixed a metadata lookup bug for parent_dir_entry in filesystems with format version 2.2 and earlier (that is, DwarFS releases before v0.5.0), where an additional level of indirection was required but missing. Fortunately, this only affected the debug output of dwarfsck: the parent= field shown with -d directory_tree would display the parent inode number rather than the parent entry number. The only other code path using parent_dir_entry effectively compensated for the missing indirection.

• Recompressing a filesystem image with sparse files, without also rebuilding the metadata, could erroneously fail with an error claiming that sparse file support could not be disabled, even without --no-sparse-files. The root cause was an unchecked std::optional access. This has been fixed.

• When rewriting a filesystem image, the bytes_in and bytes_out progress counters were updated at different times, which could lead to incorrect compression ratios being shown during progress reporting. Both counters are now updated together after compression.

• When using --format=newc and extracting a subset of hardlinked files, dwarfsextract could crash with an "unexpected deferred entry" error. This was caused by a peculiarity of libarchive's newc implementation that was not handled correctly. The bug has been fixed and is now covered by a test.

• Corrected the license information in a few headers, changing them from GPL-3.0-or-later to MIT.

Features

• Major dependency reduction / de-Meta-ing the project. fbthrift and folly are no longer dependencies of DwarFS, and the corresponding submodules have been removed from the repository. fbthrift has been replaced by a new thrift_lite library that implements the subset DwarFS actually needs, including the thrift compiler, compact protocol support, JSON serialization, debug output, and frozen-layout support. The frozen library from fbthrift has been forked and is now maintained as an internal component with all folly dependencies removed. DwarFS now relies on standard C++23, Boost, and a few new in-repo components instead. This also removes several indirect dependencies (gflags, glog, double-conversion, libevent, and on macOS also libsodium). The resulting code is simpler, the dependency footprint is smaller, and binary size is reduced in many cases. The compact protocol remains fully compatible, but the debug and JSON output formats are no longer identical to fbthrift's output.

• mkdwarfs now automatically selects the progress display mode based on whether the output is connected to a terminal and whether the current locale uses UTF-8. Previously, the default was always unicode, which could produce garbled output in non-UTF-8 environments. Addresses #326.

• The project is now compliant with the REUSE specification. All source files now carry SPDX license identifiers, a REUSE.toml file has been added, and full license texts are included in the LICENSES directory.

• New --hollow option for mkdwarfs. This allows building hollow filesystem images that preserve the structure, metadata, and file sizes of the input while replacing actual file contents with zero-filled sparse files. This is useful for testing scenarios where realistic filesystem structure matters but the actual contents do not. Fixes #131.

• mkdwarfs now supports ZSTD long-distance matching (LDM) via a new long algorithm option for --compression. This can improve compression ratios with extremely large block sizes (typically above 128 MiB), or with smaller block sizes at lower compression levels. In addition, all ZSTD compression parameters can now be tuned when DwarFS is linked against a statically linked libzstd that does not expose the division-by-zero bug described in zstd issue #4590. The statically linked release binaries contain a patched libzstd and support tuning all parameters. Fixes #322.

• New binary file categorizer (--categorize=binary). This categorizer can identify ELF (Linux/FreeBSD), PE (Windows), and Mach-O (macOS) executables and shared libraries and group them into separate categories by type and architecture. This can dramatically improve compression ratios when binaries from different platforms and architectures are mixed together.

• dwarfsextract has a new --num-disk-writers option to run multiple writer threads in parallel when extracting files to disk. This can improve throughput, especially when extracting large numbers of small files.

• dwarfsextract has new --skip-devices and --skip-specials options to skip device nodes and special files (such as sockets and FIFOs) during extraction.

• dwarfsextract now emits a warning when a pattern is provided but no matching files are found.

• dwarfsck can now export metadata to stdout instead of to a file by using --export-metadata=-.

• With mkdwarfs in --input-list mode, specifying --order=none now preserves the exact order of entries given in the input file. Previously, none was not treated as a meaningful ordering guarantee in this mode.

• New --no-check option for mkdwarfs to skip the filesystem integrity check before recompression. This can speed up recompression workflows when the source image is assumed to be valid. The individual checks are still performed during the rewrite itself. Addresses #322.

• When recompressing a filesystem image, blocks that are uncompressed in the source image are no longer unnecessa...

dwarfs-0.14.1

Bug fixes

• The new Windows code from the v0.14.0 release that implemented lstat()-like functionality to get file status information did not work correctly for reparse points that were not symbolic links (i.e. "real" symlinks or junctions). When dealing with such reparse points, the code would incorrectly determine the allocated size of the file to be zero, since it tried to determine that size from the reparse point and not from the target. This caused the progress percentage to be calculated incorrectly, since much more data was processed than was expected, leading to progress percentages well over 100%. Thankfully, @lunighty not only reported this issue, but also provided significant context by mentioning that this was on a Windows Server NTFS volume with deduplication enabled. Fortunately, this was also reproducible on "regular" Windows using a mounted WIM image. The fix will now correctly handle reparse points by re-opening the handle without FILE_FLAG_OPEN_REPARSE_POINT in order to open the target. This bug only affected Windows builds of mkdwarfs, and only when actually used on filesystems with reparse points that were not symlinks. Even then, the resulting DwarFS image would still be correct, except for a way too small total_allocated_fs_size metadata entry. However, this entry is purely informational. Huge thanks to @lunighty for both reporting and helping to debug this issue! Fixes github #307.

• Related to the above, the metadata_builder now recomputes all total sizes (total_fs_size, total_allocated_fs_size, and total_hardlink_size) as part of the build() function. This not only ensures that the totals are correct even if the allocated size changes between scanning and segmenting (which has been happening at least on ZFS volumes), but it also allows images affected by the above bug to be fixed by rebuilding the metadata.

• Instead of making the FUSE drivers fail hard when seeing the options that were removed in v0.14.0, they now just log a warning and ignore them. The options may still be fully removed in a future release. Fixes github #303.

• The pcmaudio categorizer had two minor issues discovered by @lunighty when compressing a large number of WAV files. One was reporting an unsupported format: 3/0 or unsupported format: 65,534/3 warning, which isn't very useful for the end user. These format codes correspond to IEEE floating point formats, which are indeed unsupported. However, the format appears to be quite common, so the warning has been downgraded to an info message that explicitly mentions the floating point format. The second issue was an unexpected fmt chunk size of 20 bytes, which caused the file to be rejected as a PCM audio file (meaning it was added using a generic compressor instead of FLAC). It turns out that these non-conforming fmt chunks are also quite common in practice, so the code has been changed to accept the non-conforming file, but also logging an info message mentioning the non-conformance. Fixes github #309.

• A bunch of changes were made to make the static (Linux) release binaries work on FreeBSD with the Linux emulation layer. While you can easily build DwarFS on FreeBSD natively, the static binaries might be bundled in Linux application images that you may want to run on FreeBSD as well. This required patching musl to fall back to FUTEX_WAKE if FUTEX_REQUEUE is not supported (which is the case on FreeBSD's Linux emulation); otherwise, the binaries would regularly hang when trying to shut down thread pools. Furthermore, the SFX stub did not work with the emulation layer due to lack of support for fexecve; so it now uses a fallback of temporary file + execve in case create_memfd and/or fexecve are not supported. This fallback is very similar to the one that was already in use for old Linux kernels, only that cleanup of the temporary file is now done using a child "janitor" process as soon as the execve call succeeds. Lastly, all static binaries are now branded as using the Linux ABI rather than the generic "System V" in the ELF header, which makes it unnecessary to explicitly brandelf the binaries on FreeBSD. See VHSgunzo/uruntime#21 for more details.

• The test_helpers library used for unit tests was not explicitly listing xxhash as a dependency, which caused linker errors on macOS. These unfortunately only showed up in the Homebrew CI. The dependency has been added.

• The help text for the mkdwarfs compress level option (-l) was misleading in combination with the manual page as neither mentioned that the table with details was shown only by -H / --long-help. Fixes github #312.

Features

• Added shell completion for dwarfsck and dwarfsextract. (Thanks to Ahmad Khalifa for the contribution.)

• Added sample desktop unmount handlers. (Thanks to Ahmad Khalifa for the contribution.)

Documentation

• Updated diagram in dwarfs-format.md to show new inode metadata fields.

Build

• Bumped libarchive, libressl, and libdwarf versions in the static build pipeline.

Test

• Added test for file_stat on symbolic links containing multibyte characters on Windows.

• Added test for obsolete FUSE driver options being ignored and producing a warning.

• Added test for the FUSE driver's imagesize option.

• Added more tests for the --auto-mountpoint option in the FUSE driver.

---

Full Changelog: v0.14.0...v0.14.1

SHA-256 Checksums

8c17434c4fc8c33ba303e06eb62904047ee1aab3ea7bd98d2797b04d86b426eb  dwarfs-0.14.1-Linux-aarch64.tar.xz
98cdca2e9406cbb88093bd2ea03cbd8a62b7a3aa5ba0271d22934fe4e81128ac  dwarfs-0.14.1-Linux-arm.tar.xz
531b2c318a810920d4f11f0efac12cec06627068d91291232000a883108952ab  dwarfs-0.14.1-Linux-i386.tar.xz
93ec3d9d649cdb77eed3d1b5c6533ee05e24589be63cca3b0a9c32c18c55ef1a  dwarfs-0.14.1-Linux-loongarch64.tar.xz
813d021f3025968dc7fa0797553b55bd7a5ed1c85af1f6b2c36b2ad5897294c8  dwarfs-0.14.1-Linux-ppc64le.tar.xz
1437bd811bfcf0f9d7db080075906086625ab1465ed9a068bb881da1c8f68681  dwarfs-0.14.1-Linux-ppc64.tar.xz
0a6e1e66b0b0ce63acdc37a8a68ce173d37b135cfad9ca652ca2827e4789d3ec  dwarfs-0.14.1-Linux-riscv64.tar.xz
9f20a528d9757ae7825341438c7d38eb3c91953c34c8d6bb6ace5cc4fe3d2f52  dwarfs-0.14.1-Linux-s390x.tar.xz
7dba2bc4482defc5987652643ab0d58330b74e73be0f4d2d184879e7815c2476  dwarfs-0.14.1-Linux-x86_64.tar.xz
620cf27f2e142a5f8fc05552a70704c3bf4df23c3279c6026b3f37954d0529c5  dwarfs-0.14.1.tar.xz
66f4d28514eb9cd8a28f634d76f274eb899c461cd54775f74efb6a97239c04ec  dwarfs-0.14.1-Windows-AMD64.7z
24bdd5c6adea556ca3a28f26a3774684da7efe4a5e70a13ed7149edd12b3f738  dwarfs-fuse-extract-0.14.1-Linux-aarch64
9f04547e224aa97c3187474de4d49ee12cfc1a414b9a3864f4338272201278fd  dwarfs-fuse-extract-0.14.1-Linux-aarch64.upx
fa31490ac078cd90be86cb43cf12836ad142f115ff5bf4a01771b6e07b73acab  dwarfs-fuse-extract-0.14.1-Linux-arm
6e39a297c75ae2fbb69b2d6075b32e34b3c4a386d6961e4a85b76ff944ccf30b  dwarfs-fuse-extract-0.14.1-Linux-arm.upx
cd637922614ba20b4614c37e61d0bb5ee703ccfce13ba328f9c07ed2848d9424  dwarfs-fuse-extract-0.14.1-Linux-i386
b6274b71bd695d62de21c9631e6d9f62be8a55512e072f5c6124eaa047e1fdf3  dwarfs-fuse-extract-0.14.1-Linux-i386.upx
6b1b699aba544ef407ebc33003f900fa691685c2be4195ca9690da7d7da4bcef  dwarfs-fuse-extract-0.14.1-Linux-loongarch64
948227dc2f87c37a3c584af61b189cbc04557088530335794f54a8a9a17e113f  dwarfs-fuse-extract-0.14.1-Linux-ppc64
8a60b79fc535afeb3352c24252a69db8b0dae0e176a46cc321c0e4e644eb0e08  dwarfs-fuse-extract-0.14.1-Linux-ppc64le
4cb8a50135ad0d3f93f3750e738fcd0f92751445493665d7316e2384a20ba107  dwarfs-fuse-extract-0.14.1-Linux-riscv64
aebd9c079fdafc11e5669f1a2e49192f19e5c3f78d62f8c138eb28208ed1eb8d  dwarfs-fuse-extract-0.14.1-Linux-s390x
b121f472429bfba72b79f1b5de126e8a70872fedefd4d42aba98fc5fc402ae20  dwarfs-fuse-extract-0.14.1-Linux-x86_64
0ded4f3b5c324cd662babece00bf0434fd98b9c98e6268ea745e160bc96a500b  dwarfs-fuse-extract-0.14.1-Linux-x86_64.upx
3a47eda6d496ba12fb6d9af1b1914d4ba7ab73df8df3172ee3cad66c29822cc1  dwarfs-universal-0.14.1-Linux-aarch64
ad9d6e86c120c2a4e635e4843234d9e046ac186772ad719ba508c6f1051a5f57  dwarfs-universal-0.14.1-Linux-aarch64.upx
958ab2e0ae8b1c65b14090b1f56cf062becc955c8287d31f1f7026adad01ac7a  dwarfs-universal-0.14.1-Linux-arm
33c06c3320e9c9907f0803fba3913e59f39c2b07c2a54722a0c8fbd2ea20f542  dwarfs-universal-0.14.1-Linux-arm.upx
dae5171425af53ed1357d6f63c5fc45d6a61b0c70588c7cd8b5c8eb432080673  dwarfs-universal-0.14.1-Linux-i386
7a073913b6c682816e7dbcddde1bcb8ddca9d9c32f679369df21bb7d54f0291c  dwarfs-universal-0.14.1-Linux-i386.upx
17630a7f41d5974a0747f28052bf0339140b22ce67a66fce443ebcc1bc8594bd  dwarfs-universal-0.14.1-Linux-loongarch64
ca378379612b66093e677d45616f242bd08f5e5e808283f4055530c51048b149  dwarfs-universal-0.14.1-Linux-ppc64
06472149ea534d94eb5799f6a2037edc5ff5f3146546af15b3a9d5b427be9db6  dwarfs-universal-0.14.1-Linux-ppc64le
bf827447162016098ef193c6104264a52aaef9089bab434872631a8bf6ed2ffe  dwarfs-universal-0.14.1-Linux-riscv64
a89e4152096b422bda48cb9ec1a957dd9a4745ec9cd83f2d036d00e5e2dbb2ba  dwarfs-universal-0.14.1-Linux-s390x
f3a117fd6d5b7304944b199af7fdb8086a48c509ea2e9832255d8f9a54c98587  dwarfs-universal-0.14.1-Linux-x86_64
fb4ea4a267acf19a5bf9bf0eb13f3d7d88b5f7687fca2aae7136e21810fde0cd  dwarfs-universal-0.14.1-Linux-x86_64.upx
c59967d05cb97937c642964a258f29e437f68e2f156ff71b7205dd489b0f5c00  dwarfs-universal-0.14.1-Windows-AMD64.exe

dwarfs-0.14.0

New I/O Layer, Sparse Files and Sub-Second Timestamps

New I/O Layer

A completely new I/O layer abstraction replaces the ubiquitous use of memory mapped files across the DwarFS code base. Memory-mapping is still the default, but processing is done in "segments" rather than in whole files. This required a significant amount of changes (this release adds/touches more than 5,000 lines of code and almost 10,000 lines of new tests) in almost every part of the code that was processing file data, code that previously assumed any file could simply be accessed as a contiguous piece of memory.

In the new abstraction layer, backends are pluggable and configurable through the DWARFS_IOLAYER_OPTS environment variable, letting you:

• Configure the size up to which files are mapped "eagerly", i.e. as a whole and not in segments. This is mostly relevant for 32-bit systems, on which this is set to a reasonable default (32 MiB).
• Switch from mmap() to classic read() for maximum robustness on unreliable storage or faulty hardware.

The latter is relevant if you're seeing "bus errors" (SIGBUS), as many have done in the past (#45, #50, #108, #163, #213). You can switch to the read-based backend using:

$ export DWARFS_IOLAYER_OPTS=open_mode=read

Sparse File Support

This release includes end-to-end sparse file support:

• mkdwarfs detects holes in files and preserves sparseness in the image.
• The FUSE driver exposes sparsity via lseek() where supported (Linux, FreeBSD).
• dwarfsextract writes sparse files and preserves them when targeting archive formats that support holes (e.g., tar).

Compatibility: images containing sparse files require DwarFS ≥ 0.14.0. You can use --no-sparse-files to explicitly treat sparse files as non-sparse and keep compatibility with older versions. If your input does not contain sparse files, the images remain backwards-compatible even without that flag.

Sparse file support matrix

Feature / OS	Linux	FreeBSD	macOS	Windows
Reading sparse files (mkdwarfs)	✅	✅	✅	✅
Writing sparse files (dwarfsextract)	✅	✅	✅	✅
Exposing sparse files via FUSE layer	✅	✅	❌	❌

FUSE-level sparseness requires lseek() support in the FUSE implementation (currently Linux and FreeBSD). On macOS and Windows, files are exposed via FUSE as non-sparse even though dwarfsextract can still write sparse files when extracting. Missing lseek() support is tracked here for Windows and here for macOS.

Sub-second timestamps

Configurable sub-second timestamp resolution down to nanoseconds (using --time-resolution). The default remains one second. This is fully backwards-compatible: older DwarFS versions can read images with sub-second resolution, but will ignore the sub-second part.

---

Bug fixes

• Leading dots in --input-list file paths were incorrectly treated as literal directory names instead of being expanded. This has been fixed. Fixes #292.

• The SPDX license identifier in GPL-licensed source files was incorrectly specified as GPL-3.0-only instead of GPL-3.0-or-later. This has been corrected. Fixes #275.

• Fixed an off-by-one error when recovering self_index fields in metadata, which could cause the sentinel directory to have a non-zero self_entry. While harmless by itself (since that entry is never actually used), this would cause the metadata consistency check to fail. The fix covers three aspects: correcting the off-by-one error; ensuring the self_entry recovery code does not run for the sentinel directory; and changing the metadata consistency check to only warn about a non-zero self_entry rather than fail. Running mkdwarfs with --rebuild-metadata will also reset a non-zero sentinel self_entry to zero.

• Fixed the implementation of the read operation in the FUSE driver to send positive error code values to libfuse. This was likely never triggered in practice, but in cases where parts of the filesystem image vanish while being accessed (which previously caused SIGBUS crashes), libfuse would not understand the negative error codes.

• Moved the FUSE driver binaries from sbin to bin and kept only the mount.dwarfs/mount.dwarfs2 symlinks in sbin. This better aligns with user expectations, other FUSE drivers, and the fact that the man pages are installed in section 1. (Thanks to Ahmad Khalifa for the fix.)

• The dwarfs2 binary was broken in builds using shared libraries. (Thanks to Ahmad Khalifa for the fix.)

• When setting CPU thread affinity for worker group threads via DWARFS_WORKER_GROUP_AFFINITY, the code did not CPU_ZERO the cpu_set_t structure before setting individual CPUs. This could pin threads to random CPUs in addition to the requested ones.

• The FITS categorizer would scan entire files for the end-of-header marker if their size was a multiple of 2880 bytes, causing significant slowdowns on large non-FITS files. Additional checks now ensure scanning only continues if the data truly looks like a standards-compliant FITS header.

• GCC caught a potential null-pointer dereference on error when opening a file in mkdwarfs. This has been fixed.

• Numerous fixes for 32-bit architectures, mostly related to integer overflows with file sizes larger than 4 GiB.

• Another off-by-one error caused the first regular file inode to be excluded from the file-size cache. This would be hard to notice unless that file was highly fragmented. The cache will be fixed when rebuilding the metadata.

• The FUSE driver’s enable_nlink option is now the default behavior and cannot be disabled. The previous optimization skipped building a table of hardlink counts, which produced inherently incorrect file status information (hardlinked files share an inode, so reporting a link count of 1 is wrong). The hardlink table is now stored in the metadata by default; if there are no hardlinks, it consumes no space. You can still omit the hardlink table with --no-hardlink-table, at the cost of building it on-the-fly when the filesystem image is loaded (typically fast — e.g., ~300 ms for 14 million files).

• Fixed a typo in dwarfs-format.md. (Thanks to Dennis Brakhane for spotting this and sending a PR.)

Features

• New I/O layer abstraction that supports “classic” mmap-based file access, granular mmap-based access on 32-bit systems, and fully mmap-less access if desired. This applies to all DwarFS tools. By default, tools use the most efficient method—memory-mapping whole files on 64-bit systems and mapping file segments on 32-bit systems (to conserve address space). This can be controlled via the new DWARFS_IOLAYER_OPTS environment variable described in dwarfs-env(7).

• Full support for sparse files. mkdwarfs now detects and efficiently processes sparse files, skipping holes where possible and preserving them in the filesystem image. This is supported on all platforms. The FUSE driver implements lseek() where supported by the FUSE library (currently Linux and FreeBSD); Windows and macOS fall back to showing files as non-sparse. dwarfsextract extracts sparse files as such and preserves sparse representations when extracting to archive formats that support them (e.g., tar). Note: Sparse file support is not backwards compatible; images containing sparse files cannot be processed by DwarFS versions prior to 0.14.0. By default, mkdwarfs enables sparse file support if it detects sparse input. Use --no-sparse-files to disable it and ensure compatibility with older versions.

• Support for subsecond timestamp resolution. The default remains one second, but finer resolutions (down to nanoseconds) can be specified with --time-resolution. mkdwarfs will warn if the requested resolution is finer than the native filesystem resolution. This is fully backwards compatible: older DwarFS versions will handle such images but ignore the subsecond parts. Fixes #294.

• Desktop integration for Linux. A new --auto-mountpoint option automatically creates or selects a mount-point directory, making it easier to mount DwarFS images from file managers. Desktop files and MIME type definitions are now installed to enable double-click mounting of .dwarfs files. (Thanks to Ahmad Khalifa for the implementation.)

• Shell completion for mkdwarfs (bash and zsh). (Thanks to Ahmad Khalifa for the contribution.)

• Improved error handling when DwarFS tools encounter SIGBUS (usually caused by accessing memory-mapped files on unreliable or faulty storage like network shares or flaky USB drives). When SIGBUS is caught, tools now print an error suggesting switching from mmap- to read-based I/O via DWARFS_IOLAYER_OPTS.

• dwarfsck now checks metadata consistency by default (unless --no-check is given), improving detection of filesystem image corruption.

• If sparse files are supported by the FUSE library, the FUSE driver exposes new options cache_sparse and no_cache_sparse to control whether sparse files should be cached in the kernel page cache. See dwarfs(1) for details.

• The JSON output from dwarfsck now contains a complete raw metadata dump when the detail level includes metadata_full_dump.

• dwarfsck no longer artificially limits string sizes when dumping metadata. (Thanks to Dennis Brakhane for the contribution.)

• Accelerated search for the start of a DwarFS image in files with custom headers; the new code is about four times faster, scanning at more than 6 GiB/s on a modern CPU.

• The cache size can now be configured for dwarfsck, useful with the --checksum option.

• Both dwarfsck and dwarfsextract now limit the amount of ...

dwarfs-0.13.0

FreeBSD, big-endian, and many new architectures

This release finally includes FreeBSD in the list of supported operating systems. That should make it much easier to port DwarFS to other *BSDs as well. Big-endian platforms are also supported, and the file system images (which use little-endian) are fully portable between architectures. Binary releases are now available for a wide range of architectures: aarch64, arm, i386, loongarch64, ppc64, ppc64le, riscv64, s390x, and x86_64.

Metadata manipulation

Before this release, file system metadata was immutable once a DwarFS image had been created, and the only way to manipulate metadata was to build a new image from scratch. This release adds two options to mkdwarfs:

• --rebuild-metadata allows changes/upgrades to the metadata block. This makes it easy to change how the metadata is packed, or even perform manipulations such as --chmod after the fact.
• --change-block-size allows you to change the physical block size of the file system image.

Bug fixes

• The linker configuration for the release binaries was broken. The symptom was that, very occasionally, tests would fail in CI with std::terminate being called after the exception handling code failed to unwind the stack. The root cause was that, in clang builds, code from libunwind and libstdc++ was arbitrarily mixed, which—depending on the order in which individual threads were scheduled in the unit tests—could lead to stack unwinding working flawlessly or not at all. This was one of the hardest bugs to track down this year; fortunately, the fix was quite simple. It’s possible this issue is present in previously released binaries, although there have been no reports.

• Made section index discovery more robust. Fixes #264.

• A recent kernel change (LKML thread, 2025-05-05) caused tools_test to fail on Linux 6.14 and later. This has been fixed by accepting both EPERM and ENOSYS as valid error codes for link() calls.

Features

• FreeBSD support. Everything that works on Linux should also work on FreeBSD. No static binaries are provided for FreeBSD, but the build should work out of the box once all dependencies are installed.

• Big-endian architectures. This is still experimental, even though all unit tests pass under QEMU and the benchmark suite runs on real hardware. This currently requires forked versions of folly and fsst. The changes are small, and the pull requests will hopefully be merged upstream soon. (facebook/folly#2484, cwida/fsst#36)

• Experimental 32-bit support. While DwarFS should largely “just work” on 32-bit with small images (a few hundred megabytes), limited address space is a problem due to DwarFS’s extensive use of memory-mapped files. There will be changes to limit the use of mmap in the future (primarily due to other issues), which should improve 32-bit compatibility as a side effect. Fixes #268.

• Wider CPU architecture coverage. Static binary releases (including universal binaries) are now available for x86_64, aarch64, i386, arm, ppc64, ppc64le, riscv64, s390x, and loongarch64. Building the new release binaries uncovered a few bugs in clang (llvm/llvm-project#150913), binutils (https://sourceware.org/bugzilla/show_bug.cgi?id=33223), mold (rui314/mold#1490, rui314/mold#1496, rui314/mold#1497, rui314/mold#1498), and UPX (upx/upx#925), not all of which have been fixed. As a result, the binaries use slightly different toolchains and configurations depending on the architecture. Fixes #266, #268.

• Custom self-extracting stub for universal binaries. It aims for simplicity and portability and should work on most Linux systems. It is used if UPX support for an architecture is unavailable, or if the binaries extract much faster than the UPX-compressed version. The stub also supports --extract-wrapped-binary <file> to extract the embedded binary.

• Category metadata stored by default. The category metadata for categorized blocks is now stored in the metadata block by default. This allows recompressing blocks with a metadata-dependent algorithm (e.g., FLAC) even if they were previously compressed with a metadata-independent algorithm. You can disable this with --no-category-metadata. See the mkdwarfs man page for details.

• Options for smaller metadata. The --no-category-names and --no-category-metadata options can be used to reduce metadata size. However, this makes it impossible to use metadata-dependent compression algorithms (e.g., FLAC) or to select category-specific compression when recompressing the image.

• Metadata rebuilding in mkdwarfs. In addition to recompressing, it is now possible to change metadata packing and apply operations such as --set-owner, --set-group, --set-time, --time-resolution, --chmod, or --no-create-timestamp. Note that these operations are potentially lossy and may be irreversible. By default, the history of metadata rebuilds is tracked in the metadata itself; you can disable this with --no-metadata-version-history.

• Change block size on existing images. You can now change the block size of an existing image using --change-block-size. This implies --rebuild-metadata and --recompress=all and can be useful for tuning performance without recreating the image from scratch.

• Runtime memory display in mkdwarfs. mkdwarfs now shows its current memory usage while running. Note that -L/--memory-limit still only limits the memory used for the block queue, not overall memory usage. Fixing this is on the roadmap; there’s no need to file an issue.

• dwarfsextract format controls. New options --format-options and --format-filters control the output format. There is also --format=auto to automatically “guess” the format and filters based on the output file name. (Thanks to @oxalica for the pull request.)

• dwarfsck detail level. New frozen_details detail level shows frozen_analysis content ordered by memory location instead of memory usage, and also shows the address range of each section.

• Lean LRU cache. Replaced folly’s EvictingCacheMap with a simple in-repo LRU cache implementation. This reduces external dependencies and binary size without sacrificing performance.

• Windows extended attributes. The pxattr utility now supports all extended attribute operations on Windows, including setxattr() and removexattr(). Error handling and reporting for extended attributes on Windows has also been improved.

Docs

• Updated dwarfs-format.md with more information on section types, compression metadata, and the Frozen2 binary metadata layout. (Thanks to @oxalica for asking the right questions, reporting bugs, and ultimately releasing a Rust library to read/write DwarFS images.)

• Added a notable users section to the README. (Thanks to Vitaly Zdanevich for the PRs.)

• Updated mkdwarfs docs with more information on worker threads and the requirements for bit-identical images.

• Major README overhaul: added a Quick Start section, added more links, and fixed typos and wording.

Full Changelog: v0.12.4...v0.13.0

SHA-256 Checksums

eb4a21fe560721a17059eb26b14abb894420008cc0dc990b829bdfda08e97af8  dwarfs-0.13.0-Linux-aarch64.tar.xz
3781cf4e5dde77f4e7da9900dca4250d300b57bbf5ba6640ba7f61e2efb5782f  dwarfs-0.13.0-Linux-arm.tar.xz
d7157d7a2faedea61829835060edfb1614d75b8559252f237aae38a97a684e9c  dwarfs-0.13.0-Linux-i386.tar.xz
786351112f1659d041e40ef70a4376c55f68bb783fc710ff71a1cb612b6786f5  dwarfs-0.13.0-Linux-loongarch64.tar.xz
465c0a2c14f13612a0413ba4e79f853cefdcc25a82d8ea7e74e3a349ac887983  dwarfs-0.13.0-Linux-ppc64le.tar.xz
cb44187074793b4aca9e252fb0bee269725180448e7a47abbf17e38b0b32e6ac  dwarfs-0.13.0-Linux-ppc64.tar.xz
0d8eac724b5c00f22592c68c2f61a1ae7be92aef0b236ace208e73e759e6965f  dwarfs-0.13.0-Linux-riscv64.tar.xz
52dbd8e44deede8e237d4b8c2fe1388b1478c14b1f76b63b4045a7103110acfb  dwarfs-0.13.0-Linux-s390x.tar.xz
81d2c5178b51367207df819a3a754fc3141d0f74d4ac80dbf20b1f0bd9d1be44  dwarfs-0.13.0-Linux-x86_64.tar.xz
d0654fcc1219bfd11c96f737011d141c3ae5929620cd22928e49f25c37a15dc9  dwarfs-0.13.0.tar.xz
fbbdf50657caf6be3c6864768bd2f0c2f6ea955b66e07875408e0a78bed2f9b9  dwarfs-0.13.0-Windows-AMD64.7z
78c52f9ca120e11d4a4620c614d532425ab694c0f8b6c25f04b475f10b3e0b2e  dwarfs-fuse-extract-0.13.0-Linux-aarch64
451744a2be3312fcd2968aa7821d61d97ece24177cebdb4cb8cf463409f9de7d  dwarfs-fuse-extract-0.13.0-Linux-aarch64.upx
0079697de87e14ea5bbf3a0dd0e95488d8343bd423f2ca3643462520a6c95b89  dwarfs-fuse-extract-0.13.0-Linux-arm.upx
7d3fc8474c1372a92b4301c28124fac16f858d81686ee4cafe21e8b6a1cebc14  dwarfs-fuse-extract-0.13.0-Linux-i386.upx
da9558153264fd14ba49f02d9fb26a8ef5c090a3e99a643fbeb9e88b68c3fd93  dwarfs-fuse-extract-0.13.0-Linux-loongarch64
b42ad7d4a229d22ecf219fd1759f1f1ebb0ccd28b4be16e49cf307f05b154f63  dwarfs-fuse-extract-0.13.0-Linux-ppc64
a81001d3a195eab8cdfd2b29768718ff5eae64a69174d33c5114aa04b87bdb6a  dwarfs-fuse-extract-0.13.0-Linux-ppc64le
9c9f4dcee5c78466cda8d3f3b5dccaa57be0270774e373198ab5cdc6f46a8c5d  dwarfs-fuse-extract-0.13.0-Linux-riscv64
aeae39035e52632a013f318909b38e0525dc04ba4e52179ab678807b6a3755ea  dwarfs-fuse-extract-0.13.0-Linux-s390x
04c9587395dbbdfdb6562f945daf2aa83e5ff80691e1dfeb7854dfc2e0e6993e  dwarfs-fuse-extract-0.13.0-Linux-x86_64
4a31bcbf9145fa422d3cfaf3f1f58b40cd095ba9a4bee508123f97e208684f52  dwarfs-fuse-extract-0.13.0-Linux-x86_64.upx
74b730031dfe4d4aa0caaa411ea806d02ba44f32dc2618e52739ce7...

dwarfs-0.12.4

This release mainly fixes a bug that was introduced with v0.12.0 (06f8728 to be precise). When re-compressing a file system image where some blocks cannot be compressed using the selected algorithm because of a bad_compression_ratio_error (i.e. when the compressed size is actually larger than the uncompressed size), the resulting block object was left empty, which subsequently led to a segfault.

A few small bugs have also been fixed and couple of features were added to the dwarfsck tool, mostly triggered by the discussion in #263.

Most other changes that made it into the release were related to how the static binaries are built. All the dependencies have been updated, in particular also for the Windows build. The Windows build was also switched from openssl to libressl. Overall, the size of the Windows universal binary was reduced by 30% and is now comparable in size to the Linux binaries.

Bugfixes

• Segfault on bad_compression_ratio_error. When recompressing a filesystem where some blocks cannot be compressed using the selected algorithm because of a bad_compression_ratio_error, the resulting block was left empty after the refactoring done in 06f8728.
• Add history unless --no-history is given when rewriting a file system image.
• Allow dumping frozen_layout w/o frozen_analysis in dwarfsck.
• Logging timestamps should show local time.
• Workaround a weird MSVC bug.
• Remove useless cast causing compiler warning.

Features

• More complete breakdown of metadata in dwarfsck.
• Add schema_raw_dump flag to dwarfsck --detail.

Build

• Switch static build to libressl on Windows.
• Update static build libraries.
• Update folly/fbthrift/fsst.

Other

• Introduce and use safe_localtime() to prevent issues with fmt deprecating fmt::localtime.
• Speed up a few slow tests on Windows.

Full Changelog: v0.12.3...v0.12.4

SHA-256 Checksums

1492c0796ab3479a80e4e191e651f20c005e634b73ec684edd48465830f0aab3  dwarfs-0.12.4-Linux-aarch64.tar.xz
bff9da50918cf9272dbb9322733a629ccd84efc723e355a86eafcd075081f968  dwarfs-0.12.4-Linux-x86_64.tar.xz
352d13a3c7d9416e0a7d0d959306a25908b58d1ff47fb97e30a7c8490fcff259  dwarfs-0.12.4.tar.xz
5574d6aeb970c4cbabd50526994159c0551b2dc3e940edc774ac0952e1528c93  dwarfs-0.12.4-Windows-AMD64.7z
72f688800faf74acfdc75b6d77a456930db7088ee7c7e115c4b95414ab751c93  dwarfs-fuse-extract-0.12.4-Linux-aarch64
ea182efeb3ac55f8f79ee80009854c2ac8410b37d427da90e42b96eec73c470f  dwarfs-fuse-extract-0.12.4-Linux-x86_64
06c35c1dc99bc9c19f73ceda238866f9b4a631c08ab4ad19bbf8ea5c0a3ff9f0  dwarfs-fuse-extract-mimalloc-0.12.4-Linux-aarch64
afa48dfda2692e5b3cba6c523ab66696140d48dc8c4dfd57b114c73632e45326  dwarfs-fuse-extract-mimalloc-0.12.4-Linux-x86_64
7efdbd93954f1f88898b690466dca4b2ae3d20e799247c4ee4459acb35824389  dwarfs-universal-0.12.4-Linux-aarch64
48fc469bca251b932904b33d6161fc9a964cdefdb9d4d128828b4766b9c63eaa  dwarfs-universal-0.12.4-Linux-x86_64
094eaaf821df47daaa11899927f269f57f71843b39ecd53bae70692447cb3e65  dwarfs-universal-0.12.4-Windows-AMD64.exe

dwarfs-0.12.3

This release provides a fix for cases where automatic image offset detection could fail to work correctly as well as further size optimizations of the release binaries. The dwarfs-universal binary now uses LibreSSL's libcrypto, whereas the binaries from the release tarball use OpenSSL's libcrypto. This is a trade-off favoring size for the universal binary and speed for the "regular" binaries. Note, however, that this will be imperceptible unless you use dwarfsck with either --check-integrity or --checksum.

	v0.11.3	v0.12.0	v0.12.1	v0.12.2	v0.12.3
Linux x86_64 universal binary	5,319,916	2,833,280	2,903,624	2,968,252	2,215,464
Linux aarch64 universal binary	4,637,312	2,725,864	2,588,924	2,636,912	2,180,928
Linux x86_64 fuse-extract binary (jemalloc)	-	1,183,752	-	906,016	845,984
Linux aarch64 fuse-extract binary (jemalloc)	-	1,188,760	-	913,260	885,416
Linux x86_64 fuse-extract binary (mimalloc)	-	-	1,075,536	835,500	774,804
Linux aarch64 fuse-extract binary (mimalloc)	-	-	1,059,588	839,740	811,156
Linux x86_64 binary tarball	7,736,712	3,888,104	3,698,356	3,703,712	3,600,544
Linux aarch64 binary tarball	6,791,424	3,497,140	3,271,584	3,296,380	3,258,856

Bugfixes

• Automatic image offset detection (for images using a custom header) did not work correctly if the header contained a string that would be identified as the start of a v1 section header (these were only used before dwarfs-0.3.0). If there was either "DWARFS\x02\x00" or "DWARFS\x02\x01" in the header, offset detection would fail. The check has been modified to peek further into the data and ensure this really is a v1 section header, and also checking if the next section header position can be derived from the length field. It is still possible to construct a file system image where offset detection will ultimately fail, but it is much less likely with the change.

Build

• The build process for the release binaries has been further tweaked to reduce binary size. The dwarfs-fuse-extract binary now again supports extracting files by pattern; I didn't realize that this was actually a widely used feature before dropping it in the last release. dwarfs-universal is now linked against LibreSSL's libcrypto instead of OpenSSL's. This significantly reduces the size at the expense slightly slower cryptographic hash functions. However, this will likely only be perceivable when using --tool=dwarfsck with either --check-integrity or --checksum. The binaries from the release tarballs are still linked against libcrypto from OpenSSL.

Full Changelog: v0.12.2...v0.12.3

SHA-256 Checksums

7e2c1d6f4bf8f19cedc8f050f405906e269d96197e238226f334c7ae2fb1f489  dwarfs-0.12.3-Linux-aarch64.tar.xz
9a2590deb3069d7e677604304d94226fa10b7385ccee46ba8c66f4e6c902168c  dwarfs-0.12.3-Linux-x86_64.tar.xz
bd2d54178c59e229f2280eea747479a569e6f6d38340e90360220d00988f5589  dwarfs-0.12.3.tar.xz
acbfbf5a48a8fa53dd6a39d7450c8d6849ef99d305563461596ac5c52767387a  dwarfs-0.12.3-Windows-AMD64.7z
16a450e6996ab59b89f69b0e30dab508d1e88c526586df5db5b7a43ede252398  dwarfs-fuse-extract-0.12.3-Linux-aarch64
e658a0513bc9168ff6f366b2dc19f6360408663fc2f0015653912075526c05a2  dwarfs-fuse-extract-0.12.3-Linux-x86_64
755599e8b52e36a87d6b90e860f38c7cf3c71bcebd8f63e11834cca0fbed9708  dwarfs-fuse-extract-mimalloc-0.12.3-Linux-aarch64
372c3090fd966e881978aaa7ccfcbb1476cf16a3cc3ddbaaddeac14cf141cb5d  dwarfs-fuse-extract-mimalloc-0.12.3-Linux-x86_64
9c66639a66a122d964ee297b93a638c61f25c917e9aee49339fde17320757f2a  dwarfs-universal-0.12.3-Linux-aarch64
c5e83388f41cddb59b3c490fc9fad8fbd73b134f8da5b8bba77297ab593d0efb  dwarfs-universal-0.12.3-Linux-x86_64
9dbc03ce6dfc9df25e8c9e77e5ee15f624ef244aaa9b4e235fb121761c698789  dwarfs-universal-0.12.3-Windows-AMD64.exe

dwarfs-0.12.2

This release provides a fix for a performance regression, switches the default memory allocator back to jemalloc, and further reduces the size of the dwarfs-fuse-extract binary. The latter is available as both a jemalloc and a mimalloc version. jemalloc offers a lot more configuration options that can be crucial in optimizing the memory profile of e.g. the FUSE driver. If you don't need that flexibility, you can save a few bits by using the -mimalloc version.

	v0.11.3	v0.12.0	v0.12.1	v0.12.2
Linux x86_64 universal binary	5,319,916	2,833,280	2,903,624	2,968,252
Linux aarch64 universal binary	4,637,312	2,725,864	2,588,924	2,636,912
Linux x86_64 fuse-extract binary (jemalloc)	-	1,183,752	-	906,016
Linux aarch64 fuse-extract binary (jemalloc)	-	1,188,760	-	913,260
Linux x86_64 fuse-extract binary (mimalloc)	-	-	1,075,536	835,500
Linux aarch64 fuse-extract binary (mimalloc)	-	-	1,059,588	839,740
Linux x86_64 binary tarball	7,736,712	3,888,104	3,698,356	3,703,712
Linux aarch64 binary tarball	6,791,424	3,497,140	3,271,584	3,296,380

Bugfixes

• The dwarfs-0.12.0 release introduced a performance regression where FLAC compression took more than twice as long as in the previous releases. This has been fixed. FLAC decompression was unaffected.

Build

• A few small refactoring changes to further reduce the size of the fuse-extract binary. In particular, the performance monitor and the history feature are now fully removed. Also, the functionality to extract in different archive formats as well as to extract only files matching a pattern have been removed, so the image can only be fully extracted to disk.

Full Changelog: v0.12.1...v0.12.2

SHA-256 Checksums

7d58b4125171befb5457a6318cda99607e32c2226db74de5f7449dee0e10764f  dwarfs-0.12.2-Linux-aarch64.tar.xz
61d239c0583d88443ca3e0080f1fe8bc97979a3ad67ed15ca3516e27ea7e7f53  dwarfs-0.12.2-Linux-x86_64.tar.xz
9b256d1f2bc17917cd63a1bee3bd5f505076b4d880fcf9daa18a6ca5bca35aeb  dwarfs-0.12.2.tar.xz
1ffbe8bbf44c5168aba5d0132705ad46837e4925ff57798333efcc6312bd7441  dwarfs-0.12.2-Windows-AMD64.7z
2aea873299cecc68dc0d8028d55cb00dc5c3289d12896e70e2038ea09d780c4e  dwarfs-fuse-extract-0.12.2-Linux-aarch64
88559806c8f2a98108e9ecf24926a317ea0afe655ef45f76c476307c4e71d971  dwarfs-fuse-extract-0.12.2-Linux-x86_64
99202109637d4d49d3b3945b35487deada758358b4f185869975c7c4be9870fa  dwarfs-fuse-extract-mimalloc-0.12.2-Linux-aarch64
8af6f46b5c39fa7fa9294b652bd023302599f2723f33571b3e2bf2376f420770  dwarfs-fuse-extract-mimalloc-0.12.2-Linux-x86_64
79a8e5d729650d8f26e1759228a10c2ea49ae88c1a491741f9196ce2937b4e2e  dwarfs-universal-0.12.2-Linux-aarch64
29d3195831c8ff3aca46b2a731eee7899d3735a71b870e9510adeaeb34dd135c  dwarfs-universal-0.12.2-Linux-x86_64
1232104b7c44dda3da46fdc6f7667e542e2e916ac926e2395965bcc035ae8046  dwarfs-universal-0.12.2-Windows-AMD64.exe