HARDLINK_SCOPE does not detect reflinks (ZFS block cloning / CoW filesystems)

[bakerboy448]

Description

HARDLINK_SCOPE uses Lstat() to check st_nlink for determining whether torrent files are linked to an external library. This works for traditional hardlinks (same inode, nlink increments) but does not detect reflinks/block clones.

On ZFS with block cloning enabled (and other CoW filesystems like btrfs with cp --reflink), deduplication tools like fclones dedupe create reflinks that share physical disk blocks but maintain separate inodes with independent nlink counts. These files are invisible to the current hardlink detection.

Impact

• Torrents whose files are reflinked to the media library are classified as HARDLINK_SCOPE != outside_qbittorrent
• Automation rules using HARDLINK_SCOPE (e.g., noHL tagging, share limits, cleanup rules) incorrectly treat reflinked torrents as having no library connection
• This can lead to premature cleanup of torrents that are still serving the media library via reflinks

Current Workaround

Switch deduplication from reflink mode to hardlink mode (e.g., fclones link instead of fclones dedupe). This creates traditional hardlinks that Lstat() can detect, but loses the CoW benefits of reflinks (independent metadata, separate file lifecycle).

Suggested Enhancement

Consider detecting reflinks in addition to hardlinks when building the hardlink index. Possible approaches:

1. ficlone / FICLONE ioctl — Check if two files share extents (filesystem-dependent)
2. Compare file content hashes — Expensive but filesystem-agnostic
3. filefrag / FIEMAP ioctl — Compare physical block mappings to detect shared extents on supported filesystems (btrfs, ZFS, XFS)
4. Document the limitation — At minimum, note in the HARDLINK_SCOPE documentation that reflinks are not detected

Environment

• ZFS 2.2.9 with block cloning enabled
• fclones dedupe (reflink mode) for deduplication
• qui develop branch

[bakerboy448]

Research: Realistic Detection Approaches

After investigating how existing tools detect reflinks on CoW filesystems, here's what's actually available today:

The ZFS Gap

ZFS does not support the FIEMAP ioctl. This is the primary mechanism that btrfs/XFS tools use to detect shared extents:

• openzfs/zfs#264 (Support filefrag) — still OPEN
• openzfs/zfs#7545 (Add FIEMAP support) — CLOSED without merge (2021)
• openzfs/zfs#9554 (FIEMAP followup) — still OPEN, last updated 2026-02-26

Verified locally: filefrag -v on ZFS returns "FIBMAP/FIEMAP unsupported".

How Existing Tools Detect Reflinks

Tool	Detection Method	Works on ZFS?
duperemove	FIEMAP_EXTENT_SHARED flag (0x2000) — skips already-shared extents	No
rmlint --is-reflink	FIEMAP physical offset comparison between files	No
jdupes	Does not pre-detect; relies on FIDEDUPERANGE being idempotent	N/A
fclones	Does not detect reflinks at all; re-hashes and re-clones every run	N/A

Go Libraries Available

• github.com/frostschutz/go-fibmap — wraps FS_IOC_FIEMAP, exposes FIEMAP_EXTENT_SHARED flag. Would be the right tool if ZFS supported FIEMAP.
• github.com/SeeSpotRun/go-fibmap — fork with similar API
• golang.org/x/sys/unix — provides raw ioctl constants for direct syscall usage

What ZFS Does Expose

• Pool-level stats only: zpool get bcloneused,bclonesaved,bcloneratio — shows aggregate block cloning savings (e.g., bclonesaved: 664G, bcloneratio: 6.35x)
• zdb -T — Block Reference Table dump (offline debugging tool, not suitable for programmatic use)
• No per-file shared-block query exists

Realistic Options for qui

1. Short-term: Document the limitation — Note in HARDLINK_SCOPE docs that reflinks are not detected on ZFS. Users on ZFS with block cloning should use hardlinks instead of reflinks for deduplication if they rely on HARDLINK_SCOPE automation rules.

2. Medium-term: Content-hash comparison — When building the hardlink index, hash file content (or a prefix) in addition to checking nlink. If a torrent file's hash matches a file in the media library, treat it as "linked" regardless of link type. This is filesystem-agnostic but expensive for large libraries.

3. Future: FIEMAP when available — If/when ZFS merges PR #9554, qui could use go-fibmap to check FIEMAP_EXTENT_SHARED on extents. This would be the cleanest solution and would also work on btrfs and XFS. Worth monitoring that PR.

4. Alternative: Physical path comparison with content verification — For each torrent file, search the media library for files with identical size and matching content prefix hash. This avoids relying on filesystem-level metadata entirely.

My Workaround

Switched from reflink-based deduplication (fclones dedupe) to hardlink-based (fclones link). This makes deduplication visible to Lstat() nlink counts, which HARDLINK_SCOPE detects correctly. Trade-off: loses independent file metadata (reflinks allow independent timestamps, permissions) but this doesn't matter for immutable media files.

---

Disclosure: This issue and comment were authored with the assistance of Claude AI (Anthropic), acting as instructed by @bakerboy448.

[nitrobass24]

ZFS Recommendation: Use fclones link instead of fclones dedupe

For ZFS users relying on HARDLINK_SCOPE automation rules, I think the practical recommendation is to use fclones link (hardlinks) rather than fclones dedupe (reflinks/block clones).

With hardlinks, HARDLINK_SCOPE correctly detects the library connection via nlink > 1 and shared inodes — no code changes needed.

Verified on ZFS 2.2.x:

$ ls -li /torrents/file.mkv /media/file.mkv
117610 -rw-rw-r-- 2 apps apps 20344378213 ... /media/file.mkv
117610 -rw-rw-r-- 2 apps apps 20344378213 ... /torrents/file.mkv

Same inode, nlink=2 → HARDLINK_SCOPE = outside_qbittorrent works as expected.

The trade-off (losing independent metadata per-link) is irrelevant for immutable media files, as the original issue author also noted.

Given that ZFS still has no per-file API to detect reflinks (FIEMAP support remains unmerged upstream — openzfs/zfs#9554), documenting this recommendation in the HARDLINK_SCOPE docs seems like the right path forward rather than a code-level fix. The content-hash comparison would crush setups managing thousands of torrents. If anything I think our effort is better spent working with the guy on ZFS fixes to support FIEMAP

[asardaes]

Hello there. I use ZFS and thought about this possibility, and I created automations that will convert reflinks to hardlinks when appropriate. However, I used a Python script and a custom qui container image that includes the interpreter. If you think this could be valuable, I can share my setup details, let me know.