incus copy fails with "permission denied" on encrypted ZFS when source dataset is not a clone

[palkeo]

Is there an existing issue for this?

• There is no existing issue for this bug

Is this happening on an up to date version of Incus?

• This is happening on a supported version of Incus

Incus system details

Incus 6.0.6
zfs 2.4.0
Linux 6.12

Instance details

No response

Instance log

No response

Current behavior

When copying an instance that has snapshots on an encrypted ZFS storage pool, incus copy fails with permission denied if the source dataset is not a ZFS clone (i.e. has no origin).

Error: Create instance from copy: Failed to mount "pool/incus/virtual-machines/test-vm" on
"/var/lib/incus/storage-pools/zfs-test/virtual-machines/test-vm" using "zfs": permission denied

The underlying mount(2) syscall returns EACCES because the received dataset has keystatus=unavailable.

Root cause

When the source has snapshots, CreateVolumeFromCopy in driver_zfs_volumes.go (line 674) uses zfs send -R -w | zfs receive instead of zfs clone.

The -w (raw) flag preserves encrypted blocks as-is. When the source dataset is not a clone (no origin property), ZFS sends a full stream with embedded encryption key metadata. On receive, ZFS creates a new encryption root with keystatus=unavailable, rather than inheriting encryption from the parent dataset.

Incus then tries to mount the received dataset without loading its encryption key first, and mount(2) returns EACCES.

When the source is a clone (has an origin), zfs send -R -w sends an incremental stream from the origin. In that case, the received dataset inherits the parent's encryption root and the key is already loaded. This is why VMs created directly on ZFS (which are clones of the image) can be copied successfully.

When does a dataset have no origin?

• Instance moved from a different storage backend (e.g. btrfs → ZFS)
• Maybe: instance received via migration ?
• Maybe: Clone that was promoted (zfs promote) ?

Expected behavior

No error.

Potential fixes:

1. Don't use -w (raw) for local copies within the same pool. Since source and destination share the same encryption root, a non-raw send/receive would let ZFS decrypt and re-encrypt with the parent's key automatically. The -w flag is only needed for cross-pool or remote transfers.

2. Use individual incremental sends instead of -R -w. Send each snapshot incrementally without the replication flag, avoiding the full-stream-with-own-encryption-root problem.

Steps to reproduce

# 1. Create an encrypted ZFS dataset and add it as an Incus storage pool
zpool create testpool /dev/sdX
echo "passphrase" | zfs create -o encryption=aes-256-gcm -o keyformat=passphrase \
  -o keylocation=prompt -o mountpoint=legacy testpool/incus
incus storage create zfs-test zfs source=testpool/incus

# 2. Create a btrfs storage pool as default
incus storage create default btrfs size=8GiB
incus profile device add default root disk path=/ pool=default

# 3. Create a VM on btrfs, then move it to ZFS
incus launch images:debian/13 my-vm --vm --storage default
incus stop my-vm
incus move my-vm my-vm --storage zfs-test

# 4. Verify no origin (non-clone)
zfs get origin testpool/incus/virtual-machines/my-vm
# VALUE = -

# 5. Create a snapshot
incus snapshot create my-vm snap0

# 6. Try to copy → fails
incus copy my-vm test-vm
# Error: Create instance from copy: Failed to mount ... permission denied

Without step 5 (no snapshot), the copy succeeds because Incus uses zfs clone instead of zfs send.