Skip to content

need a way to telll qemu to use discard="unmap" so that FITRIM() will pass through, to reduce image sizes #301

Open
@cmurf

Description

@cmurf

Summary: The VM's need to have virtioblk set to use discard=unmap in order for guest issuance of fstrim to pass through to the host backing file. This has the effect of creating a sparse backing file, since the holes don't take up space when compressing, it reduces image compression time, storage and download resources.

Details:

  1. By mounting an oz created image on loop and using fstrim manually, I can demonstrate the large amount of garbage stuck in the raw file that should instead be holes.
# mount /dev/mapper/loop0p3 /mnt/
# ls -ls
5129956 -rw-r--r--. 1 root root 13958643712 Jul 22 15:03
Fedora-Workstation-Rawhide-20220721.n.0.aarch64.raw
# fstrim -v /mnt
/mnt: 8.5 GiB (9149927424 bytes) trimmed
# ls -ls
3293292 -rw-r--r--. 1 root root 13958643712 Jul 22 15:04
Fedora-Workstation-Rawhide-20220721.n.0.aarch64.raw

This is about 1.75GiB of garbage needlessly compressed, stored, downloaded, and decompressed - times every nightly and person who downloads and uses the image. It's a huge waste. Similar problem exists for Fedora Cloud Base image.

First mount
892856 -rw-rw-r--. 1 chris chris 4294967296 Jul 21 14:45 Fedora-Cloud-Base-36-20220721.0.x86_64.raw
fstrim /boot (partition 2)
856796 -rw-rw-r--. 1 chris chris 4294967296 Jul 21 14:45 Fedora-Cloud-Base-36-20220721.0.x86_64.raw
fstrim / (partition 5)
429316 -rw-rw-r--. 1 chris chris 4294967296 Jul 21 14:46 Fedora-Cloud-Base-36-20220721.0.x86_64.raw

I found this file in koji along with the logs and image files for the aaarch64 Workstation image above.
https://kojipkgs.fedoraproject.org//packages/Fedora-Workstation/Rawhide/20220721.n.0/images/libvirt-raw-xz-aarch64.xml

Also attaching that xml file.
libvirt-raw-xz-aarch64.xml.txt

It contains this section

<disk device="disk" type="file">
<target dev="vda" bus="virtio"/>
<source file="Fedora-Workstation-Rawhide-20220721.n.0.aarch64.raw-xz"/>
<driver name="qemu" type="raw-xz"/>
</disk>

Instead this needs:

<driver name="qemu" type="raw-xz" cache="unsafe" discard="unmap"/>  [1][2]

Alternative:
I'm told oz uses guestfs after installation to cutomize the image, so if it's easier to do the quiescing there, that's OK too.

[1]
The cache="unsafe" is consistent with other Fedora builder and compose systems, improving performance considerably.
[2]
I'm not certain driver type "raw-xz" supports discard=unmap, type "raw" definitely does.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions