files: selinux relabel takes a long time when re-using /home partition

[dustymabe]

Bug

If I re-use a partition/filesystem for /home/ then the selinux relabel can take a significant amount of time. I suspect that we're relabeling all of the files from the re-used partition and we should probably try to narrow our scope to just files we've touched in Ignition.

Operating System Version

$ rpm-ostree status 
State: idle
Deployments:
● ostree://fedora:fedora/aarch64/coreos/testing-devel
                   Version: 34.20210615.dev.1 (2021-06-15T16:00:07Z)
                    Commit: bc0e38262d392f75d5c65cd9dab8f1da8a72a93c5297b55d726c2ce2c88d5f15
              GPGSignature: (unsigned)

It's a dev build of FCOS since we don't have official aarch64 ones yet.

Ignition Version

ignition-2.10.1-3.fc34.aarch64

Environment

Raspberry Pi 4 (B)

Expected Behavior

Quick relabel during startup.

Actual Behavior

Takes more than 2 minutes for the relabel to happen:

May 25 11:16:22 localhost ignition[670]: INFO     : files: ensureUsers: op(1): [finished] creating or modifying user "core"
May 25 11:16:22 localhost ignition[670]: INFO     : files: ensureUsers: op(2): [started]  adding ssh keys to user "core"
May 25 11:16:22 localhost ignition[670]: wrote ssh authorized keys file for user: core
May 25 11:16:22 localhost ignition[670]: INFO     : files: ensureUsers: op(2): [finished] adding ssh keys to user "core"
May 25 11:16:22 localhost ignition[670]: INFO     : files: createFilesystemsFiles: createFiles: op(3): [started]  writing file "/sysroot/etc/hostname"
May 25 11:16:22 localhost ignition[670]: INFO     : files: createFilesystemsFiles: createFiles: op(3): [finished] writing file "/sysroot/etc/hostname"
May 25 11:16:22 localhost ignition[670]: INFO     : files: createFilesystemsFiles: createFiles: op(4): [started]  writing file "/sysroot/etc/systemd/zram-generator.conf"
May 25 11:16:22 localhost ignition[670]: INFO     : files: createFilesystemsFiles: createFiles: op(4): [finished] writing file "/sysroot/etc/systemd/zram-generator.conf"
May 25 11:16:22 localhost ignition[670]: INFO     : files: createFilesystemsFiles: createFiles: op(5): [started]  writing file "/sysroot/etc/systemd/system/ostree-remount.service.d/before-rfkill.conf"
May 25 11:16:22 localhost ignition[670]: INFO     : files: createFilesystemsFiles: createFiles: op(5): [finished] writing file "/sysroot/etc/systemd/system/ostree-remount.service.d/before-rfkill.conf"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(6): [started]  processing unit "afterburn-sshkeys@core.service"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(6): [finished] processing unit "afterburn-sshkeys@core.service"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(7): [started]  processing unit "var-home.mount"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(7): op(8): [started]  writing unit "var-home.mount" at "/sysroot/etc/systemd/system/var-home.mount"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(7): op(8): [finished] writing unit "var-home.mount" at "/sysroot/etc/systemd/system/var-home.mount"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(7): [finished] processing unit "var-home.mount"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(9): [started]  setting preset to enabled for "afterburn-sshkeys@.service core"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(9): [finished] setting preset to enabled for "afterburn-sshkeys@.service core"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(a): [started]  setting preset to enabled for "var-home.mount"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(a): [finished] setting preset to enabled for "var-home.mount"
May 25 11:16:22 localhost ignition[670]: INFO     : files: op(b): [started]  relabeling 20 patterns
May 25 11:16:22 localhost ignition[670]: DEBUG    : files: op(b): executing: "setfiles" "-vF0" "-r" "/sysroot" "/sysroot/etc/selinux/targeted/contexts/files/file_contexts" "-f" "-"
May 25 11:18:47 localhost ignition[670]: INFO     : files: op(b): [finished] relabeling 20 patterns
May 25 11:18:47 localhost ignition[670]: INFO     : files: files passed
May 25 11:18:47 localhost ignition[670]: INFO     : Ignition finished successfully

Reproduction Steps

Use Ignition config with separate /home/. Here is an example Butane config:

variant: fcos
version: 1.4.0-experimental
storage:
  disks:
    - device: /dev/sda
      wipe_table: false
      partitions:
        - number: 4
          label: root
          size_mib: 51200
          resize: true
        - number: 5
          label: var-home
          size_mib: 102400
  filesystems:
    - path: /var/home
      device: /dev/disk/by-partlabel/var-home
      format: xfs
      wipe_filesystem: false
      label: var-home
      with_mount_unit: true

Install a system place a bunch of files in your home dir. For me I did some rootless podman COSA builds and the contents of my home were about 30G:

[core@rpi4 ~]$ sudo du -sh /home/
30G     /home/

Now re-install with the same Ignition config. Notice the relabel takes much more time.

[travier]

We might need a specific option in Ignition to say "do not relabel the content from this partition".

[dustymabe]

I suspect that we're relabeling all of the files from the re-used partition and we should probably try to narrow our scope to just files we've touched in Ignition.

@travier WDYT of the above suggestion?

[travier]

I think we have to relabel everything by default otherwise we could get issues with inconsistent labels from a previous installation. Potentially even security issues although I do not have an example at hand.

Thus if you explicitly decide to not relabel then you're on your own.

[dustymabe]

Hmm. Are you saying that we're doing a relabel of /home (/var/home in my example) because (and only because) it's a separate (and re-used) filesystem. I assumed it was because Ignition was creating some files in /home and we were catching some recursive relabel that way. In the former case it's a desired intentional behavior, in the latter it's a side effect and undesired unintentional behavior.

[travier]

Good point, I'm not exactly sure about what's triggering the full relabel here and what we effectively intend for re-used partitions.

[tormath1]

Hi there, after a discussion with a SELinux developer, we got the following feedback: flatcar/Flatcar#673 (comment) - which made me think to this current issue.

That would be interesting to investigate on the following suggestion on selinux.go:

-       cmd := exec.Command(distro.SetfilesCmd(), "-vF0", "-r", ut.DestDir, file_contexts, "-f", "-")
+       cmd := exec.Command(distro.SetfilesCmd(), "-DvF0", "-r", ut.DestDir, file_contexts, "-f", "-")

to see if it reduces the relabeling time.