bootc status fails with 'Multiple extra entries in /boot' after image rename

[hanthor]

Bug Report: bootc status fails with "Multiple extra entries in /boot, could not determine rollback entry" after OS/image rename

Description

When running sudo bootc status --format json on a system configured with the composefs backend, the command fails with the following error:

error: Status: Getting composefs deployment status: Getting composefs deployment status: Multiple extra entries in /boot, could not determine rollback entry

This error is encountered on Bluefin Dakota (which uses a pure bootc and composefs backend without OSTree).

---

Root Cause Analysis

1. The Duplicate Bootloader Configurations

Upon inspecting the boot entries in /boot/loader/entries/, we find four .conf files:

/boot/loader/entries/bootc_bootc-20260507-1.conf
/boot/loader/entries/bootc_bootc-20260516-0.conf
/boot/loader/entries/bootc_bluefin_dakota-20260516-0.conf
/boot/loader/entries/bootc_bluefin_dakota-20260520-1.conf

Inside the configuration files, both bootc_bootc-20260516-0.conf and bootc_bluefin_dakota-20260516-0.conf reference the exact same composefs verity digest:

• bootc_bluefin_dakota-20260516-0.conf:

  title Bluefin
  version 20260516
  linux /EFI/Linux/bootc_composefs-230a0ec9440a21baea5e14a15135e8396cbd7ea225a40162e4d2b9ad451b3551e2e7d334681dcf4ba49e86f7cad8d9966cbac8928ca9e3c4d48ab5f4c1c8cb58/vmlinuz
  initrd /EFI/Linux/bootc_composefs-230a0ec9440a21baea5e14a15135e8396cbd7ea225a40162e4d2b9ad451b3551e2e7d334681dcf4ba49e86f7cad8d9966cbac8928ca9e3c4d48ab5f4c1c8cb58/initrd
  options root=UUID=a906ce76-f256-4ba1-a260-b6b12da7fe1a rootflags=subvol=/ rw composefs=b6fa523f25053ee1f42651e1214e89d2a19c47ee26222b6fe3955914a35a9340d1e573d92c64ddf82002c9544b7e5d4d3a56a409f3e1ba031b7f429c832d4fd9 rhgb quiet rd.luks.name=3486c9a9-84b8-4448-a733-6c6a48ce2444=root
  sort-key bootc-bluefin-dakota-1

• bootc_bootc-20260516-0.conf:

  title Bluefin
  version 20260516
  linux /EFI/Linux/bootc_composefs-230a0ec9440a21baea5e14a15135e8396cbd7ea225a40162e4d2b9ad451b3551e2e7d334681dcf4ba49e86f7cad8d9966cbac8928ca9e3c4d48ab5f4c1c8cb58/vmlinuz
  initrd /EFI/Linux/bootc_composefs-230a0ec9440a21baea5e14a15135e8396cbd7ea225a40162e4d2b9ad451b3551e2e7d334681dcf4ba49e86f7cad8d9966cbac8928ca9e3c4d48ab5f4c1c8cb58/initrd
  options root=UUID=a906ce76-f256-4ba1-a260-b6b12da7fe1a rootflags=subvol=/ rw composefs=b6fa523f25053ee1f42651e1214e89d2a19c47ee26222b6fe3955914a35a9340d1e573d92c64ddf82002c9544b7e5d4d3a56a409f3e1ba031b7f429c832d4fd9 rhgb quiet rd.luks.name=3486c9a9-84b8-4448-a733-6c6a48ce2444=root
  sort-key bootc-bootc-1

This state occurred because the system transitioned from using the generic bootc image/OS name prefix to the tailored bootc_bluefin_dakota prefix.

• During updates, bootc's garbage collection and prune logic only manages boot configuration files matching the current prefix (bootc_bluefin_dakota), thereby leaving the older bootc_bootc-*.conf files orphaned in /boot/loader/entries/.
• However, when reading boot entries to compute deployment status, bootc scans all files ending in .conf under the entries directory, which includes the stale files.

---

2. Deep Dive Into Upstream containers/bootc Code

The error originates in crates/lib/src/bootc_composefs/status.rs within the composefs_deployment_status_from function:

1. Listing Bootloader Entries:
   The function retrieves all .conf boot configurations:

   let bootloader_entry_verity = list_bootloader_entries(storage)?;

   Because there are two files with the same verity digest (b6fa5...), bootloader_entry_verity contains two entries with the exact same fsverity value.

2. Categorizing Deployments:
   The code iterates through all entries to construct deployment lists:

   for BootloaderEntry {
       fsverity: verity_digest,
       ..
   } in bootloader_entry_verity
   {
       ...
       if verity_digest == booted_composefs_digest.as_ref() {
           host.spec.image = boot_entry.image.as_ref().map(|x| x.image.clone());
           host.status.booted = Some(boot_entry);
           continue;
       }
   
       if let Some(staged_deployment) = &staged_deployment {
           let staged_depl = serde_json::from_str::<StagedDeployment>(&staged_deployment)?;
   
           if verity_digest == staged_depl.depl_id {
               boot_entry.download_only = staged_depl.finalization_locked;
               host.status.staged = Some(boot_entry);
               continue;
           }
       }
   
       extra_deployment_boot_entries.push(boot_entry);
   }

   Since the duplicate verity digest (b6fa5...) is neither the booted nor the staged deployment, it gets pushed into extra_deployment_boot_entries twice.

3. Filtering Rollback Candidates:
   Further down, the code determines the rollback candidate by filtering extra_deployment_boot_entries against the set of active bootloader verity digests:

   let rollback_candidates: Vec<_> = extra_deployment_boot_entries
       .into_iter()
       .filter(|entry| {
           let verity = &entry
               .composefs
               .as_ref()
               .expect("composefs is always Some for composefs deployments")
               .verity;
           bootloader_configured_verity.contains(verity)
       })
       .collect();

   Both duplicate elements match the filter since bootloader_configured_verity contains b6fa5.... Thus, rollback_candidates ends up with a length of 2 (both containing the b6fa5... deployment information).

4. The Failure Point:
   Finally, the code runs a strict count check:

   if rollback_candidates.len() > 1 {
       anyhow::bail!("Multiple extra entries in /boot, could not determine rollback entry");
   }

   Because rollback_candidates.len() == 2, this sanity check fails and throws the error, preventing users from checking status or finalizing deployments.

---

Proposes Fixes

A. Upstream Fix (containers/bootc)

The upstream bootc status code should be resilient to duplicate bootloader configurations pointing to the same physical deployment. This can be solved by deduplicating bootloader_entry_verity by fsverity digest before processing the loop.

Inside composefs_deployment_status_from:

    // This is our source of truth
    let mut bootloader_entry_verity = list_bootloader_entries(storage)?;
    
    // Deduplicate entries by fsverity digest to avoid duplicate processing of the same deployment
    let mut seen = std::collections::HashSet::new();
    bootloader_entry_verity.retain(|entry| seen.insert(entry.fsverity.clone()));

B. Downstream Workaround (Project Bluefin / Dakota)

To unblock affected users, they can simply manually remove the stale or orphaned bootc_bootc-*.conf configuration files from the EFI/boot partition:

# 1. Mount ESP to /boot if not already mounted
sudo mount /dev/nvme0n1p1 /boot

# 2. Safely remove the orphaned configs matching the old prefix
sudo rm /boot/loader/entries/bootc_bootc-*.conf

This instantly resolves the error and allows bootc status to complete successfully.