[bug] Talos installing to wrong disk

[JackMyers001]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When adding a machine with two NVMEs to my cluster, Talos gets installed to the wrong drive.

Expected Behavior

Talos should be installed to the selected drive. The only way I've successfully installed Talos to the correct drive on this node is by physically removing all other drives.

Steps To Reproduce

Once the node has booted, I go to Omni to add it to my cluster:

I have clearly selected the 970 for installation, but consistently (three separate tries) it installs to the 980. I tried installing to the 980 in hopes that it would select the 970, but that also installed to the 980.

What browsers are you seeing the problem on?

Microsoft Edge

Anything else?

I have a node with two NVMEs installed: a 500GB Samsung 970 EVO Plus and a 2TB Samsung 980 PRO. I want to use the 970 as the boot drive and the 980 for Rook Ceph storage.

From an Arch Linux 2025.04.1 installation disk, here's the output of lsblk:

NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0     7:0    0 841.4M  1 loop /run/archiso/airootfs
sda       8:0    1  28.7G  0 disk
└─sda1    8:1    1  28.7G  0 part
nvme0n1 259:0    0   1.8T  0 disk
nvme1n1 259:1    0 465.8G  0 disk

(/dev/sda is the install USB) and lspci:

00:00.0 Host bridge: Intel Corporation Tiger Lake-H 8 cores Host Bridge/DRAM Registers (rev 04)
00:01.0 PCI bridge: Intel Corporation 11th Gen Core Processor PCIe Controller #1 (rev 04)
00:02.0 VGA compatible controller: Intel Corporation TigerLake-H GT1 [UHD Graphics]
00:06.0 PCI bridge: Intel Corporation 11th Gen Core Processor PCIe Controller #0 (rev 04)
00:0d.0 USB controller: Intel Corporation Tiger Lake-H Thunderbolt 4 USB Controller (rev 04)
00:14.0 USB controller: Intel Corporation Tiger Lake-H USB 3.2 Gen 2x1 xHCI Host Controller (rev 11)
00:14.2 RAM memory: Intel Corporation Tiger Lake-H Shared SRAM (rev 11)
00:15.0 Serial bus controller: Intel Corporation Tiger Lake-H Serial IO I2C Controller #0 (rev 11)
00:15.1 Serial bus controller: Intel Corporation Tiger Lake-H Serial IO I2C Controller #1 (rev 11)
00:15.2 Serial bus controller: Intel Corporation Device 43ea (rev 11)
00:15.3 Serial bus controller: Intel Corporation Device 43eb (rev 11)
00:16.0 Communication controller: Intel Corporation Tiger Lake-H Management Engine Interface (rev 11)
00:17.0 SATA controller: Intel Corporation Tiger Lake SATA AHCI Controller (rev 11)
00:19.0 Serial bus controller: Intel Corporation Device 43ad (rev 11)
00:19.1 Serial bus controller: Intel Corporation Device 43ae (rev 11)
00:1c.0 PCI bridge: Intel Corporation Tiger Lake-H PCI Express Root Port #5 (rev 11)
00:1d.0 PCI bridge: Intel Corporation Tiger Lake-H PCI Express Root Port #9 (rev 11)
00:1d.2 PCI bridge: Intel Corporation Device 43b2 (rev 11)
00:1e.0 Communication controller: Intel Corporation Device 43a8 (rev 11)
00:1e.3 Serial bus controller: Intel Corporation Device 43ab (rev 11)
00:1f.0 ISA bridge: Intel Corporation HM570 LPC/eSPI Controller (rev 11)
00:1f.3 Audio device: Intel Corporation Tiger Lake-H HD Audio Controller (rev 11)
00:1f.4 SMBus: Intel Corporation Tiger Lake-H SMBus Controller (rev 11)
00:1f.5 Serial bus controller: Intel Corporation Tiger Lake-H SPI Controller (rev 11)
01:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
01:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
02:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO
03:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller (rev 15)

If it's relevant, my particular motherboard has two NVMe slots: a PCIe Gen 4 x4 directly connected to the CPU, and a Gen 3 x4 connected to the PCH. I have the 970 plugged into the Gen 3 slot, and the 980 is plugged into the Gen 4 slot.

The installation media I've been using was generated through the Omni web UI. The options I've selected are:

• Talos version: 1.9.5
• Options: ISO (amd64)
• Pre-Install Extensions:
  • intel-ucode
• Secure Boot: selected

Interestingly, the order of the drives listed in the Omni UI is different to Arch (/dev/nvme0n1 for the 970 on Arch vs /dev/nvme1n1 on Talos).

I've had success installing Talos to machines with multiple drives before, but I think this is the first time I've tried installing where the boot drive isn't the "primary" drive (by device hierarchy).

[Unix4ever]

So it installs to the wrong drive even if you pick the right one in the Omni UI?

[JackMyers001]

Correct - regardless of which drive I pick in the Omni UI, Talos is installed to the 980 PRO NVMe.

[dubyatp]

I noticed this as well. I've been trying to bring my i5 NUC 9 into Omni for the past week and have had similar problems as OP. Two NVMe disks installed: a consumer drive (for boot) installed in the compute element, and the Micron enterprise SSD (for rook/ceph) installed in the backplane of the NUC9.

I carefully checked that I was installing to the right drive in the UI -- sure enough I was...

But it was installing to the other drive, my guess is it picked the SSD with the lowest capacity, which would be the Ceph drive (the cheap SSD i have was around 2 TB, while the Micron enterprise SSD is around 980GB). I realize this is a bit of an oddball setup, hence why I think it was assuming the lowest capacity was the boot drive.

There are two workarounds -- you can either use the image factory in Omni to create a "generic" x64 image that you can manually copy via dd to the intended SSD. It works, but if you attempt to upgrade your Talos version or even install extensions, it will again attempt to overwrite the other drive.

The other workaround -- which i'm doing right now, is using a 120GB SATA SSD inside a USB enclosure (the NUC9 doesn't support SATA natively), and it seems to work fine for now. I have a M.2 SATA drive of a similar size on the way; but this does seem like a bug.

[frezbo]

how was talos booted in the first place, was it a disk image downloaded from omni? If it's a disk image, Talos is already installed and using an install disk in Omni should not have any effect.

[dubyatp]

For me -- I normally boot Talos initially over PXE using the image provided by Omni