[bug] Spot instance rotation on Azure and disk encryption

[lerenn]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

We're trying to implement some spot instances on Azure, and we have some stuck clusters where machines that were probably preempted then restarted manually (not sure but I want to test this). It seems that the machines have logs but Talos API is unreachable for some reason (see 'monitor' section on one of the cluster machines for more info). Even with a machine reboot and that everything seems fine on these, Omni can't connect to them again.

It seems that the machine has troubles with the encryption of its disk and has the concerned machines seems to be in deletion, we are wondering if Omni is not deleting the encryption keys before the effective deletion of machines. This would not let machines to come back and block their deletion.

We're using the network KMS hosted in Omni to store the keys:

    systemDiskEncryption:
        state:
            provider: luks2
            keys:
                - kms:
                    endpoint: https://<redacted>-int.siderolink.omni.siderolabs.io/
                  slot: 0
        ephemeral:
            provider: luks2
            keys:
                - kms:
                    endpoint: https://<redacted>-int.siderolink.omni.siderolabs.io/
                  slot: 0

Here are the logs on the reboot:

05/03/2025 10:18:30 Linux version 6.12.13-talos (@buildkitsandbox) (gcc (GCC) 14.2.0, GNU ld (GNU Binutils) 2.43.1) #1 SMP Wed Feb 12 14:18:41 UTC 2025
05/03/2025 10:18:30 Command line: BOOT_IMAGE=/A/vmlinuz talos.platform=azure console=ttyS0,115200n8 earlyprintk=ttyS0,115200 rootdelay=300 net.ifnames=0 talos.dashboard.disabled=1 sysctl.kernel.kexec_load_disabled=1 init_on_alloc=1 slab_nomerge pti=on consoleblank=0 nvme_core.io_timeout=4294967295 printk.devkmsg=on ima_template=ima-ng ima_appraise=fix ima_hash=sha512 siderolink.api=https://<redacted>.siderolink.omni.siderolabs.io?grpc_tunnel=false&jointoken=<redacted> talos.events.sink=[fdae:41e4:649b:9303::1]:8090 talos.logging.kernel=tcp://[fdae:41e4:649b:9303::1]:8092
05/03/2025 10:18:30 BIOS-provided physical RAM map:
05/03/2025 10:18:30 BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
05/03/2025 10:18:30 BIOS-e820: [mem 0x00000000000c0000-0x00000000000fffff] reserved
05/03/2025 10:18:30 BIOS-e820: [mem 0x0000000000100000-0x000000003ff40fff] usable
05/03/2025 10:18:30 BIOS-e820: [mem 0x000000003ff41000-0x000000003ffc8fff] reserved
05/03/2025 10:18:30 BIOS-e820: [mem 0x000000003ffc9000-0x000000003fffafff] ACPI data
05/03/2025 10:18:30 BIOS-e820: [mem 0x000000003fffb000-0x000000003fffefff] ACPI NVS
05/03/2025 10:18:30 BIOS-e820: [mem 0x000000003ffff000-0x000000003fffffff] usable
05/03/2025 10:18:30 BIOS-e820: [mem 0x0000000100000000-0x00000004bfffffff] usable
05/03/2025 10:18:30 printk: legacy bootconsole [earlyser0] enabled
05/03/2025 10:18:30 NX (Execute Disable) protection: active
05/03/2025 10:18:30 APIC: Static calls initialized
05/03/2025 10:18:30 efi: EFI v2.7 by Microsoft
05/03/2025 10:18:30 efi: ACPI=0x3fffa000 ACPI 2.0=0x3fffa014 SMBIOS=0x3ff85000 SMBIOS 3.0=0x3ff83000 MEMATTR=0x3f5c7a98 INITRD=0x3eb17998 RNG=0x3ffd1018 
05/03/2025 10:18:30 random: crng init done
05/03/2025 10:18:30 SMBIOS 3.1.0 present.
05/03/2025 10:18:30 DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 08/23/2024
05/03/2025 10:18:30 DMI: Memory slots populated: 2/2
05/03/2025 10:18:30 Hypervisor detected: Microsoft Hyper-V
05/03/2025 10:18:30 Hyper-V: privilege flags low 0x2e7f, high 0x3b8030, hints 0x40c2c, misc 0xbed7b2
05/03/2025 10:18:30 Hyper-V: Nested features: 0x0
05/03/2025 10:18:30 Hyper-V: LAPIC Timer Frequency: 0xc3500
05/03/2025 10:18:30 Hyper-V: Using hypercall for remote TLB flush
05/03/2025 10:18:30 clocksource: hyperv_clocksource_tsc_page: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
05/03/2025 10:18:30 clocksource: hyperv_clocksource_msr: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
05/03/2025 10:18:30 tsc: Marking TSC unstable due to running on Hyper-V
05/03/2025 10:18:30 tsc: Detected 2445.427 MHz processor
05/03/2025 10:18:30 e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
05/03/2025 10:18:30 e820: remove [mem 0x000a0000-0x000fffff] usable
05/03/2025 10:18:30 last_pfn = 0x4c0000 max_arch_pfn = 0x400000000
05/03/2025 10:18:30 MTRR map: 4 entries (2 fixed + 2 variable; max 18), built from 8 variable MTRRs
05/03/2025 10:18:30 x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
05/03/2025 10:18:30 e820: update [mem 0x40000000-0xffffffff] usable ==> reserved
05/03/2025 10:18:30 last_pfn = 0x40000 max_arch_pfn = 0x400000000
05/03/2025 10:18:30 Kernel/User page tables isolation: force enabled on command line.
05/03/2025 10:18:30 Using GB pages for direct mapping
05/03/2025 10:18:30 Secure boot disabled
05/03/2025 10:18:30 RAMDISK: [mem 0x35bed000-0x3a447fff]
05/03/2025 10:18:30 ACPI: Early table checksum verification disabled
05/03/2025 10:18:30 ACPI: RSDP 0x000000003FFFA014 000024 (v02 VRTUAL)
05/03/2025 10:18:30 ACPI: XSDT 0x000000003FFF90E8 000064 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: FACP 0x000000003FFF8000 000114 (v06 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: DSDT 0x000000003FFD6000 01E184 (v02 MSFTVM DSDT01   00000001 MSFT 05000000)
05/03/2025 10:18:30 ACPI: FACS 0x000000003FFFE000 000040
05/03/2025 10:18:30 ACPI: OEM0 0x000000003FFF7000 000064 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: SPCR 0x000000003FFF6000 000050 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: WAET 0x000000003FFF5000 000028 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: APIC 0x000000003FFD5000 000068 (v04 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: SRAT 0x000000003FFD4000 0002F0 (v02 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: BGRT 0x000000003FFD3000 000038 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: FPDT 0x000000003FFD2000 000034 (v01 VRTUAL MICROSFT 00000001 MSFT 00000001)
05/03/2025 10:18:30 ACPI: Reserving FACP table memory at [mem 0x3fff8000-0x3fff8113]
05/03/2025 10:18:30 ACPI: Reserving DSDT table memory at [mem 0x3ffd6000-0x3fff4183]
05/03/2025 10:18:30 ACPI: Reserving FACS table memory at [mem 0x3fffe000-0x3fffe03f]
05/03/2025 10:18:30 ACPI: Reserving OEM0 table memory at [mem 0x3fff7000-0x3fff7063]
05/03/2025 10:18:30 ACPI: Reserving SPCR table memory at [mem 0x3fff6000-0x3fff604f]
05/03/2025 10:18:30 ACPI: Reserving WAET table memory at [mem 0x3fff5000-0x3fff5027]
05/03/2025 10:18:30 ACPI: Reserving APIC table memory at [mem 0x3ffd5000-0x3ffd5067]
05/03/2025 10:18:30 ACPI: Reserving SRAT table memory at [mem 0x3ffd4000-0x3ffd42ef]
05/03/2025 10:18:30 ACPI: Reserving BGRT table memory at [mem 0x3ffd3000-0x3ffd3037]
05/03/2025 10:18:30 ACPI: Reserving FPDT table memory at [mem 0x3ffd2000-0x3ffd2033]
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x3fffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000-0x4bfffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x4c0000000-0xfdfffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x1000000000-0xfcffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x10000000000-0x1ffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x20000000000-0x3ffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x40000000000-0x7ffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x80000000000-0xfffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x100000000000-0x1fffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x200000000000-0x3fffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x400000000000-0x7fffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x800000000000-0xffffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x1000000000000-0x1ffffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x2000000000000-0x3ffffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x4000000000000-0x7ffffffffffff] hotplug
05/03/2025 10:18:30 ACPI: SRAT: Node 0 PXM 0 [mem 0x8000000000000-0xfffffffffffff] hotplug
05/03/2025 10:18:30 NUMA: Node 0 [mem 0x00001000-0x3fffffff] + [mem 0x100000000-0x4bfffffff] -> [mem 0x00001000-0x4bfffffff]
05/03/2025 10:18:30 NODE_DATA(0) allocated [mem 0x4bfffb040-0x4bfffffff]
05/03/2025 10:18:30 Zone ranges:
05/03/2025 10:18:30   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
05/03/2025 10:18:30   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
05/03/2025 10:18:30   Normal   [mem 0x0000000100000000-0x00000004bfffffff]
05/03/2025 10:18:30   Device   empty
05/03/2025 10:18:30 Movable zone start for each node
05/03/2025 10:18:30 Early memory node ranges
05/03/2025 10:18:30   node   0: [mem 0x0000000000001000-0x000000000009ffff]
05/03/2025 10:18:30   node   0: [mem 0x0000000000100000-0x000000003ff40fff]
05/03/2025 10:18:30   node   0: [mem 0x000000003ffff000-0x000000003fffffff]
05/03/2025 10:18:30   node   0: [mem 0x0000000100000000-0x00000004bfffffff]
05/03/2025 10:18:30 Initmem setup node 0 [mem 0x0000000000001000-0x00000004bfffffff]
05/03/2025 10:18:30 On node 0, zone DMA: 1 pages in unavailable ranges
05/03/2025 10:18:30 On node 0, zone DMA: 96 pages in unavailable ranges
05/03/2025 10:18:30 On node 0, zone DMA32: 190 pages in unavailable ranges
05/03/2025 10:18:30 ACPI: PM-Timer IO Port: 0x408
05/03/2025 10:18:30 ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
05/03/2025 10:18:30 IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23
05/03/2025 10:18:30 ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
05/03/2025 10:18:30 ACPI: Using ACPI (MADT) for SMP configuration information
05/03/2025 10:18:30 e820: update [mem 0x3eec6000-0x3eeccfff] usable ==> reserved
05/03/2025 10:18:30 ACPI: SPCR: console: uart,io,0x3f8,115200
05/03/2025 10:18:30 CPU topo: Max. logical packages:   1
05/03/2025 10:18:30 CPU topo: Max. logical dies:       1
05/03/2025 10:18:30 CPU topo: Max. dies per package:   1
05/03/2025 10:18:30 CPU topo: Max. threads per core:   2
05/03/2025 10:18:30 CPU topo: Num. cores per package:     2
05/03/2025 10:18:30 CPU topo: Num. threads per package:   4
05/03/2025 10:18:30 CPU topo: Allowing 4 present CPUs plus 0 hotplug CPUs
05/03/2025 10:18:30 [mem 0x40000000-0xffffffff] available for PCI devices
05/03/2025 10:18:30 Booting paravirtualized kernel on Hyper-V
05/03/2025 10:18:30 clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
05/03/2025 10:18:30 setup_percpu: NR_CPUS:512 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
05/03/2025 10:18:30 percpu: Embedded 81 pages/cpu s208528 r8192 d115056 u524288
05/03/2025 10:18:30 pcpu-alloc: s208528 r8192 d115056 u524288 alloc=1*2097152
05/03/2025 10:18:30 pcpu-alloc: [0] 0 1 2 3 
05/03/2025 10:18:30 Kernel command line: BOOT_IMAGE=/A/vmlinuz talos.platform=azure console=ttyS0,115200n8 earlyprintk=ttyS0,115200 rootdelay=300 net.ifnames=0 talos.dashboard.disabled=1 sysctl.kernel.kexec_load_disabled=1 init_on_alloc=1 slab_nomerge pti=on consoleblank=0 nvme_core.io_timeout=4294967295 printk.devkmsg=on ima_template=ima-ng ima_appraise=fix ima_hash=sha512 siderolink.api=https://<redacted>.siderolink.omni.siderolabs.io?grpc_tunnel=false&jointoken=<redacted> talos.events.sink=[fdae:41e4:649b:9303::1]:8090 talos.logging.kernel=tcp://[fdae:41e4:649b:9303::1]:8092
05/03/2025 10:18:30 Unknown kernel command line parameters "BOOT_IMAGE=/A/vmlinuz", will be passed to user space.
05/03/2025 10:18:30 Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
05/03/2025 10:18:30 Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
05/03/2025 10:18:30 Fallback order for Node 0: 0 
05/03/2025 10:18:30 Built 1 zonelists, mobility grouping on.  Total pages: 4194017
05/03/2025 10:18:30 Policy zone: Normal
05/03/2025 10:18:30 mem auto-init: stack:all(zero), heap alloc:on, heap free:off
05/03/2025 10:18:30 software IO TLB: area num 4.
05/03/2025 10:18:30 SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
05/03/2025 10:18:30 allocated 33554432 bytes of page_ext
05/03/2025 10:18:30 Kernel/User page tables isolation: enabled
05/03/2025 10:18:30 ftrace: allocating 76563 entries in 300 pages
05/03/2025 10:18:30 ftrace: allocated 300 pages with 4 groups
05/03/2025 10:18:30 rcu: Hierarchical RCU implementation.
05/03/2025 10:18:30 rcu: 	RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=4.
05/03/2025 10:18:30 	Rude variant of Tasks RCU enabled.
05/03/2025 10:18:30 	Tracing variant of Tasks RCU enabled.
05/03/2025 10:18:30 rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
05/03/2025 10:18:30 rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
05/03/2025 10:18:30 RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.
05/03/2025 10:18:30 RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.
05/03/2025 10:18:30 Using NULL legacy PIC
05/03/2025 10:18:30 NR_IRQS: 33024, nr_irqs: 456, preallocated irqs: 0
05/03/2025 10:18:30 rcu: srcu_init: Setting srcu_struct sizes based on contention.
05/03/2025 10:18:30 kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)
05/03/2025 10:18:30 Console: colour dummy device 80x25
05/03/2025 10:18:30 printk: legacy console [ttyS0] enabled
05/03/2025 10:18:30 printk: legacy bootconsole [earlyser0] disabled
05/03/2025 10:18:30 ACPI: Core revision 20240827
05/03/2025 10:18:30 Failed to register legacy timer interrupt
05/03/2025 10:18:30 APIC: Switch to symmetric I/O mode setup
05/03/2025 10:18:30 Hyper-V: Host Build 10.0.20348.3030-3-0
05/03/2025 10:18:30 Hyper-V: enabling crash_kexec_post_notifiers
05/03/2025 10:18:30 Hyper-V: Using IPI hypercalls
05/03/2025 10:18:30 APIC: send_IPI() replaced with hv_send_ipi()
05/03/2025 10:18:30 APIC: send_IPI_mask() replaced with hv_send_ipi_mask()
05/03/2025 10:18:30 APIC: send_IPI_mask_allbutself() replaced with hv_send_ipi_mask_allbutself()
05/03/2025 10:18:30 APIC: send_IPI_allbutself() replaced with hv_send_ipi_allbutself()
05/03/2025 10:18:30 APIC: send_IPI_all() replaced with hv_send_ipi_all()
05/03/2025 10:18:30 APIC: send_IPI_self() replaced with hv_send_ipi_self()
05/03/2025 10:18:30 Hyper-V: Using enlightened APIC (xapic mode)
05/03/2025 10:18:30 APIC: eoi() replaced with hv_apic_eoi_write()
05/03/2025 10:18:30 APIC: read() replaced with hv_apic_read()
05/03/2025 10:18:30 APIC: write() replaced with hv_apic_write()
05/03/2025 10:18:30 APIC: icr_write() replaced with hv_apic_icr_write()
05/03/2025 10:18:30 APIC: icr_read() replaced with hv_apic_icr_read()
05/03/2025 10:18:30 Calibrating delay loop (skipped), value calculated using timer frequency.. 4890.85 BogoMIPS (lpj=9781708)
05/03/2025 10:18:30 x86/cpu: User Mode Instruction Prevention (UMIP) activated
05/03/2025 10:18:30 Last level iTLB entries: 4KB 512, 2MB 512, 4MB 256
05/03/2025 10:18:30 Last level dTLB entries: 4KB 2048, 2MB 2048, 4MB 1024, 1GB 0
05/03/2025 10:18:30 Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
05/03/2025 10:18:30 Spectre V2 : Mitigation: Retpolines
05/03/2025 10:18:30 Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
05/03/2025 10:18:30 Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
05/03/2025 10:18:30 Speculative Store Bypass: Vulnerable
05/03/2025 10:18:30 Speculative Return Stack Overflow: IBPB-extending microcode not applied!
05/03/2025 10:18:30 Speculative Return Stack Overflow: WARNING: See https://kernel.org/doc/html/latest/admin-guide/hw-vuln/srso.html for mitigation options.
05/03/2025 10:18:30 Speculative Return Stack Overflow: Vulnerable: Safe RET, no microcode
05/03/2025 10:18:30 x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
05/03/2025 10:18:30 x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
05/03/2025 10:18:30 x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
05/03/2025 10:18:30 x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
05/03/2025 10:18:30 x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'compacted' format.
05/03/2025 10:18:30 Freeing SMP alternatives memory: 64K
05/03/2025 10:18:30 pid_max: default: 32768 minimum: 301
05/03/2025 10:18:30 LSM: initializing lsm=lockdown,capability,yama,selinux,bpf,ima
05/03/2025 10:18:30 Yama: becoming mindful.
05/03/2025 10:18:30 SELinux:  Initializing.
05/03/2025 10:18:30 LSM support for eBPF active
05/03/2025 10:18:30 Mount-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
05/03/2025 10:18:30 Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
05/03/2025 10:18:30 smpboot: CPU0: AMD EPYC 7763 64-Core Processor (family: 0x19, model: 0x1, stepping: 0x1)
05/03/2025 10:18:30 Performance Events: PMU not available due to virtualization, using software events only.
05/03/2025 10:18:30 signal: max sigframe size: 1776
05/03/2025 10:18:30 rcu: Hierarchical SRCU implementation.
05/03/2025 10:18:30 rcu: 	Max phase no-delay instances is 1000.
05/03/2025 10:18:30 Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level
05/03/2025 10:18:30 smp: Bringing up secondary CPUs ...
05/03/2025 10:18:30 smpboot: x86: Booting SMP configuration:
05/03/2025 10:18:30 .... node  #0, CPUs:      #2 #1 #3
05/03/2025 10:18:31 smp: Brought up 1 node, 4 CPUs
05/03/2025 10:18:31 smpboot: Total of 4 processors activated (19584.64 BogoMIPS)
05/03/2025 10:18:31 Memory: 16235728K/16776068K available (28672K kernel code, 5603K rwdata, 16508K rodata, 5196K init, 1132K bss, 499120K reserved, 0K cma-reserved)
05/03/2025 10:18:31 devtmpfs: initialized
05/03/2025 10:18:31 x86/mm: Memory block size: 128MB
05/03/2025 10:18:31 ACPI: PM: Registering ACPI NVS region [mem 0x3fffb000-0x3fffefff] (16384 bytes)
05/03/2025 10:18:31 clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
05/03/2025 10:18:31 futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
05/03/2025 10:18:31 PM: RTC time: 09:18:28, date: 2025-03-05
05/03/2025 10:18:31 NET: Registered PF_NETLINK/PF_ROUTE protocol family
05/03/2025 10:18:31 audit: initializing netlink subsys (disabled)
05/03/2025 10:18:31 audit: type=2000 audit(1741166307.320:1): state=initialized audit_enabled=0 res=1
05/03/2025 10:18:31 thermal_sys: Registered thermal governor 'step_wise'
05/03/2025 10:18:31 thermal_sys: Registered thermal governor 'user_space'
05/03/2025 10:18:31 cpuidle: using governor menu
05/03/2025 10:18:31 acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
05/03/2025 10:18:31 dca service started, version 1.12.1
05/03/2025 10:18:31 e820: reserve RAM buffer [mem 0x3eec6000-0x3fffffff]
05/03/2025 10:18:31 e820: reserve RAM buffer [mem 0x3ff41000-0x3fffffff]
05/03/2025 10:18:31 kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
05/03/2025 10:18:31 HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
05/03/2025 10:18:31 HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
05/03/2025 10:18:31 HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
05/03/2025 10:18:31 HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
05/03/2025 10:18:31 cryptd: max_cpu_qlen set to 1000
05/03/2025 10:18:31 ACPI: Added _OSI(Module Device)
05/03/2025 10:18:31 ACPI: Added _OSI(Processor Device)
05/03/2025 10:18:31 ACPI: Added _OSI(3.0 _SCP Extensions)
05/03/2025 10:18:31 ACPI: Added _OSI(Processor Aggregator Device)
05/03/2025 10:18:31 ACPI: 1 ACPI AML tables successfully acquired and loaded
05/03/2025 10:18:31 ACPI: Interpreter enabled
05/03/2025 10:18:31 ACPI: PM: (supports S0 S5)
05/03/2025 10:18:31 ACPI: Using IOAPIC for interrupt routing
05/03/2025 10:18:31 PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
05/03/2025 10:18:31 PCI: Ignoring E820 reservations for host bridge windows
05/03/2025 10:18:31 ACPI: Enabled 1 GPEs in block 00 to 0F
05/03/2025 10:18:31 iommu: Default domain type: Translated
05/03/2025 10:18:31 iommu: DMA domain TLB invalidation policy: lazy mode
05/03/2025 10:18:31 SCSI subsystem initialized
05/03/2025 10:18:31 libata version 3.00 loaded.
05/03/2025 10:18:31 ACPI: bus type USB registered
05/03/2025 10:18:31 usbcore: registered new interface driver usbfs
05/03/2025 10:18:31 usbcore: registered new interface driver hub
05/03/2025 10:18:31 usbcore: registered new device driver usb
05/03/2025 10:18:31 pps_core: LinuxPPS API ver. 1 registered
05/03/2025 10:18:31 pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
05/03/2025 10:18:31 PTP clock support registered
05/03/2025 10:18:31 EDAC MC: Ver: 3.0.0
05/03/2025 10:18:31 efivars: Registered efivars operations
05/03/2025 10:18:31 hv_vmbus: Vmbus version:5.2
05/03/2025 10:18:31 NET: Registered PF_ATMPVC protocol family
05/03/2025 10:18:31 NET: Registered PF_ATMSVC protocol family
05/03/2025 10:18:31 NetLabel: Initializing
05/03/2025 10:18:31 NetLabel:  domain hash size = 128
05/03/2025 10:18:31 NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
05/03/2025 10:18:31 NetLabel:  unlabeled traffic allowed by default
05/03/2025 10:18:31 PCI: Using ACPI for IRQ routing
05/03/2025 10:18:31 PCI: System does not support PCI
05/03/2025 10:18:31 vgaarb: loaded
05/03/2025 10:18:31 clocksource: Switched to clocksource hyperv_clocksource_tsc_page
05/03/2025 10:18:31 VFS: Disk quotas dquot_6.6.0
05/03/2025 10:18:31 VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
05/03/2025 10:18:31 pnp: PnP ACPI init
05/03/2025 10:18:31 pnp: PnP ACPI: found 3 devices
05/03/2025 10:18:31 clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
05/03/2025 10:18:31 NET: Registered PF_INET protocol family
05/03/2025 10:18:31 IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear)
05/03/2025 10:18:31 tcp_listen_portaddr_hash hash table entries: 8192 (order: 5, 131072 bytes, linear)
05/03/2025 10:18:31 Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
05/03/2025 10:18:31 TCP established hash table entries: 131072 (order: 8, 1048576 bytes, linear)
05/03/2025 10:18:31 TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)
05/03/2025 10:18:31 TCP: Hash tables configured (established 131072 bind 65536)
05/03/2025 10:18:31 UDP hash table entries: 8192 (order: 6, 262144 bytes, linear)
05/03/2025 10:18:31 UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes, linear)
05/03/2025 10:18:31 NET: Registered PF_UNIX/PF_LOCAL protocol family
05/03/2025 10:18:31 RPC: Registered named UNIX socket transport module.
05/03/2025 10:18:31 RPC: Registered udp transport module.
05/03/2025 10:18:31 RPC: Registered tcp transport module.
05/03/2025 10:18:31 RPC: Registered tcp-with-tls transport module.
05/03/2025 10:18:31 RPC: Registered tcp NFSv4.1 backchannel transport module.
05/03/2025 10:18:31 NET: Registered PF_XDP protocol family
05/03/2025 10:18:31 PCI: CLS 0 bytes, default 64
05/03/2025 10:18:31 PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
05/03/2025 10:18:31 Unpacking initramfs...
05/03/2025 10:18:31 software IO TLB: mapped [mem 0x000000003aec6000-0x000000003eec6000] (64MB)
05/03/2025 10:18:31 kvm_intel: VMX not supported by CPU 1
05/03/2025 10:18:31 kvm_amd: SVM not supported by CPU 1
05/03/2025 10:18:31 Initialise system trusted keyrings
05/03/2025 10:18:31 Key type blacklist registered
05/03/2025 10:18:31 workingset: timestamp_bits=40 max_order=22 bucket_order=0
05/03/2025 10:18:31 squashfs: version 4.0 (2009/01/31) Phillip Lougher
05/03/2025 10:18:31 NFS: Registering the id_resolver key type
05/03/2025 10:18:31 Key type id_resolver registered
05/03/2025 10:18:31 Key type id_legacy registered
05/03/2025 10:18:31 nfs4filelayout_init: NFSv4 File Layout Driver Registering...
05/03/2025 10:18:31 nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
05/03/2025 10:18:31 Key type cifs.spnego registered
05/03/2025 10:18:31 Key type cifs.idmap registered
05/03/2025 10:18:31 fuse: init (API version 7.41)
05/03/2025 10:18:31 SGI XFS with ACLs, security attributes, scrub, repair, quota, no debug enabled
05/03/2025 10:18:31 ceph: loaded (mds proto 32)
05/03/2025 10:18:31 integrity: Platform Keyring initialized
05/03/2025 10:18:31 Freeing initrd memory: 74092K
05/03/2025 10:18:31 NET: Registered PF_ALG protocol family
05/03/2025 10:18:31 Key type asymmetric registered
05/03/2025 10:18:31 Asymmetric key parser 'x509' registered
05/03/2025 10:18:31 Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
05/03/2025 10:18:31 io scheduler mq-deadline registered
05/03/2025 10:18:31 io scheduler kyber registered
05/03/2025 10:18:31 io scheduler bfq registered
05/03/2025 10:18:31 hv_vmbus: registering driver hv_pci
05/03/2025 10:18:31 hv_vmbus: registering driver hyperv_fb
05/03/2025 10:18:31 IPMI message handler: version 39.2
05/03/2025 10:18:31 hyperv_fb: Synthvid Version major 3, minor 5
05/03/2025 10:18:31 ipmi device interface
05/03/2025 10:18:31 hyperv_fb: Screen resolution: 1024x768, Color depth: 32, Frame buffer size: 8388608
05/03/2025 10:18:31 ipmi_si: IPMI System Interface driver
05/03/2025 10:18:31 ipmi_si: Unable to find any System Interface(s)
05/03/2025 10:18:31 IPMI poweroff: Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot
05/03/2025 10:18:31 Console: switching to colour frame buffer device 128x48
05/03/2025 10:18:31 ioatdma: Intel(R) QuickData Technology Driver 5.00
05/03/2025 10:18:31 Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
05/03/2025 10:18:31 00:00: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
05/03/2025 10:18:31 00:01: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
05/03/2025 10:18:31 Non-volatile memory driver v1.3
05/03/2025 10:18:31 Linux agpgart interface v0.103
05/03/2025 10:18:31 ACPI: bus type drm_connector registered
05/03/2025 10:18:31 loop: module loaded
05/03/2025 10:18:31 rbd: loaded (major 252)
05/03/2025 10:18:31 Guest personality initialized and is inactive
05/03/2025 10:18:31 VMCI host device registered (name=vmci, major=10, minor=126)
05/03/2025 10:18:31 Initialized host personality
05/03/2025 10:18:31 Loading iSCSI transport class v2.0-870.
05/03/2025 10:18:31 iscsi: registered transport (tcp)
05/03/2025 10:18:31 hv_vmbus: registering driver hv_storvsc
05/03/2025 10:18:31 scsi host1: storvsc_host_t
 SUBSYSTEM=scsi
 DEVICE=+scsi:host1
05/03/2025 10:18:31 wireguard: WireGuard 1.0.0 loaded. See www.wireguard.com for information.
05/03/2025 10:18:31 scsi host0: storvsc_host_t
 SUBSYSTEM=scsi
 DEVICE=+scsi:host0
05/03/2025 10:18:31 wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
05/03/2025 10:18:31 tun: Universal TUN/TAP device driver, 1.6
05/03/2025 10:18:31 scsi 0:0:0:0: Direct-Access     Msft     Virtual Disk     1.0  PQ: 0 ANSI: 5
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:0
05/03/2025 10:18:31 hv_vmbus: registering driver hv_netvsc
05/03/2025 10:18:31 sd 0:0:0:0: Attached scsi generic sg0 type 0
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:0
05/03/2025 10:18:31 sd 0:0:0:0: [sda] 134217728 512-byte logical blocks: (68.7 GB/64.0 GiB)
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:0
05/03/2025 10:18:31 usbcore: registered new interface driver cdc_acm
05/03/2025 10:18:31 cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
05/03/2025 10:18:31 usbcore: registered new interface driver uas
05/03/2025 10:18:31 sd 0:0:0:0: [sda] 4096-byte physical blocks
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:0
05/03/2025 10:18:31 sd 0:0:0:0: [sda] Write Protect is off
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:0
05/03/2025 10:18:31 sd 0:0:0:0: [sda] Mode Sense: 0f 00 10 00
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:0
05/03/2025 10:18:31 sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:0
05/03/2025 10:18:31 scsi 0:0:0:1: Direct-Access     Msft     Virtual Disk     1.0  PQ: 0 ANSI: 5
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:1
05/03/2025 10:18:31 usbcore: registered new interface driver usb-storage
05/03/2025 10:18:31 usbcore: registered new interface driver usbserial_generic
05/03/2025 10:18:31 usbserial: USB Serial support registered for generic
05/03/2025 10:18:31 i8042: PNP: No PS/2 controller found.
05/03/2025 10:18:31 hv_vmbus: registering driver hyperv_keyboard
05/03/2025 10:18:31 mousedev: PS/2 mouse device common for all mice
05/03/2025 10:18:31 hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#3 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
 SUBSYSTEM=vmbus
 DEVICE=+vmbus:f8b3781a-1e82-4818-a1c3-63d806ec15bb
05/03/2025 10:18:32 hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#10 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
 SUBSYSTEM=vmbus
 DEVICE=+vmbus:f8b3781a-1e82-4818-a1c3-63d806ec15bb
05/03/2025 10:18:32 rtc_cmos 00:02: registered as rtc0
 SUBSYSTEM=pnp
 DEVICE=+pnp:00:02
05/03/2025 10:18:32 rtc_cmos 00:02: alarms up to one month, 114 bytes nvram
 SUBSYSTEM=pnp
 DEVICE=+pnp:00:02
05/03/2025 10:18:32 input: AT Translated Set 2 keyboard as /devices/LNXSYSTM:00/LNXSYBUS:00/ACPI0004:00/VMBUS:00/d34b2567-b9b6-42b9-8778-0a4ec0b955bf/serio0/input/input0
05/03/2025 10:18:32 sd 0:0:0:1: Attached scsi generic sg1 type 0
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:1
05/03/2025 10:18:32 sd 0:0:0:1: [sdb] 67108864 512-byte logical blocks: (34.4 GB/32.0 GiB)
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:1
05/03/2025 10:18:32 device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
05/03/2025 10:18:32 sd 0:0:0:1: [sdb] 4096-byte physical blocks
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:1
05/03/2025 10:18:32 sd 0:0:0:1: [sdb] Write Protect is off
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:1
05/03/2025 10:18:32 sd 0:0:0:1: [sdb] Mode Sense: 0f 00 10 00
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:1
05/03/2025 10:18:32 sd 0:0:0:1: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:1
05/03/2025 10:18:32 hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#15 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
 SUBSYSTEM=vmbus
 DEVICE=+vmbus:f8b3781a-1e82-4818-a1c3-63d806ec15bb
05/03/2025 10:18:32 hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#22 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
 SUBSYSTEM=vmbus
 DEVICE=+vmbus:f8b3781a-1e82-4818-a1c3-63d806ec15bb
05/03/2025 10:18:32 device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: dm-devel@lists.linux.dev
05/03/2025 10:18:32 amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled
05/03/2025 10:18:32  sdb: sdb1
05/03/2025 10:18:32  sda: sda1 sda2 sda3 sda4 sda5 sda6
05/03/2025 10:18:32 sd 0:0:0:0: [sda] Attached SCSI disk
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:0
05/03/2025 10:18:32 sdhci: Secure Digital Host Controller Interface driver
05/03/2025 10:18:32 sd 0:0:0:1: [sdb] Attached SCSI disk
 SUBSYSTEM=scsi
 DEVICE=+scsi:0:0:0:1
05/03/2025 10:18:32 sdhci: Copyright(c) Pierre Ossman
05/03/2025 10:18:32 hid: raw HID events driver (C) Jiri Kosina
05/03/2025 10:18:32 usbcore: registered new interface driver usbhid
05/03/2025 10:18:32 usbhid: USB HID core driver
05/03/2025 10:18:32 hv_utils: Registering HyperV Utility Driver
05/03/2025 10:18:32 hv_vmbus: registering driver hv_utils
05/03/2025 10:18:32 hv_vmbus: registering driver hv_balloon
05/03/2025 10:18:32 hv_utils: Heartbeat IC version 3.0
05/03/2025 10:18:32 NET: Registered PF_LLC protocol family
05/03/2025 10:18:32 hv_utils: Shutdown IC version 3.2
05/03/2025 10:18:32 GACT probability NOT on
05/03/2025 10:18:32 hv_utils: TimeSync IC version 4.0
05/03/2025 10:18:32 Mirror/redirect action on
05/03/2025 10:18:32 hv_balloon: Using Dynamic Memory protocol version 2.0
05/03/2025 10:18:32 Simple TC action Loaded
05/03/2025 10:18:32 netem: version 1.3
05/03/2025 10:18:32 u32 classifier
05/03/2025 10:18:32     input device check on
05/03/2025 10:18:32     Actions configured
05/03/2025 10:18:32 xt_time: kernel timezone is -0000
05/03/2025 10:18:32 IPVS: Registered protocols (TCP, UDP)
05/03/2025 10:18:32 IPVS: Connection hash table configured (size=4096, memory=32Kbytes)
05/03/2025 10:18:32 IPVS: ipvs loaded.
05/03/2025 10:18:32 IPVS: [rr] scheduler registered.
05/03/2025 10:18:32 IPVS: [wrr] scheduler registered.
05/03/2025 10:18:32 IPVS: [lc] scheduler registered.
05/03/2025 10:18:32 IPVS: [sh] scheduler registered.
05/03/2025 10:18:32 ipip: IPv4 and MPLS over IPv4 tunneling driver
05/03/2025 10:18:32 gre: GRE over IPv4 demultiplexor driver
05/03/2025 10:18:32 Initializing XFRM netlink socket
05/03/2025 10:18:32 NET: Registered PF_INET6 protocol family
05/03/2025 10:18:32 Segment Routing with IPv6
05/03/2025 10:18:32 In-situ OAM (IOAM) with IPv6
05/03/2025 10:18:32 mip6: Mobile IPv6
05/03/2025 10:18:32 sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
05/03/2025 10:18:32 NET: Registered PF_PACKET protocol family
05/03/2025 10:18:32 Bridge firewalling registered
05/03/2025 10:18:32 NET: Registered PF_X25 protocol family
05/03/2025 10:18:32 X25: Linux Version 0.2
05/03/2025 10:18:32 RPC: Registered rdma transport module.
05/03/2025 10:18:32 RPC: Registered rdma backchannel transport module.
05/03/2025 10:18:32 l2tp_core: L2TP core driver, V2.0
05/03/2025 10:18:32 NET: Registered PF_PHONET protocol family
05/03/2025 10:18:32 8021q: 802.1Q VLAN Support v1.8
05/03/2025 10:18:32 DCCP: Activated CCID 2 (TCP-like)
05/03/2025 10:18:32 DCCP: Activated CCID 3 (TCP-Friendly Rate Control)
05/03/2025 10:18:32 DCCP is deprecated and scheduled to be removed in 2025, please contact the netdev mailing list
05/03/2025 10:18:32 sctp: Hash tables configured (bind 256/256)
05/03/2025 10:18:32 NET: Registered PF_RDS protocol family
05/03/2025 10:18:32 Key type dns_resolver registered
05/03/2025 10:18:32 Key type ceph registered
05/03/2025 10:18:32 libceph: loaded (mon/osd proto 15/24)
05/03/2025 10:18:32 openvswitch: Open vSwitch switching datapath
05/03/2025 10:18:32 NET: Registered PF_VSOCK protocol family
05/03/2025 10:18:32 hv_vmbus: registering driver hv_sock
05/03/2025 10:18:32 hv_vmbus: unregistering driver hv_sock
05/03/2025 10:18:32 mpls_gso: MPLS GSO support
05/03/2025 10:18:32 IPI shorthand broadcast: enabled
05/03/2025 10:18:32 AES CTR mode by8 optimization enabled
05/03/2025 10:18:32 sched_clock: Marking stable (1924005700, 75487200)->(2195078700, -195585800)
05/03/2025 10:18:32 registered taskstats version 1
05/03/2025 10:18:32 Loading compiled-in X.509 certificates
05/03/2025 10:18:32 Loaded X.509 cert 'Sidero Labs, Inc.: Build time throw-away kernel key: 2ac1cb1ac1f7a0c7f4be5f964407f7b37436564a'
05/03/2025 10:18:32 Demotion targets for Node 0: null
05/03/2025 10:18:32 integrity: Loading X.509 certificate: UEFI:db
05/03/2025 10:18:32 integrity: Loaded X.509 cert 'Microsoft Windows Production PCA 2011: a92902398e16c49778cd90f99e4f9ae17c55af53'
05/03/2025 10:18:32 ima: No TPM chip found, activating TPM-bypass!
05/03/2025 10:18:32 ima: Allocated hash algorithm: sha512
05/03/2025 10:18:32 ima: No architecture policies found
05/03/2025 10:18:32 PM:   Magic number: 1:580:322
05/03/2025 10:18:32 tty tty47: hash matches
 SUBSYSTEM=tty
 DEVICE=c4:47
05/03/2025 10:18:32 printk: legacy console [netcon0] enabled
05/03/2025 10:18:32 netconsole: network logging started
05/03/2025 10:18:32 clk: Disabling unused clocks
05/03/2025 10:18:32 Freeing unused kernel image (initmem) memory: 5196K
05/03/2025 10:18:32 Write protecting the kernel read-only data: 47104k
05/03/2025 10:18:32 Freeing unused kernel image (rodata/data gap) memory: 1924K
05/03/2025 10:18:32 x86/mm: Checked W+X mappings: passed, no W+X pages found.
05/03/2025 10:18:32 x86/mm: Checking user space page tables
05/03/2025 10:18:32 x86/mm: Checked W+X mappings: passed, no W+X pages found.
05/03/2025 10:18:32 Run /init as init process
05/03/2025 10:18:32   with arguments:
05/03/2025 10:18:32     /init
05/03/2025 10:18:32   with environment:
05/03/2025 10:18:32     HOME=/
05/03/2025 10:18:32     TERM=linux
05/03/2025 10:18:32     BOOT_IMAGE=/A/vmlinuz
05/03/2025 10:18:32 [talos] [initramfs] TPM device is not available
05/03/2025 10:18:32 [talos] [initramfs] TPM device is not available, skipping PCR extension
05/03/2025 10:18:32 [talos] [initramfs] booting Talos v1.9.4
05/03/2025 10:18:32 [talos] [initramfs] CPU: AMD EPYC 7763 64-Core Processor, 2 core(s), 2 thread(s) per core
05/03/2025 10:18:32 [talos] [initramfs] x86_64 microarchitecture level: 3
05/03/2025 10:18:32 [talos] [initramfs] mounting the rootfs
05/03/2025 10:18:32 [talos] [initramfs] enabling system extension schematic 05ea230d881327f05c73d3dd7ec2def1808b8d13d680fbda8adef45b52982436
05/03/2025 10:18:32 [talos] [initramfs] enabling system extension intel-ice-firmware 20250109
05/03/2025 10:18:32 loop0: detected capacity change from 0 to 8
05/03/2025 10:18:32 loop1: detected capacity change from 0 to 8
05/03/2025 10:18:32 loop2: detected capacity change from 0 to 143936
05/03/2025 10:18:32 [talos] [initramfs] bind mounting /lib/firmware
05/03/2025 10:18:32 [talos] [initramfs] entering the rootfs
05/03/2025 10:18:32 [talos] [initramfs] moving mounts to the new rootfs
05/03/2025 10:18:32 [talos] [initramfs] changing working directory into /root
05/03/2025 10:18:32 [talos] [initramfs] moving /root to /
05/03/2025 10:18:32 [talos] [initramfs] changing root directory
05/03/2025 10:18:32 [talos] [initramfs] cleaning up initramfs
05/03/2025 10:18:32 [talos] [initramfs] selinux: disabled, not loading policy
05/03/2025 10:18:32 [talos] [initramfs] TPM device is not available, skipping PCR extension
05/03/2025 10:18:32 [talos] [initramfs] executing /sbin/init
05/03/2025 10:18:32 [talos] machined logger initialized
05/03/2025 10:18:32 [talos] platform information {"component": "early-startup", "mode": "cloud"}
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:2): action=dont_measure fsmagic=0x9fa0 res=1
05/03/2025 10:18:32 ima: policy update completed
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:3): action=dont_measure fsmagic=0x62656572 res=1
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:4): action=dont_measure fsmagic=0x64626720 res=1
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:5): action=dont_measure fsmagic=0x1021994 res=1
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:6): action=dont_measure fsmagic=0x1cd1 res=1
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:7): action=dont_measure fsmagic=0x42494e4d res=1
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:8): action=dont_measure fsmagic=0x73636673 res=1
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:9): action=dont_measure fsmagic=0xf97cff8c res=1
05/03/2025 10:18:32 audit: type=1807 audit(1741166312.752:10): action=dont_measure fsmagic=0x43415d53 res=1
05/03/2025 10:18:32 [talos] initializing cgroups {"component": "early-startup", "root": "/"}
05/03/2025 10:18:32 [talos] early startup done {"component": "early-startup", "duration": "86.5959ms"}
05/03/2025 10:18:32 [talos] initialize sequence: 10 phase(s)
05/03/2025 10:18:32 [talos] phase systemRequirements (1/10): 1 tasks(s)
05/03/2025 10:18:32 [talos] task enforceKSPPRequirements (1/1): starting
05/03/2025 10:18:32 [talos] fetching azure instance config from: "http://169.254.169.254/metadata/instance/compute?api-version=2021-12-13&format=json"
05/03/2025 10:18:32 [talos] assigned address {"component": "controller-runtime", "controller": "network.AddressSpecController", "address": "127.0.0.1/8", "link": "lo"}
05/03/2025 10:18:32 [talos] retrying error: failed to download config from "http://169.254.169.254/metadata/instance/compute?api-version=2021-12-13&format=json": Get "http://169.254.169.254/metadata/instance/compute?api-version=2021-12-13&format=json": dial tcp 169.254.169.254:80: connect: network is unreachable
05/03/2025 10:18:32 [talos] pre-created iptables-nft table 'mangle'/'KUBE-IPTABLES-HINT' {"component": "controller-runtime", "controller": "network.NfTablesChainController"}
05/03/2025 10:18:32 [talos] setting resolvers {"component": "controller-runtime", "controller": "network.ResolverSpecController", "resolvers": ["1.1.1.1", "8.8.8.8"], "searchDomains": []}
05/03/2025 10:18:32 [talos] setting resolvers {"component": "controller-runtime", "controller": "network.ResolverSpecController", "resolvers": ["1.1.1.1", "8.8.8.8"], "searchDomains": []}
05/03/2025 10:18:32 [talos] task enforceKSPPRequirements (1/1): done, 17.0925ms
05/03/2025 10:18:32 [talos] setting time servers {"component": "controller-runtime", "controller": "network.TimeServerSpecController", "addresses": ["time.cloudflare.com"]}
05/03/2025 10:18:32 [talos] nftables chains updated {"component": "controller-runtime", "controller": "network.NfTablesChainController", "chains": []}
05/03/2025 10:18:32 [talos] phase systemRequirements (1/10): done, 50.5741ms
05/03/2025 10:18:32 [talos] setting time servers {"component": "controller-runtime", "controller": "network.TimeServerSpecController", "addresses": ["time.cloudflare.com"]}
05/03/2025 10:18:32 [talos] phase earlyServices (2/10): 5 tasks(s)
05/03/2025 10:18:32 [talos] task startContainerd (5/5): starting
05/03/2025 10:18:32 [talos] task startUdevd (1/5): starting
05/03/2025 10:18:32 [talos] task startAuditd (3/5): starting
05/03/2025 10:18:32 [talos] task startSyslogd (4/5): starting
05/03/2025 10:18:32 [talos] task startMachined (2/5): starting
05/03/2025 10:18:32 [talos] TPM device is not available, skipping PCR extension
05/03/2025 10:18:32 [talos] service[containerd](Starting): Starting service
05/03/2025 10:18:32 [talos] service[containerd](Preparing): Running pre state
05/03/2025 10:18:32 [talos] service[containerd](Preparing): Creating service runner
05/03/2025 10:18:32 [talos] service[udevd](Starting): Starting service
05/03/2025 10:18:32 [talos] service[udevd](Preparing): Running pre state
05/03/2025 10:18:32 [talos] service[machined](Starting): Starting service
05/03/2025 10:18:32 [talos] service[machined](Preparing): Running pre state
05/03/2025 10:18:32 [talos] service[machined](Preparing): Creating service runner
05/03/2025 10:18:32 [talos] service[auditd](Starting): Starting service
05/03/2025 10:18:32 [talos] service[auditd](Preparing): Running pre state
05/03/2025 10:18:32 [talos] service[machined](Running): Service started as goroutine
05/03/2025 10:18:32 [talos] task startAuditd (3/5): done, 29.7205ms
05/03/2025 10:18:32 [talos] service[syslogd](Starting): Starting service
05/03/2025 10:18:32 [talos] service[auditd](Preparing): Creating service runner
05/03/2025 10:18:32 [talos] service[udevd](Preparing): Creating service runner
05/03/2025 10:18:32 [talos] service[syslogd](Waiting): Waiting for service "machined" to be "up"
05/03/2025 10:18:32 [talos] task startSyslogd (4/5): done, 40.0583ms
05/03/2025 10:18:33 [talos] service[containerd](Running): Process Process(["/bin/containerd" "--address" "/system/run/containerd/containerd.sock" "--state" "/system/run/containerd" "--root" "/system/var/lib/containerd"]) started with PID 1179
05/03/2025 10:18:33 [talos] service[auditd](Running): Service started as goroutine
05/03/2025 10:18:33 [talos] service[udevd](Running): Process Process(["/sbin/systemd-udevd" "--resolve-names=never"]) started with PID 1183
05/03/2025 10:18:33 [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "META", "phase": "waiting -> ready", "location": "/dev/sda4", "parentLocation": "/dev/sda"}
05/03/2025 10:18:33 [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "waiting -> failed", "error": "volume is encrypted, but no encryption config provided", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
05/03/2025 10:18:33 [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "volume is encrypted, but no encryption config provided", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
05/03/2025 10:18:33 [talos] service[udevd](Running): Health check successful
05/03/2025 10:18:33 [talos] task startUdevd (1/5): done, 390.5929ms
05/03/2025 10:18:33 [talos] setting resolvers {"component": "controller-runtime", "controller": "network.ResolverSpecController", "resolvers": ["168.63.129.16"], "searchDomains": []}
05/03/2025 10:18:33 [talos] assigned address {"component": "controller-runtime", "controller": "network.AddressSpecController", "address": "100.64.2.9/23", "link": "eth0"}
05/03/2025 10:18:33 [talos] created route {"component": "controller-runtime", "controller": "network.RouteSpecController", "destination": "default", "gateway": "100.64.2.1", "table": "main", "link": "eth0", "priority": 1024, "family": "inet4"}
05/03/2025 10:18:33 [talos] created route {"component": "controller-runtime", "controller": "network.RouteSpecController", "destination": "168.63.129.16/32", "gateway": "100.64.2.1", "table": "main", "link": "eth0", "priority": 1024, "family": "inet4"}
05/03/2025 10:18:33 [talos] created route {"component": "controller-runtime", "controller": "network.RouteSpecController", "destination": "169.254.169.254/32", "gateway": "100.64.2.1", "table": "main", "link": "eth0", "priority": 1024, "family": "inet4"}
05/03/2025 10:18:33 [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "volume is encrypted, but no encryption config provided", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
05/03/2025 10:18:33 [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "volume is encrypted, but no encryption config provided", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
05/03/2025 10:18:33 [talos] service[containerd](Running): Health check successful
05/03/2025 10:18:33 [talos] task startContainerd (5/5): done, 1.0223757s
05/03/2025 10:18:33 [talos] service[machined](Running): Health check successful
05/03/2025 10:18:33 [talos] service[syslogd](Preparing): Running pre state
05/03/2025 10:18:33 [talos] service[syslogd](Preparing): Creating service runner
05/03/2025 10:18:33 [talos] task startMachined (2/5): done, 1.0310148s
05/03/2025 10:18:33 [talos] service[syslogd](Running): Service started as goroutine
05/03/2025 10:18:33 [talos] phase earlyServices (2/10): done, 1.043719s
05/03/2025 10:18:33 [talos] phase usb (3/10): 1 tasks(s)
05/03/2025 10:18:33 [talos] service[auditd](Running): Health check successful
05/03/2025 10:18:33 [talos] task waitForUSB (1/1): starting
05/03/2025 10:18:33 [talos] task waitForUSB (1/1): waiting 1 second(s) for USB storage
05/03/2025 10:18:34 [talos] adjusting time (slew) by 1.628927ms via 162.159.200.123, state TIME_OK, status STA_PLL | STA_NANO {"component": "controller-runtime", "controller": "time.SyncController"}
05/03/2025 10:18:34 [talos] service[syslogd](Running): Health check successful
05/03/2025 10:18:35 [talos] task waitForUSB (1/1): done, 1.0074043s
05/03/2025 10:18:35 [talos] phase usb (3/10): done, 1.0112465s
05/03/2025 10:18:35 [talos] phase meta (4/10): 1 tasks(s)
05/03/2025 10:18:35 [talos] task reloadMeta (1/1): starting
05/03/2025 10:18:35 [talos] fetching network config from "http://169.254.169.254/metadata/instance/network/interface?api-version=2021-12-13&format=json"
05/03/2025 10:18:35 [talos] fetching load balancer metadata from: "http://169.254.169.254/metadata/loadbalancer?api-version=2021-12-13&format=json"
05/03/2025 10:18:35 [talos] setting hostname {"component": "controller-runtime", "controller": "network.HostnameSpecController", "hostname": "vm-francecentral-1-azure-francecentral-cp-ephemeral-2", "domainname": ""}
05/03/2025 10:18:35 [talos] setting hostname {"component": "controller-runtime", "controller": "network.HostnameSpecController", "hostname": "vm-francecentral-1-azure-francecentral-cp-ephemeral-2", "domainname": ""}
05/03/2025 10:18:35 [talos] META: loading from /dev/sda4
05/03/2025 10:18:35 [talos] META: loaded 1 keys
05/03/2025 10:18:35 [talos] task reloadMeta (1/1): done, 106.620028ms
05/03/2025 10:18:35 [talos] phase meta (4/10): done, 110.961536ms
05/03/2025 10:18:35 [talos] phase mountSystem (8/10): 1 tasks(s)
05/03/2025 10:18:35 [talos] task mountStatePartition (1/1): starting
05/03/2025 10:18:35 [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "volume is encrypted, but no encryption config provided", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
05/03/2025 10:18:35 [talos] configuring siderolink connection {"component": "controller-runtime", "controller": "siderolink.ManagerController", "peer_endpoint": "136.144.56.184:10068", "next_peer_endpoint": ""}
05/03/2025 10:18:35 [talos] siderolink connection configured {"component": "controller-runtime", "controller": "siderolink.ManagerController", "endpoint": "https://<redacted>.siderolink.omni.siderolabs.io?grpc_tunnel=false&jointoken=<redacted>", "node_uuid": "080928fe-c01c-4be0-a476-c0d1fe3dd4d0", "node_address": "fdae:41e4:649b:9303:8ccf:3fff:d2ef:3511/64"}
05/03/2025 10:18:35 [talos] created new link {"component": "controller-runtime", "controller": "network.LinkSpecController", "link": "siderolink", "kind": "wireguard"}
05/03/2025 10:18:35 [talos] assigned address {"component": "controller-runtime", "controller": "network.AddressSpecController", "address": "fdae:41e4:649b:9303:8ccf:3fff:d2ef:3511/64", "link": "siderolink"}
05/03/2025 10:18:35 [talos] reconfigured wireguard link {"component": "controller-runtime", "controller": "network.LinkSpecController", "link": "siderolink", "peers": 1}
05/03/2025 10:18:35 [talos] changed MTU for the link {"component": "controller-runtime", "controller": "network.LinkSpecController", "link": "siderolink", "mtu": 1280}
05/03/2025 10:19:20 hv_balloon: Max. dynamic memory size: 16384 MB
05/03/2025 10:38:32 [talos] controller failed {"component": "controller-runtime", "controller": "v1alpha1.EventsSinkController", "error": "error publishing event: rpc error: code = Unavailable desc = error reading from server: read tcp [fdae:41e4:649b:9303:8ccf:3fff:d2ef:3511]:38278->[fdae:41e4:649b:9303::1]:8090: read: connection reset by peer"}

Another machine in the same problem seems to loop over this:

[80555.350200] [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "error opening encrypted volume: no handlers available to get encryption keys from: 1 error occurred:\n\t* encryption key rejected\n\n", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
[80555.350860] hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#66 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
[80555.383170]  sda: sda1 sda2 sda3 sda4 sda5 sda6
[80582.019895] [talos] opening encrypted volume {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE"}
[80585.367922] [talos] failed to call key handler {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "slot": 0, "error": "encryption key rejected"}
[80585.372379] [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "error opening encrypted volume: no handlers available to get encryption keys from: 1 error occurred:\n\t* encryption key rejected\n\n", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
[80585.373045] hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#18 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
[80585.402915]  sda: sda1 sda2 sda3 sda4 sda5 sda6
[80612.020768] [talos] opening encrypted volume {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE"}
[80615.352810] [talos] failed to call key handler {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "slot": 0, "error": "encryption key rejected"}
[80615.357776] [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "error opening encrypted volume: no handlers available to get encryption keys from: 1 error occurred:\n\t* encryption key rejected\n\n", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
[80615.358458] hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#283 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
[80615.387286]  sda: sda1 sda2 sda3 sda4 sda5 sda6
[80642.020194] [talos] opening encrypted volume {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE"}
[80645.348606] [talos] failed to call key handler {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "slot": 0, "error": "encryption key rejected"}
[80645.353372] [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "error opening encrypted volume: no handlers available to get encryption keys from: 1 error occurred:\n\t* encryption key rejected\n\n", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
[80645.361960] hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#65 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
[80645.395109]  sda: sda1 sda2 sda3 sda4 sda5 sda6
[80672.020638] [talos] opening encrypted volume {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE"}
[80675.356465] [talos] failed to call key handler {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "slot": 0, "error": "encryption key rejected"}
[80675.361409] [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "error opening encrypted volume: no handlers available to get encryption keys from: 1 error occurred:\n\t* encryption key rejected\n\n", "location": "/dev/sda5", "parentLocation": "/dev/sda"}
[80675.362154] hv_storvsc f8b3781a-1e82-4818-a1c3-63d806ec15bb: tag#308 cmd 0x5a status: scsi 0x2 srb 0x86 hv 0xc0000001
[80675.410838]  sda: sda1 sda2 sda3 sda4 sda5 sda6
[80702.020293] [talos] opening encrypted volume {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE"}
[80705.376842] [talos] failed to call key handler {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "slot": 0, "error": "encryption key rejected"}
[80705.382850] [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "error opening encrypted volume: no handlers available to get encryption keys from: 1 error occurred:\n\t* encryption key rejected\n\n", "location": "/dev/sda5", "parentLocation": "/dev/sda"}

Expected Behavior

Graceful cluster deletion.

Steps To Reproduce

1. Create a cluster with encrypted VM on Azure
2. Shutdown the machine on Azure only
3. Delete the cluster
4. Restart the machine on Azure

What browsers are you seeing the problem on?

No response

Anything else?

No response

[smira]

It's not clear from the description, but does the machine lose its disk contents on shutdown? If it does, it's expected that Omni won't handle it without force deletion.

[lerenn]

The eviction policy of Azure spot instance is on Deallocate, so the disk is preserved between shutdowns.

[smira]

If the disk is persisted, nothing should happen at all. Neither Omni or Talos cares whether a VM got shutdown.

Machine UUID should persist, but in your logs it seems more like the disk is lost, or machine UUID changed, or both.

[lerenn]

On Azure, it seems to be the same disk. What makes you think that the disk is different ?

[smira]

I have no idea, but your issue calls for it. If the machine doesn't lose/change anything on shutdown/power up, there will be no issue, so something is happening to your VMs.

You might be installing Talos to ephemeral local storage vs. network-attached storage. Hard to guess.

[lerenn]

I'll investigate further ! Thanks for the info :)

[lerenn]

I can confirm that Talos is installed on network-attached storage that is preserved between machine shutdowns.

Data is not destroyed.

The interesting line is the following:

[    4.689933] [talos] volume status {"component": "controller-runtime", "controller": "block.VolumeManagerController", "volume": "STATE", "phase": "failed -> failed", "error": "volume is encrypted, but no encryption config provided", "location": "/dev/sda5", "parentLocation": "/dev/sda"}

We still have the problem between normal reboots and we are using Omni KMS.

[smira]

This is a "normal" error which happens when the machine boots up, it doesn't have META discovered yet, so doesn't know how to decrypt STATE. It should transition from this to actual unlock.