cloud-config complications

[max06]

Good morning,

I'm currently trying some basic cloud-config options in my MachineRegistration. The issue: Sometimes things work, sometimes not. I want to know if I'm just too stupid to read and follow your documentation or if I'm actually hitting bugs.

Sadly debugging gets complicated when things are not working - that also prevents me from logging in due to no configured password.

apiVersion: elemental.cattle.io/v1beta1
kind: MachineRegistration
metadata:
  name: cluster-002
  namespace: fleet-default
spec:
  config:
    cloud-config:
      users:
      - name: root
        passwd: topsecret
      ssh_authorized_keys:
      - ssh-ed25519 AAAAC...QKCU1RR user1
      - ssh-ed25519 AAAAC...nwZkfEYQ user2
    elemental:
      install:
        device-selector:
        - key: Name
          operator: In
          values:
          - /dev/sda
          - /dev/vda
          - /dev/nvme0
        - key: Size
          operator: Gt
          values:
          - 25Gi
        reboot: true
        snapshotter:
          type: btrfs
      registration:
        auth: tpm
        emulate-tpm: true
        emulated-tpm-seed: -1
      reset:
        enabled: true
        reboot: true
        reset-oem: true
        reset-persistent: true
  machineInventoryLabels:
    elemental.cattle.io/CpuTotalCores: ${System Data/CPU/Total Cores}
    elemental.cattle.io/CpuTotalThreads: ${System Data/CPU/Total Threads}
    elemental.cattle.io/NumDisks: ${System Data/Block Devices/Number Devices}
    elemental.cattle.io/TotalMemoryBytes: ${System Data/Memory/Total Physical Bytes}
    elemental.cattle.io/edge: cluster-002
    elemental.cattle.io/machineUUID: ${System Information/UUID}
    elemental.cattle.io/manufacturer: ${System Information/Manufacturer}
    elemental.cattle.io/productName: ${System Information/Product Name}
    elemental.cattle.io/serialNumber: ${System Information/Serial Number}
  machineName: ${System Data/Runtime/Hostname}.cluster-002

This is being deployed (and working!) on physical hardware. It's also pure cloud-init.

---

Let's try the same but with pure yip syntax:

...
spec:
  config:
    cloud-config:
      stages:
        boot:
        - name: "Setup users"
          users: 
            root: 
              passwd: "topsecret"
              homedir: "/root"
        - name: "Setup ssh keys"
          authorized_keys:
            root:
              - id25519 ...

This works, but causes root to become a bit schizophrenic: It suddenly has a new uid and gid (65335), while also existing with uid 0. I can login using my ssh key, but I can not read ~/.ssh/authorized_keys after logging in.

---

Using a wrong cloud-init config - for example due to a users groups not being defined as a list like demonstrated in the documentation - causes the provisioning to fail with a weird "could not find xy.yaml" error repeating over and over again.

---

Using mixed configuration options, cloud-init and yip at the same time does not work. But it also doesn't break. It simply doesn't apply the changes.

...
spec:
  config:
    cloud-config:
      users:
      - name: root
        passwd: topsecret
      ssh_authorized_keys:
      - ssh-ed25519 AAAAC...QKCU1RR user1
      - ssh-ed25519 AAAAC...nwZkfEYQ user2
      stages:
        boot:
        - files:
          - path: /root/foo
            content: |
                      Hello there.
            permissions: 0644
            owner: 0
            group: 0

Some kind of linting, or error-checking or json-schema would be awesome. Anything that saves me from doing try&error runs over and over again. This takes a lot of time with physical hardware....

For the beginning I'd be already happy if someone could tell me if I messed up the last example.

[DodoLeDev]

I am facing the same problem you mentioned by mixing yip and cloud-config, and because all these files ends in /oem partition, I was thinking if there is not an option to manually install files in this partition, with the write_files directive...

[max06]

Since we had to modify the used image to work with our network setup (bgp-fabric, ovs), we put almost everything directly into our image instead of using cloud-init or yip. Only the root password is left in there, and it works great.

Also some time ago, I noticed the little word "or" in the documentation, saying that you basically can use cloud-init OR yip style config, not both at the same time. It's just that one word, with so much meaning...