ignition file passed using --ignition flag is ignored

[cmoulliard]

Issue

The ignition cfg file passed used the flag --ignition is ignored no matter if we use a relative or fullpath to access it

I use vfkit: 0.6.0 on macos: 14.7.2

export IMG=fedora-coreos-41.aarch64.raw

vfkit \
--cpus 2 \
--memory 2048 \
--ignition /Users/cmoullia/code/_temp/vfkit/dev/my-cfg.json \
--bootloader efi,variable-store=dev/efi-variable-store,create \
--device virtio-blk,path=$IMG \
--device virtio-input,keyboard \
--device virtio-input,pointing \
--device virtio-net,nat,mac=5a:94:ef:e4:0c:ee \
--device rosetta,mountTag=rosetta,install \
--restful-uri tcp://localhost:60195 \
--device virtio-rng \
--device virtio-vsock,port=1025,socketURL=dev/default.sock,listen \
--device virtio-gpu,width=800,height=600 \
--gui \
--device virtio-serial,logFilePath=dev/default.log

Ignition file

This file has been populated using butane

{
  "ignition": {
    "version": "3.1.0"
  },
  "passwd": {
    "users": [
      {
        "groups": [
          "wheel",
          "plugdev"
        ],
        "homeDir": "/home/user1",
        "name": "user1",
        "noCreateHome": true,
        "passwordHash": "$y$j9T$oDXijWyyIphUF/uI8/QdU0$ITomDBxgnCxOUb0eYK3qfO1MSZUPFSmHmkL6PHvZ1I6",
        "sshAuthorizedKeys": [
          "ssh-rsa AAAAB3NzaC1y...1ecj7WSL9 ch007m@gmail.com"
        ],
        "shell": "/bin/bash"
      }
    ]
  }
}

Note: The file has been validated using ignition-validate

podman run --pull=always --rm -i quay.io/coreos/ignition-validate:release - < /Users/cmoullia/code/_temp/vfkit/dev/my-cfg.json
Trying to pull quay.io/coreos/ignition-validate:release...
Getting image source signatures
Copying blob sha256:934243d352caa23618dd8f45a7b3b56bb92642443564818e12d791a6d6cda10d
Copying config sha256:8f00f124fa6798eba184789111d42c900a1314bbe1812043ca58eac0d1bdfc54
Writing manifest to image destination

Debug

No error occurs when the ignition file is added to the VM

~/code/_temp/vfkit on main •
❯ ./start-vm.sh
INFO[0000] &{2 2048    {[efi variable-store=/Users/cmoullia/code/_temp/vfkit/dev/efi-variable-store create] true}  [virtio-blk,path=fedora-coreos-41.aarch64.raw virtio-input,keyboard virtio-input,pointing virtio-net,nat,mac=5a:94:ef:e4:0c:ee rosetta,mountTag=rosetta,install virtio-rng virtio-vsock,port=1025,socketURL=/Users/cmoullia/code/_temp/vfkit/dev/default.sock,listen virtio-serial,logFilePath=/Users/cmoullia/code/_temp/vfkit/dev/default.log virtio-gpu,width=800,height=600] tcp://localhost:60195 debug true /Users/cmoullia/code/_temp/vfkit/dev/my-cfg.ign} 
INFO[0000] boot parameters: &{EFIVariableStorePath:/Users/cmoullia/code/_temp/vfkit/dev/efi-variable-store CreateVariableStore:true} 
INFO[0000]                                              
INFO[0000] virtual machine parameters:                  
INFO[0000]      vCPUs: 2                                    
INFO[0000]      memory: 2048 MiB                            
INFO[0000]                                              
INFO[0000] Adding virtio-blk device (imagePath: fedora-coreos-41.aarch64.raw) 
INFO[0000] Adding virtio-input keyboard device          
INFO[0000] Adding virtio-input pointing device          
INFO[0000] Adding virtio-net device (nat: true macAddress: [5a:94:ef:e4:0c:ee]) 
DEBU[0000] ignition socket: /var/folders/28/g86pgjxj0wl1nkd_85c2krjw0000gn/T/ignition.sock 
INFO[0000] Adding virtio-fs device                      
INFO[0000] Adding virtio-rng device                     
INFO[0000] Adding virtio-vsock device                   
INFO[0000] Adding virtio-serial device (logFile: /Users/cmoullia/code/_temp/vfkit/dev/default.log) 
DEBU[0000] Setting up graphics device with 800x600 resolution. 
INFO[0000] Adding virtio-gpu device                     
DEBU[0000] virtio-vsock device already present, not adding a second one 
INFO[0000] virtual machine is running                   
INFO[0000] Exposing vsock port 1025 on /Users/cmoullia/code/_temp/vfkit/dev/default.sock (listening) 
INFO[0000] Exposing vsock port 1024 on /var/folders/28/g86pgjxj0wl1nkd_85c2krjw0000gn/T/ignition.sock (listening) 
INFO[0000] waiting for VM to stop  

[lstocchi]

This was discussed and solved on slack.
By using the latest Fedora CoreOS AppleHV image and updating the ignition file (we removed the group plugdev as not available and set noCreateHome to false) fixes the issue.

I'll keep this open as a reminder to update the documentation to mention that the Fedora CoreOS AppleHV image is needed.

[cfergeau]

Most likely related to the default data source ignition will use, there is an applehv-specific one. This must be the main difference between the various images.

[cmoulliard]

I'll keep this open as a reminder to update the documentation to mention that the Fedora CoreOS AppleHV image is needed.

Is it possible that vfkit checks if the image used is an AppleHV one ?

[cmoulliard]

By using the latest Fedora CoreOS AppleHV image and updating the ignition file (we removed the group plugdev as not available and set noCreateHome to false) fixes the issue.

Moving to AppleHV fixed the issue about the ignition file not loaded. Why such a file was not loaded is still a mysterious as it has nothing to do with AppleHV ?

[cfergeau]

I'll keep this open as a reminder to update the documentation to mention that the Fedora CoreOS AppleHV image is needed.

Is it possible that vfkit checks if the image used is an AppleHV one ?

That's not easy to detect, the problem is that inside the guest, ignition must be configured to fetch its config file using an applehv specific method (https://coreos.github.io/ignition/getting-started/#providing-a-config). We'd need to mount/parse the content of the VM image in order to do that.
We could add more debug messages to the ignition code, including a suggestion that the image may not be running ignition for the applehv platform if the ignition config was not fetched in N seconds.

[cmoulliard]

We could add more debug messages to the ignition code, including a suggestion that the image may not be running ignition for the applehv platform if the ignition config was not fetched in N seconds.

+1