[RFE] Setup ephemeral networking to report provisioning complete

[anhvoms]

Current situation

Currently azure-init relies on the VM's network to pull provisioning metadata from IMDS and to report provisioning complete to Azure platform

Impact

azure-init is vulnerable to guest network issues that could prevent it from getting provisioning metadata and reporting provisioning complete. The most common scenario is the default route being setup over non-primary interface, which will block traffic to IMDS/wireserver. There will also be other issues such as handling the VF (Virtual Function) network interface for VMs with accelerated networking enabled.

Ideal future situation

azure-init should setup an ephemeral DHCP lease over the correct primary nic to pull information from IMDS and also uses the same lease to report provisioning complete. The lease should be released once done.

[t-lo]

This is an interesting challenge for provisioning agents; we briefly discussed this at the Image-Based Linux summit last year.

The underlying problem here is that network set-up is intrinsically distribution specific, which would have repercussions on maintenance load. Also, messing with network settings at early boot (like adding a network and IP address to the guest system, even if just ephemerally), potentially with regular network set-up happening in parallel, can be a random source of issues.

So one of the recommendations / proposals made at the summit to fundamentally solve this issue was to not mess with guest networking at all but instead utilise a user-space networking stack to "privately" manage connections to IMDS.

Food for thought 😸

[anhvoms]

Properly setup network according to how the customers want it is hard, but setting up an "ephemeral" network (pick the correct primary nic, issue a dhcp request using dhclient or dhcpd) is much simpler thought.
Setting up azure-init's own dhcp lease will also allow the possibility of supporting pre-provisioning