Cloud-init meta data during inspection

[fhaftmann]

User Story

As an operator
I would like to have cloud-init meta data specified in baremetalhost resource available during inspection
to be able to use cloud-init meta data for passing host-specific data relevant for layer 3 (BGP) networking

Detailed Description

Building a cloud environment using layer 3 routing requires a specific configuration of the servers (e.g. via frr) which involves e.g. a unique ASN.

It is tempting to use cloud-init meta data to pass such server-specific data.

(This of course requires also that the used images have logic built-in to interpret that meta data accordingly, but this is a different story and need not be treated here.)

However if I understand the documentation and the implementation correctly (v0.9.0 and v0.11.0), cloud-init meta data is not available during inspection.

It is only required that an existing meta data given for a baremetalhost resource is honoured during inspection – the default meta data settings provided later on after provisioning are not relevant here.

Anything else you would like to add:

I am happy to implement that – but what would be a suitable way to go?

a) Mimic the existing preprovisioningNetworkDataName into a preprovisioningMetaDataName?

b) Using the existing meta data also during inspection?

c) …

/kind feature

[Rozzii]

So just to summarize the current preprovisioningNetworkData that uses cloud-init compatible openstack network data format is not sufficient for your network management needs.
Ref:

• https://docs.openstack.org/nova/latest/user/metadata.html#openstack-format-metadata
• Example: https://docs.openstack.org/nova/latest/_downloads/9119ca7ac90aa2990e762c08baea3a36/network_data.json

You would like to have a new preprovisioningMetaDataName that would work similarly.

---

If I remember correctly but I have to check, "preprovNetworkData" already had an API endpoint in Ironic side so we were able to supply 2 different Network data because they were passed to Ironic in 2 separate way and had 2 different destinations on the Ironic API.

Metadata has dedicated Ironic Node API endpoint that could be used during inspection. We might be able to convince the Ironic community to allow us to implement such endpoint but it will be not as simple as copying the network data approach.

To address you immediate needs have you checked this? https://github.com/metal3-io/metal3-docs/blob/main/design/baremetal-operator/bmh_non-bootable_iso.md
This is available since 1.11 https://doc.crds.dev/github.com/metal3-io/baremetal-operator/metal3.io/DataImage/v1alpha1@v0.11.0
so it would be interesting to try to attach this non bootbale ISO to your machine and see what modifications if any would we need to implement to make this "non bootable ISO" available during inspection so you would be able to send any sort of data not just cloud-init

[Rozzii]

/triage needs-informtion

[metal3-io-bot]

@Rozzii: The label(s) triage/needs-informtion cannot be applied, because the repository doesn't have them.

Details

In response to this:

/triage needs-informtion

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[dtantsur]

I'm pretty strongly against preprovisioningMetaDataName in the API but we need to solve this. Maybe we just make Ironic generate something plausible, a minimal possible version of meta data?

It would help if you filed a request to Ironic's tracker directly (https://bugs.launchpad.net/ironic) since this limitation is coming from Ironic.

Note that you can use Glean instead of cloud-init as a workaround.

[fhaftmann]

Thanks for all the valuable feedback.

I want to clarify some fine points:

• There is no need for us to have different provisoning vs. preprovisioning meta data.
• We know that metal³ propagates meta data to ironic before provisioning; it would be sufficient to make sure it is propagated before inspection already – maybe this is already the case, but we didn't check so far; if somebody knowns about that already and can comment here, this would be helpful.
• Our ironic contains a patch which provides simple default meta data for inspection.
• We still need to figure out how we can use the officially known meta data at that place (and address this to Ironic upstream).

[Rozzii]

Metadata has dedicated Ironic Node API endpoint that could be used during inspection. We might be able to convince the Ironic community to allow us to implement such endpoint but it will be not as simple as copying the network data approach.

I meant to write here: "Metadata has NO dedicated Ironic Node API endpoint..."

• We know that metal³ propagates meta data to ironic before provisioning; it would be sufficient to make sure it is propagated before inspection already – maybe this is already the case, but we didn't check so far; if somebody knowns about that already and can comment here, this would be helpful.

I think BMO propagates the metadata as part of the "provisioning" , If I remember correctly BMO asks Ironic to transfer the config-drive data as part of the provisioning and Ironic uses that during the config-drive creation. That is why I was saying that for preprovisioning network_data BMO utilizes a separate workflow that is made possible by the Ironic node API.

I would second @dtantsur 's opinion about opening an Ironic issue on launchpad because this is an Ironic limitation.

[fhaftmann]

After some rounds of analysis, we came to the conclusion that the attribute instance_info of an Ironic node could be used (in a first implementation) to store cloud-init meta data and include that into the generated ISO boot image.

So, our plan looks now as follows:

a) A preliminary implementation

• Patch ironic to include meta data found in instance_info.meta_data into the generated ISO boot image.

• Patch bmo to fill instance_info.meta_data with a baremetalhosts’s associated meta data.

b) Discuss a final soluton with ironic

[fhaftmann]

We have a preliminary, partially tested implementation for future discussion.

Preliminary implementations

Patch ironic to include meta data found in instance_info.meta_data into the generated ISO boot image.

On top of quay.io/metal3-io/ironic:v28.0.0, this looks quite compact:

diff --git a/ironic/drivers/modules/image_utils.py b/ironic/drivers/modules/image_utils.py
index 56ccb7de6..e0302f9ba 100644
--- /usr/lib/python3.9/site-packages/ironic/drivers/modules/image_utils.py
+++ /usr/lib/python3.9/site-packages/ironic/drivers/modules/image_utils.py
@@ -592,6 +592,19 @@ def prepare_deploy_iso(task, params, mode, d_info):
         inject_files[network_data] = (
             'openstack/latest/network_data.json'
         )
+        # cloud-init will not use network_data if no meta_data is available
+        # in configdrive, so meta_data is added implicitly
+        ipa_hostname = 'ipa-' + task.node.name.partition('~')[2]
+        meta_data = {
+            "name": ipa_hostname,
+            "local-hostname": ipa_hostname,
+            "local_hostname": ipa_hostname,
+            "uuid": task.node.uuid
+        }
+        meta_data = json.dumps(meta_data, indent=2).encode('utf-8')
+        inject_files[meta_data] = (
+            'openstack/latest/meta_data.json'
+        )
 
     return prepare_iso_image(inject_files=inject_files)

Patch bmo to fill instance_info.meta_data with a baremetalhosts’s associated meta data.

This is actually more involved; but the core idea can be seen in the following diff:

https://github.com/metal3-io/baremetal-operator/compare/v0.9.3...fhaftmann:baremetal-operator:github/meta-data-proposal?expand=1

(The change required some re-engineering of unit test infrastructure, which would happen in separate commits in an officical PR.)

Questions

• In general: what do you think about the general direction?

• I agree that there should be a dedicated data slot for preprovisioning meta data in Ironic. You have proposed Launchpad for raising attention for that – anyway https://bugs.launchpad.net/ironic seems to provide only an issue type »bug«. Is it common practice there to categorize also functional extensions as »bugs«?

[Rozzii]

I think the general goal is acceptable and it would be beneficial to the project.
Let's continue discussing the implementation details but I will accept the general feature request itself.
/triage accepted
/kind feature