HostClaim Implementation

[pierrecregut]

As discussed during nov 2025 MTM, this is a discussion on the implementation of HostClaims covering at least:

• task breakdown and progress
• details not covered by the blueprint especially when they interact with other bmo or capm3 new features
• new needs (eg: data templating for HostClaims used without capm3)
• test strategy, documentation, etc.

[pierrecregut]

[New Component] HostClaim Data Templater

This is a summary of a requirement expressed by Didrik Koren and @lentzi90 on metal3 slack.
When a HostClaim is used without CAPM3, its configuration, especially the network configuration, will depend on the BareMetalHost it is associated to. So we need a template mechanism similar to Metal3DataTemplate except that it will limit the use of labels/annotations to the BareMetalHost (and potentially HostClaim) but that does not refer to Machine or Metal3Machine:

• We will defer this work item after the initial integration with CAPM3. It will probably require its own blueprint.
• How is the template associated to the HostClaim ?
  • an umbrella resource that associates such templates to HostClaims and will render it when the HostClaim is associated (risk: multiple umbrella resources targeting a single HostClaim unless the umbrella resource has the name of the HostClaim).
  • an annotation or field on the HostClaim that is only used for this use-case and not with the capm3 provider.
• Templates custom resources at the capm3 or HostClaim level will share 99% of the content (no cluster reference for HostClaims) but they will probably have to be duplicated. Probably the same for the rendering logic unless we push it in a separate package in baremetal-operator used by capm3 provider.

[nerok]

This covers a lot of my thoughts on the network config. I did start to wonder if there could be reasons to have networking both for the infra provider and the consumer, say infra knows that the machine needs to bond to specific interfaces, but might not know which vlan or IPs are in use. But after considering it, I think it would be to complex.

Though I see that the "vendor", and maybe "metadata" of cloud-init could really get to use here too, allowing the infra provider to create them.

For the actual network data templater I believe there are two approaches, the simple and the structured one. The simple would be a text template, say Go-template, just with access to all of hardware data and/or BMH. The structured one would require a go-library to generate a config-drive network config (openstack network config? Don't feel like I have a good name for that, I only notice that it is quite different from the one in cloud-init documentation), and possibly some selectors or something to specify the correct field in the BMH or HardwareData.

Which approach is better? Don't really know, I added both in my case, but used the structured (opinionated) one. That one required the physical links to match the links in HD, and got their mac-address from there, then it propagated the mac-address based on the "parent interface" (bond has members, vlan is on a existing interface etc).

[pierrecregut]

There was a discussion on this topic during MTM for preprovisionning data (from 20:30 in the recording) that evolved to a discussion on making network data more generic. To make more clear what I meant by templates with functions, a raw prototype in Kanod is here : https://gitlab.com/Orange-OpenSource/kanod/host-baremetal/-/blob/main/internal/controller/network_data_template.go?ref_type=heads . But for HostClaims without capm3 it would not be understandable to have a solution that is not similar to the standard case with capm3.

[pierrecregut]

[Detail] Status of the HostClaim

• Expose as much as possible through Kubernetes conditions (no reason to support obsolete capi conditions).
• No Phase or FailureMessage/Reason
• Initial proposal: distinguish at least Associated and Synchronized condition (most errors were used after the synchronization and could easily be captured by the Synchronization condition). Both summarize in a Ready condition.
• We should also expose the status of the BareMetalHost as conditions. There are two major stable states that can be exposed as conditions : Provisioned and Available, the others can be seen as intermediate steps toward those goals and expressed as reasons why the condition is (not yet) fulfilled. With CapM3, we are only interested in the Provisioned condition: the BMH was available when it was chosen and the HostClaim will be dropped when the Metal3Machine is replaced. But it may not be the case for HostClaims used without CAPM3 and we could also think of alternate implemetnation of node-reuse. So Available should probably also be exposed.
• We must not expose to many details at that level. There must be a clean separation between what consumers and hardware teams see.

[nerok]

An issue I experienced when trying this myself was that one machine failed, but I had multiple other machines matching the same selector (the selector chose any machine of a particular kind, on purpose). I added support for failing to another machine if deploy to one failed (and tracked both the machine currently in use and all that had previously failed, and if all failed, we rolled back to the first and retried that again).

If this is something we want in HostClaim, it might need to be configurable, as this might not be what everyone wants, and it might also depend from machine to machine or differ based on HostDeployPolicy.

Another issue which occured in the same situation was when we created multiple machines at once, they tried to take the same machine, but that quickly got resolved after using consumerRef. Not that important, just as a note.

[pierrecregut]

The Kanod hostclaim prototype and the coming metal3 version use the same choice/locking logic as the metal3 capi provider between a metal3machine and a bareMetalHost. The main question is how to avoid duplicating the code despite the few important variations.

[pierrecregut]

On failure recovery, groups of machines, etc. HostClaim is about binding a single workload to a BareMetalHost. There may be a need for an equivalent of MachineSets, MachineDeployments at the level of HostClaim but then there is also a need for a generic liveness probe to implement it correctly and this is a different topic.
My opinion is that when you really need that, you are better off pretending your machine is a cluster or integrate it into an existing cluster. The added kubelet will be the liveness probe and it will host all your telemetry needs without reinventing the wheel even if the main workload is not a container.

[pierrecregut]

[Tests] HardwareData e2e tests

HostClaims rely a lot on the separation BareMetalHost/HardwareData to hide the BMH from the end user. Current e2e test only look at the hardwareDetails in status (potentially fed by an annotation of the resource). We probably should add e2e tests on HardwareData and provide the HardwareData with inspection disabled in the e2e tests of HostClaims.

[lentzi90]

We recently added a simple test case that makes use of HardwareData for external inspection. This should hopefully be useful:

baremetal-operator/test/e2e/external_inspection_test.go

Line 251 in 11b070e

It("should skip inspection and become available when HardwareData exists and BMH has inspection disabled", func() {

.

[pierrecregut]

[Task Breakdown]

• Implementation of HostClaim resource and controller with companion HostDeployPolicy resource in baremetal-operator project
  • Definition of custom resources
  • Controller implementation and unit testing
  • E2E tests
• Using HostClaims in cluster-api-provider-metal3
  • modifications of the Metal3Machine CRD (namespace in selector, identityRef, etc.)
  • implementation of the Metal3Machine reconciliation with HostClaims as a Metal3MachineManager, hook to attach it to the Metal3MachineController.
  • metal3data rendering when the m3m is attached to a hostclaim abstraction of BMH behind a data getter.
  • support of remediation with HostClaims.
  • Tests
• Documentation of HostClaims and their use
  • hostclaim resource and association with baremetalhost
  • usage with the cluster api provider
• Using hostclaims in Metal3 devenv
• Templating for HostClaims (without capm3).

[pierrecregut]

@Rozzii may be, I could open issues for the main items in the relevant projects and sub-issues for level 2 items so that that the discussion can stay organized when there is some kind of agreement on the break-down ? What is the current practice in Metal3 projects ? I would modify the plan above to point to those issues.

[pierrecregut]

#2856 defines the hostclaim resource. The full implementation in baremetal operator is visible on branch hostclaim: https://github.com/pierrecregut/baremetal-operator/tree/hostclaim

Commits are huge but splitting them does not make a lot of sense.

An embryo of documentation in the book is here https://github.com/lroussarie/metal3-docs/tree/hostclaim