resmgr: cluster roles, clusters, provider networks — full zero-to-hero support - #43
Merged
Merged
Conversation
pcd_host_role was non-functional against PCD 2026.4. Create and Delete
issued PUT/DELETE /resmgr/v2/hosts/<id>/roles/<name>, but resmgr has no
writable roles sub-resource on v2 and answers 404 RoleNotFound; role
assignment exists only on v1. Every apply failed.
Read was broken independently: it compared the configured role name
against GET /v2/hosts/<id>, whose roles are mapped uber-roles
("hypervisor") rather than the granular pf9-* names roles are assigned
by. The match never succeeded, so Terraform dropped the resource from
state and recreated it on every plan. Fixing only the write path would
have left that drift in place.
Add ResmgrV1Client alongside ResmgrV2Client and point pcd_host_role's
Create, Read and Delete at v1. Blueprints and host configs stay on v2 —
/resmgr/v1/blueprint and /resmgr/v1/hostconfigs are both 404, so the two
versions are not interchangeable and the provider needs both. An
endpoint_overrides entry for "resmgr" now names the service rather than
an API version: the needed version is applied to it, replacing any
version already present, so one override serves both clients.
Separately, pcd_cluster_blueprint showed a storage_backends_json diff on
every plan. resmgr echoes the blob with its own spacing in insertion
order while jsonencode() emits compact output with sorted keys —
semantically equal, textually different, which Terraform reports as an
in-place update that never converges. Canonicalise the read-back in both
the resource and the data source.
Verified end-to-end against a 2026.4 CE lab: an empty region goes to
blueprint + host config + assignment + 9 roles in one apply, and the
following plan reports no changes.
A fresh PCD region could not be brought from empty to a running VM by Terraform alone. Closing that required three new capabilities and one endpoint correction, each verified end-to-end against a 2026.4 CE lab. New resource pcd_host_cluster_role: assigns the resmgr v2 cluster roles (hypervisor, image-library, persistent-storage, dns) — the API the PCD UI onboards hosts with. The control plane expands a cluster role into its granular pf9-* roles with settings computed from the blueprint and host config; persistent-storage names blueprint backends, hypervisor takes the host cluster 2026.4 requires. wait_until_converged blocks until role_status = ok, tolerating the transient failed flaps normal onboarding produces, so one apply can onboard a hypervisor and boot an instance on it. Writes retry through the transient 409 RoleUpdateConflict resmgr answers while a host converges. New resource pcd_cluster: host clusters (host groups), the unit hypervisors join and the scope for VM HA, auto-rebalancing, GPU and CPU-model settings. Required by the hypervisor cluster role. pcd_networking_network gains segments (provider-network attributes, mirroring openstack_networking_network_v2): without it neither physical nor external networks could be created. A single segment is sent as top-level provider:* attributes — the form OVN accepts — and multiple segments use the multi-provider form. Create-only, like upstream. ImageV2Client now prefers the Glance ADMIN endpoint, falling back to public when absent. PCD runs two Glance deployments sharing one database: the control-plane pod behind the public endpoint stores uploads in pod-local file storage where no hypervisor can fetch them (boot fails with "Image not found in any configured backend"); the image-library host behind the admin endpoint stores them on the blueprint's storage backend. The UI uploads via admin for this reason. endpoint_overrides.image remains the escape hatch where the admin endpoint is not routable. pcd_host_role's documentation now warns that granular assignment applies default role settings and wedges the host on settings-bearing roles (pf9-cindervolume-config), directing onboarding to pcd_host_cluster_role. Verified: empty region -> volume type, blueprint (Synology backends), host config, cluster, three cluster roles -> role_status ok with a live Cinder pool -> provider network, subnet, security groups, image (stored on the backend), flavor, instance ACTIVE at 172.16.122.23 with a 1GB attached volume — then terraform plan: "No changes."
Two defects surfaced by running the full empty-region-to-instance apply against a freshly deployed CE, where every timing window is at its tightest. wait_until_converged trusted the host's aggregate role_status, which is computed over the roles assigned at that moment. With several cluster roles assigned concurrently, the aggregate reads "ok" in the window after one role's PUT lands and before the others do, so the wait returned in seconds and un-gated downstream resources — the CirrOS image upload raced a Glance that was not serving yet. The wait now also requires the specific cluster role's granular marker (pf9-glance-role for image-library, pf9-ostackhost-neutron for hypervisor, pf9-cindervolume-base for persistent-storage, pf9-designate for dns) to report applied before returning. pcd_cluster creation answered 500 "Request Failed" when the region was minutes old: resmgr depends on the compute control plane being warm, and the PCD UI health-checks Nova before offering its create-cluster dialog at all. The identical request succeeds shortly after. Create now retries 500s for a bounded window (8 minutes, 15s interval); a genuinely invalid request still fails once the window closes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A fresh PCD region could not be brought from empty to a running VM by Terraform alone. This PR closes every gap, verified end-to-end against a 2026.4 CE lab.
Fixed
pcd_host_rolecould not assign roles at all — Create/Delete used resmgr v2, which has no writable roles sub-resource (404 RoleNotFound); assignment lives on v1. Read compared against v2's uber-role names so the resource also drifted permanently. Both paths now use v1.pcd_cluster_blueprintdiffedstorage_backends_jsonon every plan — resmgr echoes the blob with its own spacing/ordering; the read-back is now canonicalised to matchjsonencode().pcd_host_cluster_role.wait_until_convergedcould return early —role_statusaggregates only currently-assigned roles; the wait now also requires the specific role's granular marker to report applied.pcd_clustercreation failed on fresh regions — resmgr answers 500 until the compute control plane warms (the UI health-checks Nova before offering the dialog); Create retries 500s for a bounded window.Added
pcd_host_cluster_role— the resmgr v2 cluster-role (uber-role) API the PCD UI onboards hosts with: server-side expansion into granular roles with settings computed from the blueprint and host config.backendsfor persistent-storage,host_clusterfor hypervisor, optionalwait_until_converged, 409-retries while hosts converge.pcd_cluster— host clusters (host groups): required by the hypervisor role; carries VM HA / rebalancing / GPU / CPU settings.pcd_networking_network.segments— provider-network attributes mirroringopenstack_networking_network_v2; without it neither physical nor external networks could be created.endpoint_overrides.imageremains the escape hatch.Validation
Single untouched
terraform applyagainst a freshly deployed empty region: volume type → blueprint (Synology backend) → host config → cluster → three cluster roles → converged hypervisor with a live Cinder pool → provider network → subnet → security groups → image (stored on the backend) → instance ACTIVE with an attached volume in 4m38s — followed byterraform plan: "No changes." The destroy path is exercised too (role deauth through 409s, full region teardown).