Skip to content

fix(resmgr): terraform destroy deletes the cluster blueprint (PCD-9783) - #49

Merged
PF9-pushkar merged 3 commits into
mainfrom
pushkar/pcd-9783-blueprint-destroy
Sep 4, 2026
Merged

fix(resmgr): terraform destroy deletes the cluster blueprint (PCD-9783)#49
PF9-pushkar merged 3 commits into
mainfrom
pushkar/pcd-9783-blueprint-destroy

Conversation

@PF9-pushkar

@PF9-pushkar PF9-pushkar commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Fixes PCD-9783: terraform destroy of a pcd_cluster_blueprint reported success but left the blueprint in PCD.

Root cause

Delete was a deliberate no-op (introduced with the resource in fffa1d5): destroy dropped the resource from state and never called resmgr, on the reasoning that a region's single blueprint is normally imported and should only stop being managed. That contradicts how Terraform resources are expected to behave, and nothing short of the PCD UI could remove the blueprint.

Change

  • Delete now issues DELETE /resmgr/v2/blueprint/<name>. A 404 (already gone) is success; any other refusal fails the destroy instead of being hidden.
  • Schema description and generated docs now say destroy deletes the blueprint, that dependents (pcd_host_config via cluster_name, clusters, host roles) must go first, and that terraform state rm is the way to stop managing an imported blueprint without deleting it.
  • Unit tests pin the HTTP method and path and the status handling (200/202/204/404 succeed; 409/500 fail).
  • CHANGELOG entry under Unreleased, called out as a behaviour change for anyone who relied on destroy being state-only. Ships on the 0.1.x train (next: 0.1.10).
  • Also bumps the indirect google.golang.org/grpc to 1.83.1 for Dependabot alert 4 (GHSA-vp52-pcj8-j9qc), with a Security changelog entry; same minimal-exposure reasoning as the 0.1.9 gRPC bump.

Verification

Static: go build, go vet, gofmt, golangci-lint (0 issues), go test ./internal/... all green.

Live, on the CE lab (PCD 2026.4), using a dev build of this branch: the region's existing blueprint ce-lab-bp, cluster ce-lab, host config hyp-2nic and hyp1's hypervisor, image-library and persistent-storage roles were imported into a scratch state (workloads cleared first via the API, since PCD refuses the storage deauth while volumes exist), then destroyed with a single terraform destroy:

Plan: 0 to add, 0 to change, 6 to destroy.
pcd_host_cluster_role.hyp1["persistent-storage"]: Destruction complete after 1s
pcd_host_cluster_role.hyp1["image-library"]: Destruction complete after 1s
pcd_host_cluster_role.hyp1["hypervisor"]: Destruction complete after 3s
pcd_host_config.hyp_2nic: Destruction complete after 0s
pcd_cluster_blueprint.ce_lab_bp: Destruction complete after 0s
pcd_cluster.ce_lab: Destruction complete after 0s
Destroy complete! Resources: 6 destroyed.

Afterwards GET /resmgr/v2/blueprint, /clusters and /hostconfigs all returned empty lists. Terraform's graph ordered the host config (which references the blueprint by name) before the blueprint, so resmgr never saw the blueprint delete while a dependent still stood. hyp1's roles were still converging off at that moment (resmgr's asynchronous deauth); once it landed, the testsuite's region_empty judge reported ok=True detail=empty with both hosts at roles=[].

Fresh-lab zero-to-hero (854b7e7). The lab was then rebuilt from scratch (delete-ce-lab / create-ce-lab, one hypervisor, no config) and the pcd-tf-testsuite z2h case was run in its apply-then-destroy mode against a dev build of this branch: blueprint, Synology-backed storage and volume type, flat network/subnet/security group, hypervisor + image-library + persistent-storage roles on hyp1, a cirros image and a VM with an attached volume. Every judge passed (blueprint/hostconfig/cluster JSON match, roles ok, cinder-volume up, image volume-backed, server ACTIVE, volume in-use, plan clean, ping). One terraform destroy then removed all of it, ending with the suite's region_empty judge green. It took the suite's usual three attempts while resmgr's asynchronous deauth refused the host-config unassign (403 HostInAuthState); the blueprint delete happened through the provider on the final pass. 17 min end to end.

Note for pcd-tf-testsuite

Its teardown calls api.delete_blueprint("ts-bp") with the comment "provider Delete is state-only by design". That call already tolerates 404, so it keeps working, but the comment is now stale.

pcd_cluster_blueprint's Delete was a deliberate no-op: destroy removed the
resource from state and never called resmgr, on the reasoning that a region's
single blueprint is normally imported and should merely stop being managed.
That is not how a Terraform resource is expected to behave. `terraform
destroy` reported the blueprint destroyed while it lived on in the region, and
nothing short of the PCD UI could remove it.

Delete now issues DELETE /resmgr/v2/blueprint/<name>. A blueprint that is
already gone (404) is the outcome destroy wants, so it is success; any other
refusal fails the destroy instead of being hidden, since a destroy that
reports success over a blueprint that still exists was the bug. Users who only
want to stop managing an imported blueprint have `terraform state rm`, which
the schema description and docs now say.

Verified on the CE lab (2026.4) by importing the region's blueprint, cluster,
host config and hyp1's three cluster roles into a scratch state and running
one `terraform destroy`: all six were removed in dependency order and
GET /resmgr/v2/blueprint listed nothing afterwards. Terraform's graph puts the
host config (which references the blueprint by name) before the blueprint, so
resmgr never saw the delete while a dependent still stood.
Dependabot alert 4 on main: gRPC-Go heap memory exhaustion via HTTP/2 DATA
frame fragmentation, fixed in 1.83.1. The provider's gRPC server only serves
the local Terraform CLI over a private channel, so exposure was minimal; bumped
all the same. go mod tidy also moved genproto/googleapis/rpc to the matching
snapshot.
@PF9-pushkar
PF9-pushkar merged commit 88954f2 into main Sep 4, 2026
5 checks passed
@PF9-pushkar
PF9-pushkar deleted the pushkar/pcd-9783-blueprint-destroy branch September 4, 2026 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant