Skip to content

linode: ephemeral StorageClass uses Delete reclaim, not Retain - #1313

Merged
travagliad merged 2 commits into
mainfrom
claude/pmm-qa-hooks-permissions-verify-pt9u2v
Sep 3, 2026
Merged

linode: ephemeral StorageClass uses Delete reclaim, not Retain#1313
travagliad merged 2 commits into
mainfrom
claude/pmm-qa-hooks-permissions-verify-pt9u2v

Conversation

@travagliad

Copy link
Copy Markdown
Contributor

Problem

Follow-up to #1311. The tagged StorageClass that create-lke installs kept LKE's default Retain reclaim policy. On a live cluster, every churned or deleted PVC (a StatefulSet reschedule, an operator recreating a claim) then leaves its Block Storage volume behind as a Released PV that keeps billing until the whole cluster is torn down.

Observed on a live HA cluster (pmm-ha-PMM-15304-ha): 35 of 54 PVs Released, 830 GB parked. These are not a cross-cluster leak — they carry the live cluster's pmm-qa-run tag, so the orphan sweep correctly keeps them and will reap them when the cluster dies — but they waste money for the cluster's whole life (up to its 24–48 h TTL).

Change

One line: the ephemeral StorageClass now uses reclaimPolicy: Delete instead of Retain. A deleted PVC frees its volume immediately, while the CSI controller is alive — no Released buildup during the cluster's life.

Teardown safety is unchanged: an abrupt lke cluster-delete still can't run the CSI controller, but volumes are tagged at birth (volumeTags, #1311) so prune-lke-orphans.sh reaps whatever survives. Delete only removes the within-life Released accumulation. The StorageClass name is kept so charts referencing it still resolve.

This is intentionally not the race-hardening from the closed #1312 (that concern was judged not worth the change) — only the reclaim-policy flip.

Validation

  • shellcheck -S warning and bash -n clean.
  • Root cause confirmed live: 35/54 Released PVs on a running cluster under the Retain SC; Delete is the standard reclaim policy for throwaway storage and removes that buildup at the source.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DBhvZuEUQU8ym347Xat4Yy


Generated by Claude Code

Follow-up to #1311. The tagged StorageClass kept the LKE default's Retain reclaim
policy, so on a live cluster every churned/deleted PVC (StatefulSet reschedule,
operator recreation) leaves its Block Storage volume behind as a Released PV that
bills until the cluster is torn down. Observed on a live HA cluster: 35 of 54 PVs
Released, 830 GB parked.

Flip the ephemeral cluster's StorageClass to reclaimPolicy: Delete so a deleted
PVC frees its volume immediately while the CSI controller is alive. The tag-based
sweep still reaps whatever survives an abrupt cluster-delete (volumes are tagged
at birth via volumeTags); Delete only removes the within-life Released buildup.
The StorageClass name is unchanged so charts referencing it still resolve.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DBhvZuEUQU8ym347Xat4Yy
Comment thread .claude/skills/linode-ha-provisioning/scripts/create-lke-pmm-ha.sh Outdated
@claude

claude Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Lint (shellcheck -S warning) is green on the head SHA, but no workflow exercises create-lke-pmm-ha.sh, so nothing in CI proves the recreated StorageClass provisions or reclaims — and the Validation section records only static checks, not a live provision under Delete. .claude/** has no suite reference in the review skill, so this diff got the generic checks only.

@coderabbitai

coderabbitai Bot commented Sep 3, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

Next included review available in 56 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 9342a9e7-c05d-4a2d-a92e-d70b8699f1ca

📥 Commits

Reviewing files that changed from the base of the PR and between 29d3547 and 06d3662.

📒 Files selected for processing (1)
  • .claude/skills/linode-ha-provisioning/scripts/create-lke-pmm-ha.sh

Walkthrough

The LKE provisioning script changes the linode-block-storage-retain StorageClass reclaimPolicy from Retain to Delete. A comment explains that deleted PVCs release their volumes immediately. The StorageClass name remains unchanged.

Merge Risk: 🔵 Low · up to 29d35

The StorageClass will delete volumes for deleted PVCs as intended, but its explanatory comment incorrectly describes the default reclaim policy and should be corrected before merge.

🚥 Pre-merge checks | ✅ 2 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The PR does not implement the primary requirements in linked issue #1312: race-safe StorageClass replacement with kubectl replace --force, retry handling, and non-fatal failure behavior. Implement the race-safe and non-fatal StorageClass replacement described in issue #1312, or link the PR to an issue that covers the reclaim-policy change instead.
Out of Scope Changes check ⚠️ Warning The reclaimPolicy change is outside the scope of linked issue #1312, which focuses on race-safe and non-fatal StorageClass replacement. Remove the reclaimPolicy change from this PR, or update the linked issues so they explicitly include the ephemeral StorageClass reclaim-policy objective.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: the ephemeral StorageClass now uses the Delete reclaim policy instead of Retain.
Description check ✅ Passed The description explains the Retain-policy problem, the Delete-policy change, preserved teardown behavior, and validation results. It is directly related to the changeset.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 67b121b3-4fca-4caf-88e8-044ccdbad503

📥 Commits

Reviewing files that changed from the base of the PR and between 5fdce76 and 29d3547.

📒 Files selected for processing (1)
  • .claude/skills/linode-ha-provisioning/scripts/create-lke-pmm-ha.sh
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • percona/pmm-qa (manual)
  • percona/pmm (manual)

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

Comment thread .claude/skills/linode-ha-provisioning/scripts/create-lke-pmm-ha.sh Outdated
@travagliad
travagliad merged commit e105d12 into main Sep 3, 2026
41 checks passed
@travagliad
travagliad deleted the claude/pmm-qa-hooks-permissions-verify-pt9u2v branch September 3, 2026 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants