feat: add JBOD additional PVC support for ClickHouse replicas#136
Merged
Conversation
GrigoryPervakov
force-pushed
the
jbod
branch
2 times, most recently
from
dbb4b87 to
d2a6c41
Compare
| // per-pod PVCs named <name>-<statefulset>-<ordinal>. | ||
| // Use for JBOD / multi-disk storage layouts. | ||
| // +optional | ||
| AdditionalDataVolumeClaimSpecs []AdditionalVolumeClaimSpec `json:"additionalDataVolumeClaimSpecs,omitempty"` |
Member
There was a problem hiding this comment.
It may be enough to define
AdditionalVolumeClaimTemplates []v1.PersistentVolumeClaim
Then the operator can check whether any VolumeMounts are set in ContainerSpec and generate default ones if none are present, as well as generate configuration entries in this case.
This will give the user the ability to control mount params and use the feature in different scenarios
WDYT?
Member
|
@matanper, can you check if such an approach solves your problem? |
Introduce additionalDataVolumeClaimSpecs with validation, templating, and reconciliation support so replicas can mount and configure multiple persistent disks while safely updating per-template PVC specs without immutable StatefulSet failures.
- Use a single volume with all disks for true JBOD round-robin distribution instead of one volume per disk (which was tiered storage) - Remove redundant MountPath fallback in storageJbodConfigGenerator.Generate; WithDefaults() already guarantees a non-empty value - Validate duplicate mountPaths across additionalDataVolumeClaimSpecs, including implicit defaults colliding with explicit paths
Remove dead ReplicaUpdateInput.DataVolumeClaimSpec field, rename primaryPVCName to targetPVCName for clarity, always preserve immutable volumeClaimTemplates on StatefulSet updates, fix potential panic on empty MountPath, and extract AdditionalDiskBasePath constant.
Handle additional disk updates for existing ClickHouse clusters by creating per-replica PVCs separately from StatefulSet volumeClaimTemplates, updating pod mounts to use explicit PVC volumes, and allowing additive additionalDataVolumeClaimSpecs updates while blocking remove/rename operations.
…lickHouse compatibility Disk names used in additionalDataVolumeClaimSpecs serve dual purposes: as Kubernetes PVC/volume names (requiring lowercase alphanumeric + hyphens) and as ClickHouse disk identifiers (forbidding hyphens). Previously there was no format validation, allowing names like disk-backfill-1 to pass admission but fail in ClickHouse, or names with underscores to pass but fail Kubernetes PVC creation. - Add webhook validation enforcing Kubernetes DNS label rules (lowercase alphanumeric and hyphens, must start/end with alphanumeric) - Add matching +kubebuilder:validation:Pattern marker to the CRD type so the API server enforces the same constraint even without the webhook - Regenerate CRD manifests - Sanitize hyphens → underscores in the ClickHouse JBOD config generator so users use Kubernetes-valid names and the operator handles the ClickHouse naming requirement transparently
Contributor
Author
|
@GrigoryPervakov Yes I think it can work for me |
|
Docs PR opened: ClickHouse/mintlify-docs-dev#215 Synced upstream Kubernetes Operator docs adding JBOD multi-disk storage, three new guides (monitoring, scaling, TLS), and API reference updates. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce additionalDataVolumeClaimSpecs with validation, templating, and reconciliation support so replicas can mount and configure multiple persistent disks while safely updating per-template PVC specs without immutable StatefulSet failures.
Why
ClickHouse multi-disk (JBOD) deployments currently require workarounds and do not have first-class support in the operator API. This change adds explicit support for attaching additional per-replica PVCs and configuring them in ClickHouse, while preserving safe reconciliation behavior with StatefulSet immutability constraints.
What
This PR adds a new ClickHouseCluster field, spec.additionalDataVolumeClaimSpecs, to define extra per-replica PVC templates (name,