Add distributed snapshotting support to kubernetes-distributed deployment#653
Open
mnencia wants to merge 1 commit intokubernetes-csi:masterfrom
Open
Add distributed snapshotting support to kubernetes-distributed deployment#653mnencia wants to merge 1 commit intokubernetes-csi:masterfrom
mnencia wants to merge 1 commit intokubernetes-csi:masterfrom
Conversation
k8s-ci-robot
added
release-note
Contributor
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: mnencia The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Contributor
|
Welcome @mnencia! |
Contributor
|
Hi @mnencia. Thanks for your PR. I'm waiting for a kubernetes-csi member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
needs-ok-to-test
k8s-ci-robot
removed
the
do-not-merge/work-in-progress
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
…ment Add the csi-snapshotter sidecar with --node-deployment=true to the distributed DaemonSet deployment, enabling per-node snapshot operations for node-local volumes. To support multiple sidecar RBAC roles, introduce a unified ServiceAccount (csi-hostpathplugin-sa) with explicit ClusterRoleBindings and RoleBindings for the provisioner and snapshotter roles, following the same pattern used by the kubernetes-latest deployment. The deploy script now installs the snapshot CRDs and snapshot-controller (with --enable-distributed-snapshotting=true) if not already present, and patches an existing snapshot-controller if it lacks the flag. The upstream snapshot-controller RBAC has node read permissions commented out, so the script uncomments them before applying, as documented in the external-snapshotter README. The destroy script cleans up the snapshot-controller and its RBAC resources, as well as the VolumeSnapshotClass. Snapshot E2E tests are enabled via snapshotDataSource in test-driver.yaml. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
mnencia
added a commit
to cloudnative-pg/cloudnative-pg
that referenced
this pull request
Apr 13, 2026
Replace the single-node StatefulSet csi-hostpath deployment with the distributed DaemonSet variant, which runs the CSI driver on every node with per-node provisioning, snapshotting, and resizing via --node-deployment sidecars. This enables multi-node test scenarios (pod failover, node drain, etc.) where PVCs can be created on different nodes instead of all landing on the same one. The resizer sidecar uses a custom image (ghcr.io/mnencia/csi-resizer:node-deployment) built from kubernetes-csi/external-resizer#573, pending upstream merge. The distributed deployment manifests are sourced from mnencia/csi-driver-host-path#dev/651-with-resizer, pending kubernetes-csi/csi-driver-host-path#653. Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
What this PR does / why we need it:
The distributed DaemonSet deployment (
deploy/kubernetes-distributed/) only includes the csi-provisioner sidecar. This adds the csi-snapshotter with--node-deployment=true, so snapshots work on each node's local volumes.To support multiple sidecar RBAC roles, this introduces a unified ServiceAccount (
csi-hostpathplugin-sa) with explicit ClusterRoleBindings for both the provisioner and snapshotter roles, following the same pattern used by thekubernetes-latestdeployment.The deploy script now handles the full snapshot infrastructure: CRD installation, snapshot-controller deployment with
--enable-distributed-snapshotting=true, and node-reader RBAC (required for distributed snapshotting but commented out in the upstream snapshot-controller RBAC). If the snapshot-controller was already deployed (e.g., by prow.sh) without the flag, the script patches it. The destroy script cleans up all of these resources.Snapshot E2E tests are enabled via
snapshotDataSourceandSnapshotClassin test-driver.yaml.Which issue(s) this PR fixes:
Part of #651
Special notes for your reviewer:
Builds on prior work in #392 by @denisok, which went stale before merging.
The distributed snapshotting feature in external-snapshotter (kubernetes-csi/external-snapshotter#585) requires coordination between two components:
--enable-distributed-snapshotting=trueto label VolumeSnapshotContent objects with node affinity--node-deployment=trueto filter by those labelsThe upstream snapshot-controller RBAC has Node read permissions commented out. The deploy script applies them via a separate
snapshot-controller-node-readerClusterRole.Tested on a 3-node Kind cluster: provisioning on different workers, snapshot creation and deletion all work correctly.
Does this PR introduce a user-facing change?: