Skip to content

Use hard-coded kubelet path for s3-plugin container#656

Merged
yerzhan7 merged 3 commits into
awslabs:mainfrom
tadiwa-aizen:fix-microk8s-socket-path
Mar 24, 2026
Merged

Use hard-coded kubelet path for s3-plugin container#656
yerzhan7 merged 3 commits into
awslabs:mainfrom
tadiwa-aizen:fix-microk8s-socket-path

Conversation

@tadiwa-aizen
Copy link
Copy Markdown
Contributor

@tadiwa-aizen tadiwa-aizen commented Dec 4, 2025

*Issue #577 *

Description

On MicroK8s (snap install), kubelet uses /var/snap/microk8s/common/var/lib/kubelet as its root directory. In CSI Driver v2, Unix socket paths built from this base exceed Linux's 108-byte UNIX_PATH_MAX limit, causing bind() / connect() to fail with invalid argument. The driver either fails to register with kubelet or fails to communicate with Mountpoint pods, making S3 mounts impossible. This affects any Kubernetes distribution with a long kubelet path.

Fix
Hard-code the container-side mountPath to /var/lib/kubelet regardless of the host path. The host path remains
configurable via node.kubeletPath (used as the volume source), but inside the container all paths are short.

Replace the single KUBELET_PATH env var with:

  • CONTAINER_KUBELET_PATH=/var/lib/kubelet — path as seen inside the container
  • HOST_KUBELET_PATH — path as seen on the host (from node.kubeletPath)

Add KubeletHostPathToContainerPath() to translate host paths (received from kubelet RPCs) to container paths in both NodePublishVolume and NodeUnpublishVolume.

Verification
Tested on Ubuntu 22.04 with MicroK8s (snap, channel 1.32) and node.kubeletPath=/var/snap/microk8s/common/var/lib/kubelet.

Without the fix, the driver fails to register as reported in Issue #577 — the registration socket path exceeds Linux's 108-byte limit.

With the fix — driver registers successfully and all 3 node containers are running:

$ microk8s kubectl get pods -n kube-system | grep s3-csi
s3-csi-controller-558c76774b-4mc8f         1/1     Running   0          5m
s3-csi-node-spmkn                          3/3     Running   0          5m
$ microk8s kubectl logs -n kube-system -l app=s3-csi-node -c node-driver-registrar
I0322 14:02:26.423604       1 main.go:145] "Version" version="v2.16.0-eksbuild.1"
I0322 14:02:26.428943       1 node_register.go:56] "Starting Registration Server" socketPath="/registration/s3.csi.aws.com-reg.sock"
I0322 14:02:26.429167       1 node_register.go:66] "Registration Server started" socketPath="/registration/s3.csi.aws.com-reg.sock"
I0322 14:02:26.965157       1 main.go:102] "Received NotifyRegistrationStatus call" status="plugin_registered:true"

S3 mount works — workload pod writes to the bucket:

$ microk8s kubectl exec s3-app -- ls -la /data/
$ aws s3 ls tadiwaom-s3-test/
...
2026-03-22 21:30:34         26 Sun Mar 22 21:30:32 UTC 2026.txt

Testing
Manual Testing with E2E tests passing: (rosa was not set up in my CI)
https://github.com/tadiwa-aizen/mountpoint-s3-csi-driver/actions/runs/23405969976

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@tadiwa-aizen tadiwa-aizen requested a review from a team as a code owner December 4, 2025 17:32
Comment thread charts/aws-mountpoint-s3-csi-driver/templates/node.yaml Outdated
Comment thread pkg/driver/node/node.go Outdated
Comment thread pkg/driver/node/node_test.go
Comment thread pkg/util/kubelet.go
Comment thread pkg/driver/node/node.go Outdated
Comment thread pkg/util/kubelet.go Outdated
Comment thread pkg/util/kubelet.go
Comment thread charts/aws-mountpoint-s3-csi-driver/templates/node.yaml Outdated
Comment thread pkg/util/kubelet.go Outdated
Comment thread pkg/driver/node/node.go Outdated
Comment thread pkg/util/kubelet.go Outdated
Comment thread pkg/util/kubelet.go
Comment thread pkg/driver/node/node.go Outdated
Comment thread pkg/driver/node/node.go Outdated
Comment thread pkg/driver/node/node.go Outdated
Comment thread pkg/driver/node/node.go
Signed-off-by: Tadiwa Magwenzi <tadiwaom@amazon.com>
Signed-off-by: Tadiwa Magwenzi <tadiwaom@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants