Description
On Bottlerocket 1.57.0, ephemeral volumes backed by the container-image-csi-driver fail to mount with the following error:
MountVolume.SetUp failed for volume "target": rpc error: code = Internal
desc = mount failed: exit status 32, output: mount: wrong fs type, bad option,
bad superblock on overlay
Pods requesting container image-backed ephemeral volumes get stuck in Pending indefinitely.
Root Cause
Bottlerocket 1.57.0 upgraded the kernel from 6.1.161 to 6.1.163. This kernel version introduced a regression that rejects overlay filesystem mounts using the context= SELinux option. The driver passes context= when mounting container image layers via nsenter to suppress per-file SELinux relabelling (which causes pod startup latency).
Kernel 6.1.163 rejects this option, causing all overlay mounts to fail.
Bottlerocket 1.57.0 release notes confirm no SELinux policy changes were made, isolating the issue to a behavioral change in kernel 6.1.163 itself.
Steps to Reproduce
- Deploy the driver on a Kubernetes cluster running Bottlerocket 1.57.0 nodes
- Create a pod with a container image-backed ephemeral volume
- Observe the pod stuck in Pending with the mount error above
Environment
- Bottlerocket version: 1.57.0
- Kernel: 6.1.163
- Driver version: v2.1.5 (current main)
- Runtime: containerd
Additional Context
Clusters running Bottlerocket 1.56.0 (kernel 6.1.161) are unaffected. The context= option was intentionally introduced to prevent SELinux per-file relabelling on ephemeral volume mounts, which causes significant pod startup latency on SELinux-enforcing nodes. Any fix needs to account for this — simply removing context= is not a viable solution.
Description
On Bottlerocket 1.57.0, ephemeral volumes backed by the container-image-csi-driver fail to mount with the following error:
Pods requesting container image-backed ephemeral volumes get stuck in Pending indefinitely.
Root Cause
Bottlerocket 1.57.0 upgraded the kernel from
6.1.161to6.1.163. This kernel version introduced a regression that rejects overlay filesystem mounts using the context= SELinux option. The driver passes context= when mounting container image layers via nsenter to suppress per-file SELinux relabelling (which causes pod startup latency).Kernel 6.1.163 rejects this option, causing all overlay mounts to fail.
Bottlerocket 1.57.0 release notes confirm no SELinux policy changes were made, isolating the issue to a behavioral change in kernel 6.1.163 itself.
Steps to Reproduce
Environment
Additional Context
Clusters running Bottlerocket 1.56.0 (kernel 6.1.161) are unaffected. The context= option was intentionally introduced to prevent SELinux per-file relabelling on ephemeral volume mounts, which causes significant pod startup latency on SELinux-enforcing nodes. Any fix needs to account for this — simply removing context= is not a viable solution.