Explore defaulting CSIDriver fsGroupPolicy to File instead of ReadWriteOnceWithFSType

[jadarsie]

Describe the request

Related to #67

Explain why AKS Engine needs it

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[zohebk8s]

@jadarsie I would like to contribute by adding few documents & information. I’ve added pdf documents as well FYR. Please have a look, I’m pretty sure that this information will be helpful & useful.

pod as non-root user - usecase.pdf
case-study-AzureStackHub.pdf

In Azure AKS Pubic Cloud, the container storage component works like below:

“Container Storage component supports volume ownership and permission change via fsGroup, and Kubernetes may use fsGroup to change permissions and ownership of the volume to match user requested fsGroup in the pod's SecurityPolicy regardless of fstype or access mode”
Specific to ext4 Filesystem: In Azure AKS, we don’t need to explicitly define this parameter in Storage Class “fstype: ext4”. As by default, it will consider “fstype: ext4”.

In Azure Stack Hub, the container storage component works like below:

“The volumes will be examined to determine if volume ownership and permissions should be modified to match the pod's security policy.

Changes will only occur if the fsType is defined and the persistent volume's accessModes contains ReadWriteOnce.

Here the condition is, in Storage Class we need to define fstype: ext4. Then only kubernetes may use fsGroup to change permissions and ownership of the volume.

[zohebk8s]

@jadarsie Adding more details. As it's all about "CSIDriver"

In Azure AKS - Public Cloud - CSIDriver Configuration/Parameters:

In Azure Stack Hub - CSIDriver Configuration/Parameters:

The following modes are supported:

None: Indicates that volumes will be mounted with no modifications, as the CSI volume driver does not support these operations.

File: Indicates that the CSI volume driver supports volume ownership and permission change via fsGroup, and Kubernetes may use fsGroup to change permissions and ownership of the volume to match user requested fsGroup in the pod's SecurityPolicy regardless of fstype or access mode.

ReadWriteOnceWithFSType: Indicates that volumes will be examined to determine if volume ownership and permissions should be modified to match the pod's security policy. Changes will only occur if the fsType is defined and the persistent volume's accessModes contain ReadWriteOnce.

Note: For Azure Stack Hub, we need to define "fsType: ext4" parameter in storageclass. As in CSIDriver the mode is "ReadWriteOnceWithFSType"

[zohebk8s]

@jadarsie As It's all about CSIDriver, so change/update should be at the CSIDriver level.

I’ve checked the CSIDriver version 1.20.0 is already having the parameter "fsGroupPolicy: File"

Solution: If using older versions of CSIDriver like 1.10.0, then upgrade the CSIDriver version to below:(It should be tested first in Sabdbox/lower environment clusters first, before implementing in Production)

kubectl delete CSIDriver disk.csi.azure.com

kubectl apply -f https://github.com/kubernetes-sigs/azuredisk-csi-driver/blob/v1.20.0/deploy/csi-azuredisk-driver.yaml

[jadarsie]

Thx @zohebk8s for your investigation.

Meanwhile, would it work for you to disable the CSI addon in your API Model and manually deploy the Helm chart? fsGroupPolicy: File is set by default.

• This is the value that controls the CSIDriver spec: enableFSGroupPolicy.
• This is how the value is used in the csi-azuredisk-driver.yaml template.

[zohebk8s]

@jadarsie Thanks for acknowledging my efforts, I hope the information which I’ve shared above is helpful. I just thought to share this content with you & community.

Going forward, if we need new AKS clusters(Azure Stack Hub). Then I will deploy with helm using above option.