Fix csi manila upgrade case

[ricolin]

Fix manila storageclass upgrade
provisioner is immutable. for PV. PV in cluster that is built with storageclass that changes provisioner will not be able to upgrade to new storageclass. As we use same storageclass name, old cluster is forcing to upgrade to new storageclass and fail as it ask for provisioner change in PV.

Fix cinder_csi_enabled and manila_csi_enabled type
These two type are passing as string in cluster.labels not bool.

Relate to #740

[okozachenko1203]

The good thing is that we can delete SC(StorageClass) in use without any issue.
So I'd prefer to check current SC's provisioner field and delete if that is old one(without proto prefix). And create a new one. And all those operations in the code, not manual.

[ricolin]

The upgrade is stuck with existing PVs that already have "provisioner" defined and which cannot be easily modified.

[okozachenko1203]

@ricolin i am a bit lost, PV doesn't have provisioner field. It only includes StorageClass field.
So once you delete old storageclass and create new one using the same name, it should be ok, no?

[ricolin]

@okozachenko1203
It's in PVC on

kind: PersistentVolumeClaim
metadata:
  annotations:
    ...
    volume.beta.kubernetes.io/storage-provisioner: manila.csi.openstack.org
    volume.kubernetes.io/storage-provisioner: manila.csi.openstack.org
    ...

And here in PV

apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/provisioned-by: manila.csi.openstack.org
spec:
  ...
  csi:
    ...
    driver: manila.csi.openstack.org
    ...
    volumeAttributes:
      ...
      storage.kubernetes.io/csiProvisionerIdentity: 1722446784123-8081-manila.csi.openstack.org
    ...

[ricolin]

Also currently possible workaround is to recreate PVC with new storage driver and move PV to bond with new PVC as follow:

First, change the delete policy to retain
export target_pv=pvc-7cde8337-d7a7-4ed1-9645-a9983aec3be3
kubectl get pv/$target_pv -o yaml | sed 's/persistentVolumeReclaimPolicy.*/persistentVolumeReclaimPolicy: Retain/g' | kubectl replace -f -
 
recreate pvc and sc
Clear dependency on PV (like attach from pod) to make sure pv can change to status Released
delete pvc manila-pvc like
kubectl delete pvc manila-pvc
recreate sc with "nfs.manila.csi.openstack.org" as provisioner
 
create pvc with
with volumeName: $target_pv
 
remove entire claimRef scope in $target_pv
 
it will re-bond with new pvc
 
After we can edit
metadata.annotations."pv.kubernetes.io/provisioned-by" to nfs.manila.csi.openstack.org
 
as for spec.csi.driver
It can't be changed, but I think it should be okay in this case.
 
Change back the reclaim policy:
kubectl get pv/$target_pv -o yaml | sed 's/persistentVolumeReclaimPolicy.*/persistentVolumeReclaimPolicy: Delete/g' | kubectl replace -f -