pubsubplus-ha chart PVC gets stuck on PreSync hook when we deploy it via ArgoCD

[nikhilbhadane10]

Hi Team,
We are trying to deploy the pubsubplus-ha via ArgoCD, But while creating the PVC it get stuck on PVC PreSync hook.

Message: "waiting for completion of hook /PersistentVolumeClaim/data-solace-pubsubplus-internal-2"

We tried to deploy the helm chart on AWS (EKS) & Azure (AKS) via ArgoCD app, but same result.
As per your documentation[1] & [2], we should use the storageClass with WaitForFirstConsumer binding mode. But that not works when we try to deploy the helm-chart via ArgoCD apps.

[1] https://docs.solace.com/Cloud/Deployment-Considerations/installing-ps-cloud-k8s-eks-specific-req.htm#eks_storage_class
[2] https://docs.solace.com/Cloud/Deployment-Considerations/installing-ps-cloud-k8s-aks-specific-req.htm

Is there any way we can fix this issue by doing some change on helm-chart templates or Can we use EFS/Azure file StorageClass for PVC creation? Is it recommended for prod environments?

[bczoma]

Hi @nikhilbhadane10 , this hook is controlled by https://github.com/SolaceProducts/pubsubplus-kubernetes-helm-quickstart/blob/master/pubsubplus/templates/solacePVCForMonitor.yaml
You could try to remove it in your local chart. However, chances are there may be general issues with the storage class and the rest of the deployment still fails. For comparison, you could also try if it works without the ArgoCD apps.

[nikhilbhadane10]

Hi @bczoma ,
Thanks for your inputs, I have tried pubsubplus-ha by removing the solacePVCForMonitor.yaml file locally it started the deployment of statefulset but somehow it is failing with healthchecks. I have also tried by commenting the hook annotation from PVC manifests but no luck.
Is there any alternatives we can try to deploy the pubsubplus-ha with ArgoCD deployment.

[nikhilbhadane10]

Hi Team,
Any update on the above issue?

[heidyv01]

Hi,
I'm having the same issues with the ArgoCD deployment. Would be great if there were some news on this issue as it makes monitoring unavailable for productiion environments using Argo...
Thanks!

[heidyv01]

I found an easy workaround that might help until there is a solution for this issue. Just manually run a temp pod with the PVC claim so the PVC gets a first consumer and all other resources are deployed. Then you can just delete the pod.
The pod could look like this:

apiVersion: v1
kind: Pod
metadata:
  name: task-pv-pod
  namespace: solace
spec:
  volumes:
    - name: task-pv-storage
      persistentVolumeClaim:
        claimName: data-solace-pubsubplus-2
  containers:
    - name: task-pv-container
      image: nginx
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: task-pv-storage

[nikhilbhadane10]

Readiness probes gets failed after some time. ArgoCD app goes into Progressing state. It seems same issue when we tried to delete solacePVCForMonitor.yaml from helm-chart & deploy it without PVC file via ArgoCD.