Couldn't find any host available to create Volume

[jLemmings]

I've deployed the synology-csi with the helm chart. All the pods started as expected without any errors. Once I create a PVC with the synology-csi-iscsi-retain storageclass it fails and I am unsure where the issue lies.

The deployment

---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: synology-csi
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  destination:
    namespace: synology-csi
    name: in-cluster
  project: default
  sources:
    - chart: synology-csi
      repoURL: https://christian-schlichtherle.github.io/synology-csi-chart
      targetRevision: 0.10.1
      helm:
        releaseName: synology-csi
        values: |
          storageClasses:
            iscsi-retain:
              reclaimPolicy: Retain
    - path: applications/synology-csi
      repoURL: git.git
      targetRevision: HEAD
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: synology-csi-client-info
  namespace: synology-csi
spec:
  refreshInterval: "1h"
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: synology-csi-client-info
    template:
      type: Opaque
      data:
        client-info.yaml: |
          ---
          clients:
          - host: 1.1.1.1
            https: true
            password: "{{ .password }}"
            port: 5001
            username: "{{ .username }}"
  data:
    - secretKey: username
      remoteRef:
        key: synology
        property: username
    - secretKey: password
      remoteRef:
        key: synology
        property: password
---
apiVersion: v1
kind: Namespace
metadata:
  name: synology-csi

The errors

k logs -n synology-csi synology-csi-controller-0:

I0106 18:52:48.997171       1 connection.go:194] GRPC request: {"capacity_range":{"required_bytes":5000000000},"name":"pvc-8f96d873-f46d-4413-93b3-9ff2791ef487","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8f96d873-f46d-4413-93b3-9ff2791ef487","csi.storage.k8s.io/pvc/name":"test-claim","csi.storage.k8s.io/pvc/namespace":"default"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":5}}]}
I0106 18:52:48.997277       1 event.go:298] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-claim", UID:"8f96d873-f46d-4413-93b3-9ff2791ef487", APIVersion:"v1", ResourceVersion:"571003318", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/test-claim"
I0106 18:52:49.096191       1 reflector.go:378] sigs.k8s.io/sig-storage-lib-external-provisioner/v9/controller/controller.go:845: forcing resync
I0106 18:52:52.218867       1 connection.go:200] GRPC response: {}
I0106 18:52:52.218878       1 connection.go:201] GRPC error: rpc error: code = Internal desc = Couldn't find any host available to create Volume
I0106 18:52:52.218888       1 controller.go:816] CreateVolume failed, supports topology = false, node selected false => may reschedule = false => state = Finished: rpc error: code = Internal desc = Couldn't find any host available to create Volume
I0106 18:52:52.218914       1 controller.go:1075] Final error received, removing PVC 8f96d873-f46d-4413-93b3-9ff2791ef487 from claims in progress
W0106 18:52:52.218922       1 controller.go:934] Retrying syncing claim "8f96d873-f46d-4413-93b3-9ff2791ef487", failure 5
E0106 18:52:52.218937       1 controller.go:957] error syncing claim "8f96d873-f46d-4413-93b3-9ff2791ef487": failed to provision volume with StorageClass "synology-csi-iscsi-retain": rpc error: code = Internal desc = Couldn't find any host available to create Volume
I0106 18:52:52.218997       1 event.go:298] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-claim", UID:"8f96d873-f46d-4413-93b3-9ff2791ef487", APIVersion:"v1", ResourceVersion:"571003318", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "synology-csi-iscsi-retain": rpc error: code = Internal desc = Couldn't find any host available to create Volume
I0106 18:53:01.026563       1 controller.go:1366] provision "default/test-claim" class "synology-csi-iscsi-retain": started
I0106 18:53:01.026630       1 connection.go:193] GRPC call: /csi.v1.Controller/CreateVolume
I0106 18:53:01.026689       1 event.go:298] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-claim", UID:"8f96d873-f46d-4413-93b3-9ff2791ef487", APIVersion:"v1", ResourceVersion:"571003318", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/test-claim"
I0106 18:53:01.026638       1 connection.go:194] GRPC request: {"capacity_range":{"required_bytes":5000000000},"name":"pvc-8f96d873-f46d-4413-93b3-9ff2791ef487","parameters":{"csi.storage.k8s.io/pv/name":"pvc-8f96d873-f46d-4413-93b3-9ff2791ef487","csi.storage.k8s.io/pvc/name":"test-claim","csi.storage.k8s.io/pvc/namespace":"default"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":5}}]}
I0106 18:53:04.351677       1 connection.go:200] GRPC response: {}
I0106 18:53:04.351689       1 connection.go:201] GRPC error: rpc error: code = Internal desc = Couldn't find any host available to create Volume
I0106 18:53:04.351701       1 controller.go:816] CreateVolume failed, supports topology = false, node selected false => may reschedule = false => state = Finished: rpc error: code = Internal desc = Couldn't find any host available to create Volume
I0106 18:53:04.351750       1 controller.go:1075] Final error received, removing PVC 8f96d873-f46d-4413-93b3-9ff2791ef487 from claims in progress
W0106 18:53:04.351761       1 controller.go:934] Retrying syncing claim "8f96d873-f46d-4413-93b3-9ff2791ef487", failure 6
E0106 18:53:04.351779       1 controller.go:957] error syncing claim "8f96d873-f46d-4413-93b3-9ff2791ef487": failed to provision volume with StorageClass "synology-csi-iscsi-retain": rpc error: code = Internal desc = Couldn't find any host available to create Volume
I0106 18:53:04.351801       1 event.go:298] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"test-claim", UID:"8f96d873-f46d-4413-93b3-9ff2791ef487", APIVersion:"v1", ResourceVersion:"571003318", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "synology-csi-iscsi-retain": rpc error: code = Internal desc = Couldn't find any host available to create Volume

k describe pvc:

Events:
  Type     Reason                Age                     From                                                                     Message
  ----     ------                ----                    ----                                                                     -------
  Normal   ExternalProvisioning  2m20s (x26 over 8m35s)  persistentvolume-controller                                              Waiting for a volume to be created either by the external provisioner 'csi.san.synology.com' or manually by the system administrator. If volume creation is delayed, please verify that the provisioner is running and correctly registered.
  Normal   Provisioning          6s (x10 over 8m35s)     csi.san.synology.com_k8s-master-01_70c3ef6d-9b51-4b65-804e-0baeab7372a6  External provisioner is provisioning volume for claim "default/test-claim"
  Warning  ProvisioningFailed    2s (x10 over 8m29s)     csi.san.synology.com_k8s-master-01_70c3ef6d-9b51-4b65-804e-0baeab7372a6  failed to provision volume with StorageClass "synology-csi-iscsi-retain": rpc error: code = Internal desc = Couldn't find any host available to create Volume

Synology UI screenshots

As the LUN's and iSCSI objects were created I assume I am not too far off. What am I missing or where should I check further to figure out what is not configured correctly?

[landre-cerp]

Same here, not found why, yet.