failed to install on source: failed to install helm chart: context deadline exceeded

[awakzdev]

Describe the bug
Migration is failing due to helm chart installation error.

To Reproduce
Two different GKE clusters on different zones.
Command used :

pv-migrate migrate `
>   --source-context gke_carwiz-97e29_europe-north1-a_production `
>   --source-namespace monitoring `
>   --source-kubeconfig "C:\Users\zenmy\.kube\config" `
>   --ignore-mounted `
>   --dest-context gke_carwiz-97e29_me-west1-a_date-gke-cluster `
>   --dest-namespace monitoring `
>   --log-format json `
>   monitoring-grafana monitoring-grafana

Console output

{"level":"info","msg":"Starting migration","time":"2024-02-05T17:43:20+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","level":"info","msg":"PVC monitoring-grafana is mounted to node gke-production-pool-large-2-5c1e5f04-wlza, ignoring...","source":"monitoring-grafana","source_ns":"monitoring","time":"2024-02-05T17:43:23+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","level":"info","msg":"Will attempt 3 strategies: mnt2, svc, lbsvc","source":"monitoring-grafana","source_ns":"monitoring","strategies":"mnt2, svc, lbsvc","time":"2024-02-05T17:43:23+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","id":"abede","level":"info","msg":"Attempting strategy: mnt2","source":"monitoring-grafana","source_ns":"monitoring","strategy":"mnt2","time":"2024-02-05T17:43:23+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","error":"unaccepted","id":"abede","level":"warning","msg":"Migration failed with this strategy, will try with the remaining strategies","source":"monitoring-grafana","source_ns":"monitoring","strategy":"mnt2","time":"2024-02-05T17:43:23+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","id":"eaebe","level":"info","msg":"Attempting strategy: svc","source":"monitoring-grafana","source_ns":"monitoring","strategy":"svc","time":"2024-02-05T17:43:23+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","error":"unaccepted","id":"eaebe","level":"warning","msg":"Migration failed with this strategy, will try with the remaining strategies","source":"monitoring-grafana","source_ns":"monitoring","strategy":"svc","time":"2024-02-05T17:43:23+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","id":"adcce","level":"info","msg":"Attempting strategy: lbsvc","source":"monitoring-grafana","source_ns":"monitoring","strategy":"lbsvc","time":"2024-02-05T17:43:23+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","level":"info","msg":"Generating SSH key pair","source":"monitoring-grafana","source_ns":"monitoring","time":"2024-02-05T17:43:23+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","id":"adcce","level":"info","msg":"Cleaning up","source":"monitoring-grafana","source_ns":"monitoring","time":"2024-02-05T17:44:38+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","id":"adcce","level":"info","msg":"Cleanup done","source":"monitoring-grafana","source_ns":"monitoring","time":"2024-02-05T17:45:11+02:00"}
{"dest":"monitoring-grafana","dest_ns":"monitoring","error":"failed to install on source: failed to install helm chart: context deadline exceeded","id":"adcce","level":"warning","msg":"Migration failed with this strategy, will try with the remaining strategies","source":"monitoring-grafana","source_ns":"monitoring","strategy":"lbsvc","time":"2024-02-05T17:45:11+02:00"}
Error: migration failed: all strategies failed for this migration
Usage:
  pv-migrate migrate <source-pvc> <dest-pvc> [flags]

Aliases:
  migrate, m

Flags:
  -C, --dest-context string            context in the kubeconfig file of the destination PVC
  -d, --dest-delete-extraneous-files   delete extraneous files on the destination by using rsync's '--delete' flag
  -H, --dest-host-override string      the override for the rsync host destination when it is run over SSH, in cases when you need to target a different destination IP on rsync for some reason. By default, it is determined by used strategy and differs across strategies. Has no effect for mnt2 and local strategies
  -K, --dest-kubeconfig string         path of the kubeconfig file of the destination PVC
  -N, --dest-namespace string          namespace of the destination PVC
  -P, --dest-path string               the filesystem path to migrate in the destination PVC (default "/")
      --helm-set strings               set additional Helm values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
      --helm-set-file strings          set additional Helm values from respective files specified via the command line (can specify multiple or separate values with commas: key1=path1,key2=path2)      
      --helm-set-string strings        set additional Helm STRING values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
  -t, --helm-timeout duration          install/uninstall timeout for helm releases (default 1m0s)
  -f, --helm-values strings            set additional Helm values by a YAML file or a URL (can specify multiple)
  -h, --help                           help for migrate
  -i, --ignore-mounted                 do not fail if the source or destination PVC is mounted
      --lbsvc-timeout duration         timeout for the load balancer service to receive an external IP. Only used by the lbsvc strategy (default 2m0s)
  -o, --no-chown                       omit chown on rsync
  -b, --no-progress-bar                do not display a progress bar
  -x, --skip-cleanup                   skip cleanup of the migration
  -c, --source-context string          context in the kubeconfig file of the source PVC
  -k, --source-kubeconfig string       path of the kubeconfig file of the source PVC
  -R, --source-mount-read-only         mount the source PVC in ReadOnly mode (default true)
  -n, --source-namespace string        namespace of the source PVC
  -p, --source-path string             the filesystem path to migrate in the source PVC (default "/")
  -a, --ssh-key-algorithm string       ssh key algorithm to be used. Valid values are rsa,ed25519 (default "ed25519")
  -s, --strategies strings             the comma-separated list of strategies to be used in the given order (default [mnt2,svc,lbsvc])

Global Flags:
      --log-format string   log format, must be one of: json, fancy (default "fancy")
      --log-level string    log level, must be one of: trace, debug, info, warn, error, fatal, panic (default "info")

{"level":"error","msg":"Error: migration failed: all strategies failed for this migration","time":"2024-02-05T17:45:11+02:00"}

Version

• Source and destination Kubernetes versions - source 1.25.16-gke.1041000, destination 1.26.10-gke.1101000
• Source and destination container runtimes - containerd
• pv-migrate version and architecture - pv-migrate version 1.7.1 (commit: 1affa11b175d20969b9d6f2879c09dc94f0b4a0f) (build date: 2023-10-09T21:56:55Z)
• Installation method - scoop
• Source and destination PVC type, size and accessModes

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 annotations:
   pv.kubernetes.io/bind-completed: "yes"
   pv.kubernetes.io/bound-by-controller: "yes"
   pv.kubernetes.io/migrated-to: pd.csi.storage.gke.io
   volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd
   volume.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd
 finalizers:
 - kubernetes.io/pvc-protection
 labels:
   app.kubernetes.io/instance: monitoring
   app.kubernetes.io/managed-by: Helm
   app.kubernetes.io/name: grafana
   app.kubernetes.io/version: 9.2.3
   argocd.argoproj.io/instance: monitoring-monitoring
   helm.sh/chart: grafana-6.43.3
 name: monitoring-grafana
 namespace: monitoring
spec:
 accessModes:
 - ReadWriteOnce
 resources:
   requests:
     storage: 10Gi
 storageClassName: standard
 volumeMode: Filesystem
status:
 accessModes:
 - ReadWriteOnce
 capacity:
   storage: 10Gi
 phase: Bound