In case of invalid GUI job id error return, volume cloning and restore operations should be retried

[vrushch]

Describe the bug
if GUI return invalid job id then driver is not retrying mmxcp, this results in pvc being in Pending state.

To Reproduce
Steps to reproduce the behavior:

1. Create clones in 1000's to hit this issue

Expected behavior
driver should retry mmxcp when invalid job id is obtained from GUI

Environment
Please run the following an paste your output here:

# Deployment
[root@remote-csi-master yamls]# kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:38:50Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:32:41Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
[root@remote-csi-master yamls]#

[root@remote-csi-worker-2 ~]# rpm -qa | grep gpfs
gpfs.msg.en_US-5.1.2-1.noarch
gpfs.adv-5.1.2-1.x86_64
gpfs.gpl-5.1.2-1.noarch
gpfs.afm.cos-1.0.0-4.x86_64
gpfs.librdkafka-5.1.2-1.el7.x86_64
gpfs.java-5.1.2-1.x86_64
gpfs.base-5.1.2-1.x86_64
gpfs.license.adv-5.1.2-1.x86_64
gpfs.crypto-5.1.2-1.x86_64
gpfs.gskit-8.0.55-19.1.x86_64
gpfs.docs-5.1.2-1.noarch
gpfs.compression-5.1.2-1.x86_64
gpfs.gss.pmsensors-5.1.2-1.el7.x86_64
[root@remote-csi-worker-2 ~]#

[Jainbrt]

@amdabhad could you please confirm if this is fixed with latest gui?

[amdabhad]

Hi @Jainbrt , this is not yet fixed.

[amdabhad]

Another instance of volume cloning failure due to mmxcp failure: #843