odf subvolume ls fails for subvolumes with snapshots / clones

[AaronJaegerVA]

odf subvolume ls fails for subvolumes clones while they are being created from snapshots. This situation occurs frequently when using OpenShift APIs for Data Protection (OADP) to back up OpenShift workloads with the CSI snapshot and Data Mover features enabled.

$ odf subvolume ls
Filesystem  Subvolume  SubvolumeGroup  State
ocs-storagecluster-cephfilesystem csi-vol-21a1fd4a-4fd5-4fe2-bc41-e14608624227 csi in-use
ocs-storagecluster-cephfilesystem csi-vol-b6288008-1abc-49ff-b7b0-3f0c23f8be7a csi in-use
ocs-storagecluster-cephfilesystem csi-vol-ff2cd530-51c3-450b-aa4d-90fa4645abe6 csi in-use
ocs-storagecluster-cephfilesystem csi-vol-444b2472-77b7-4b92-8b2b-f5543581a21d csi in-use
ocs-storagecluster-cephfilesystem csi-vol-97986bc0-64eb-43de-ae97-64463460b6a4 csi in-use
ocs-storagecluster-cephfilesystem csi-vol-ddcbe1c1-f1c5-4785-a74e-98f1ef12f7a5 csi in-use
ocs-storagecluster-cephfilesystem csi-vol-0937be17-f9b9-401b-b4cb-7a587bda1ef0 csi in-use
ocs-storagecluster-cephfilesystem csi-vol-45a6f71c-2b37-47f0-b86f-a567e4bf1b3f csi in-use
ocs-storagecluster-cephfilesystem csi-vol-6f811c2f-6f53-425d-ac34-33908e0e8a69 csi in-use
Error: Error EAGAIN: subvolume 'csi-vol-e906270e-c148-4ee7-9ec8-d4e8cf0e2453' is not ready for operation info
. failed to run command. command terminated with exit code 11%!(EXTRA string=failed to get subvolume info)
Error: failed to get subvolume state: "csi-vol-e906270e-c148-4ee7-9ec8-d4e8cf0e2453" "Error EAGAIN: subvolume 'csi-vol-e906270e-c148-4ee7-9ec8-d4e8cf0e2453' is not ready for operation info\n. failed to run command. command terminated with exit code 11"

[subhamkrai]

@AaronJaegerVA The error should be coming from ceph side, @yati1998 @OdedViner any idea?

[AaronJaegerVA]

@subhamkrai

The trouble is, when the error comes, the command stops and other volumes are not listed. If there was error handling, things would be much better.

[subhamkrai]

@AaronJaegerVA For this command, in the backend it runs ceph command that why it is like that but definetly we can try to work on handling the error.

[yati1998]

is it possible to get the subvolume info output for this: csi-vol-e906270e-c148-4ee7-9ec8-d4e8cf0e2453 , it is coming from ceph only cause we dont have state set. We need to check what is the issue with subvolume, also is it same for all the clones?
If possible feel free to share the setup with me, I can help debug

[AaronJaegerVA]

That subvolume likely no longer exists. It is a temporary clone created by OADP that gets deleted after the backup job completes. If information from another subvolume clone created by OADP in the future will suffice, what ceph commands would I execute to collect it?