Specify path for cephfs dynamic provisioner

[musicderp]

Hey, I'm trying to create a cephfs storage class and specify the folder that dynamically provisioned volumes are saved to, and I'm not quite sure how to do this. I read about subvolume groups but I wasn't sure if those were what I wanted (#1884 (comment))

Here is what my storage class looks like:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-cephfs-test-sc
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: cephfs.csi.ceph.com
parameters:
  # (required) String representing a Ceph cluster to provision storage from.
  # Should be unique across all Ceph clusters in use for provisioning,
  # cannot be greater than 36 bytes in length, and should remain immutable for
  # the lifetime of the StorageClass in use.
  # Ensure to create an entry in the configmap named ceph-csi-config, based on
  # csi-config-map-sample.yaml, to accompany the string chosen to
  # represent the Ceph cluster in clusterID below
  clusterID: redacted

  # (required) CephFS filesystem name into which the volume shall be created
  # eg: fsName: myfs
  fsName: cephfs

  # (optional) Ceph pool into which volume data shall be stored
  pool: cephfs_test

  # (optional) Comma separated string of Ceph-fuse mount options.
  # For eg:
  # fuseMountOptions: debug

  # (optional) Comma separated string of Cephfs kernel mount options.
  # Check man mount.ceph for mount options. For eg:
  # kernelMountOptions: readdir_max_bytes=1048576,norbytes

  # The secrets have to contain user and/or Ceph admin credentials.
  csi.storage.k8s.io/provisioner-secret-name: ceph-key
  csi.storage.k8s.io/provisioner-secret-namespace: cephcsi
  csi.storage.k8s.io/controller-expand-secret-name: ceph-key
  csi.storage.k8s.io/controller-expand-secret-namespace: cephcsi
  csi.storage.k8s.io/node-stage-secret-name: ceph-key
  csi.storage.k8s.io/node-stage-secret-namespace: cephcsi

  # (optional) The driver can use either ceph-fuse (fuse) or
  # ceph kernelclient (kernel).
  # If omitted, default volume mounter will be used - this is
  # determined by probing for ceph-fuse and mount.ceph
  mounter: kernel

  # (optional) Prefix to use for naming subvolumes.
  # If omitted, defaults to "csi-vol-".
  # volumeNamePrefix: "foo-bar-"

reclaimPolicy: Retain
allowVolumeExpansion: true
mountOptions: []

I have other stuff on my cluster so I would ideally have the dynamic provisioner work in /kubernetes/volumes/

I am also aware that I'm able to do manually created static volumes with rootPath under a PV, but that's not quite what I'm looking to do here.

Thanks!

[Rakshith-R]

You'll need to use subvolumegroup in the configmap.

Creating a subvolumegroup kubernetes on the cluster and specifying it in the configmap should be enough.

https://github.com/ceph/ceph-csi/blob/devel/deploy/csi-config-map-sample.yaml