Fix object path trimming in Bucket Object Path mount support #288

Prachi030510 · 2025-12-26T12:32:49Z

Describe your changes

Issue ticket number and link

Checklist before requesting a review

I have performed a self-review of my code
If it is a core feature, I have added thorough tests.
I have made sure existing UTs are not impacted
I have executed all necessary linter and made sure internal pipeline/travis is not broken

Prachi030510 · 2025-12-26T12:49:32Z

s3fs

I1226 11:47:45.703645       1 server.go:168] GRPC response:
I1226 11:47:46.157230       1 server.go:163] GRPC call: /csi.v1.Controller/CreateVolume
I1226 11:47:46.157338       1 controllerserver.go:59] CSIControllerServer-CreateVolume: Request: name:"pvc-0c5c105b-4a69-4637-91f3-eab4a55de77e" capacity_range:{required_bytes:268435456} volume_capabilities:{mount:{mount_flags:"multipart_size=62" mount_flags:"max_dirty_data=51200" mount_flags:"parallel_count=8" mount_flags:"max_stat_cache_size=100000" mount_flags:"retries=5" mount_flags:"kernel_cache"} access_mode:{mode:SINGLE_NODE_WRITER}} parameters:{key:"client" value:"awss3"} parameters:{key:"cosEndpoint" value:"https://s3.au-syd.cloud-object-storage.appdomain.cloud"} parameters:{key:"csi.storage.k8s.io/pv/name" value:"pvc-0c5c105b-4a69-4637-91f3-eab4a55de77e"} parameters:{key:"csi.storage.k8s.io/pvc/name" value:"objectpath-s3fs-gfinal-pvc"} parameters:{key:"csi.storage.k8s.io/pvc/namespace" value:"default"} parameters:{key:"locationConstraint" value:"au-syd-standard"} parameters:{key:"mounter" value:"s3fs"} accessibility_requirements:{requisite:{segments:{key:"topology.kubernetes.io/region" value:"au-syd"} segments:{key:"topology.kubernetes.io/zone" value:"au-syd-1"}} requisite:{segments:{key:"topology.kubernetes.io/region" value:"au-syd"} segments:{key:"topology.kubernetes.io/zone" value:"au-syd-2"}} requisite:{segments:{key:"topology.kubernetes.io/region" value:"au-syd"} segments:{key:"topology.kubernetes.io/zone" value:"au-syd-3"}} preferred:{segments:{key:"topology.kubernetes.io/region" value:"au-syd"} segments:{key:"topology.kubernetes.io/zone" value:"au-syd-2"}} preferred:{segments:{key:"topology.kubernetes.io/region" value:"au-syd"} segments:{key:"topology.kubernetes.io/zone" value:"au-syd-3"}} preferred:{segments:{key:"topology.kubernetes.io/region" value:"au-syd"} segments:{key:"topology.kubernetes.io/zone" value:"au-syd-1"}}}
I1226 11:47:46.157569       1 controllerserver.go:69] Got a request to create volume: pvc-0c5c105b-4a69-4637-91f3-eab4a55de77e
I1226 11:47:46.157597       1 controllerserver.go:76] Volume capability: mount:{mount_flags:"multipart_size=62" mount_flags:"max_dirty_data=51200" mount_flags:"parallel_count=8" mount_flags:"max_stat_cache_size=100000" mount_flags:"retries=5" mount_flags:"kernel_cache"} access_mode:{mode:SINGLE_NODE_WRITER}
I1226 11:47:46.157624       1 controllerserver.go:86] CreateVolume Parameters:
	map[client:awss3 cosEndpoint:https://s3.au-syd.cloud-object-storage.appdomain.cloud csi.storage.k8s.io/pv/name:pvc-0c5c105b-4a69-4637-91f3-eab4a55de77e csi.storage.k8s.io/pvc/name:objectpath-s3fs-gfinal-pvc csi.storage.k8s.io/pvc/namespace:default locationConstraint:au-syd-standard mounter:s3fs]
I1226 11:47:46.157645       1 controllerserver.go:89] req.GetSecrets() length:	0
I1226 11:47:46.157657       1 controllerserver.go:93] Did not find the secret that matches pvc name. Fetching custom secret from PVC annotations
I1226 11:47:46.177603       1 controllerserver.go:111] pvc annotations:
	map[cos.csi.driver/secret:objectpath-s3fs-final-secret cos.csi.driver/secret-namespace:default kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{"cos.csi.driver/secret":"objectpath-s3fs-final-secret","cos.csi.driver/secret-namespace":"default"},"name":"objectpath-s3fs-gfinal-pvc","namespace":"default"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"256Mi"}},"storageClassName":"ibm-object-storage-standard-s3fs"}}
 volume.beta.kubernetes.io/storage-provisioner:cos.s3.csi.ibm.io volume.kubernetes.io/storage-provisioner:cos.s3.csi.ibm.io]
I1226 11:47:46.187014       1 controllerserver.go:493] -parseCustomSecret-
I1226 11:47:46.187045       1 controllerserver.go:133] custom secret parameters parsed successfully, length of custom secret: 11
I1226 11:47:46.187060       1 controllerserver.go:136] volume_id:"pvc-0c5c105b-4a69-4637-91f3-eab4a55de77e" objectPath found in secret: "/data"
I1226 11:47:46.187068       1 controllerserver.go:449] - getObjectStorageCredentialsFromSecret-
I1226 11:47:46.187084       1 controllerserver.go:201] cosEndpoint and locationConstraint getting paased to ObjectStorageSession: https://s3.au-syd.cloud-object-storage.appdomain.cloud, au-syd-standard
I1226 11:47:46.187186       1 controllerserver.go:207] Bucket name provided: psa-addon-object-path-bucket-4
I1226 11:47:46.187212       1 controllerserver.go:208] Check if the provided bucket already exists: psa-addon-object-path-bucket-4
I1226 11:47:46.798916       1 controllerserver.go:268] create volume: pvc-0c5c105b-4a69-4637-91f3-eab4a55de77e
I1226 11:47:46.798949       1 server.go:168] GRPC response: volume:{capacity_bytes:268435456 volume_id:"pvc-0c5c105b-4a69-4637-91f3-eab4a55de77e" volume_context:{key:"bucketName" value:"psa-addon-object-path-bucket-4"} volume_context:{key:"client" value:"awss3"} volume_context:{key:"cosEndpoint" value:"https://s3.au-syd.cloud-object-storage.appdomain.cloud"} volume_context:{key:"csi.storage.k8s.io/pv/name" value:"pvc-0c5c105b-4a69-4637-91f3-eab4a55de77e"} volume_context:{key:"csi.storage.k8s.io/pvc/name" value:"objectpath-s3fs-gfinal-pvc"} volume_context:{key:"csi.storage.k8s.io/pvc/namespace" value:"default"} volume_context:{key:"locationConstraint" value:"au-syd-standard"} volume_context:{key:"mounter" value:"s3fs"} volume_context:{key:"objectPath" value:"/data"} volume_context:{key:"userProvidedBucket" value:"true"}}

mssachan

Added comments. PTAL.
Also remove e2e test results.txt file attached in the PR.

pkg/mounter/mounter-rclone.go

pkg/mounter/mounter-s3fs.go

pkg/s3client/s3client_test.go

pkg/mounter/mounter-rclone.go

Prachi030510 · 2025-12-29T10:38:25Z

rclone -

I1229 10:35:01.184820       1 server.go:168] GRPC response:
I1229 10:35:05.015198       1 server.go:163] GRPC call: /csi.v1.Controller/CreateVolume
I1229 10:35:05.015276       1 controllerserver.go:59] CSIControllerServer-CreateVolume: Request: name:"pvc-ce088145-7ada-4e65-b895-3cd5c6571eb8"  capacity_range:{required_bytes:268435456}  volume_capabilities:{mount:{mount_flags:"acl=private"  mount_flags:"bucket_acl=private"  mount_flags:"upload_cutoff=256Mi"  mount_flags:"chunk_size=64Mi"  mount_flags:"max_upload_parts=64"  mount_flags:"upload_concurrency=20"  mount_flags:"copy_cutoff=1Gi"  mount_flags:"memory_pool_flush_time=30s"  mount_flags:"disable_checksum=true"}  access_mode:{mode:SINGLE_NODE_WRITER}}  parameters:{key:"client"  value:"awss3"}  parameters:{key:"cosEndpoint"  value:"https://s3.au-syd.cloud-object-storage.appdomain.cloud"}  parameters:{key:"csi.storage.k8s.io/pv/name"  value:"pvc-ce088145-7ada-4e65-b895-3cd5c6571eb8"}  parameters:{key:"csi.storage.k8s.io/pvc/name"  value:"objectpath-rclone-gafinal-pvc1"}  parameters:{key:"csi.storage.k8s.io/pvc/namespace"  value:"default"}  parameters:{key:"locationConstraint"  value:"au-syd-standard"}  parameters:{key:"mounter"  value:"rclone"}  accessibility_requirements:{requisite:{segments:{key:"topology.kubernetes.io/region"  value:"au-syd"}  segments:{key:"topology.kubernetes.io/zone"  value:"au-syd-1"}}  requisite:{segments:{key:"topology.kubernetes.io/region"  value:"au-syd"}  segments:{key:"topology.kubernetes.io/zone"  value:"au-syd-2"}}  requisite:{segments:{key:"topology.kubernetes.io/region"  value:"au-syd"}  segments:{key:"topology.kubernetes.io/zone"  value:"au-syd-3"}}  preferred:{segments:{key:"topology.kubernetes.io/region"  value:"au-syd"}  segments:{key:"topology.kubernetes.io/zone"  value:"au-syd-3"}}  preferred:{segments:{key:"topology.kubernetes.io/region"  value:"au-syd"}  segments:{key:"topology.kubernetes.io/zone"  value:"au-syd-1"}}  preferred:{segments:{key:"topology.kubernetes.io/region"  value:"au-syd"}  segments:{key:"topology.kubernetes.io/zone"  value:"au-syd-2"}}}
I1229 10:35:05.015527       1 controllerserver.go:69] Got a request to create volume: pvc-ce088145-7ada-4e65-b895-3cd5c6571eb8
I1229 10:35:05.015539       1 controllerserver.go:76] Volume capability: mount:{mount_flags:"acl=private"  mount_flags:"bucket_acl=private"  mount_flags:"upload_cutoff=256Mi"  mount_flags:"chunk_size=64Mi"  mount_flags:"max_upload_parts=64"  mount_flags:"upload_concurrency=20"  mount_flags:"copy_cutoff=1Gi"  mount_flags:"memory_pool_flush_time=30s"  mount_flags:"disable_checksum=true"}  access_mode:{mode:SINGLE_NODE_WRITER}
I1229 10:35:05.015566       1 controllerserver.go:86] CreateVolume Parameters:
	map[client:awss3 cosEndpoint:https://s3.au-syd.cloud-object-storage.appdomain.cloud csi.storage.k8s.io/pv/name:pvc-ce088145-7ada-4e65-b895-3cd5c6571eb8 csi.storage.k8s.io/pvc/name:objectpath-rclone-gafinal-pvc1 csi.storage.k8s.io/pvc/namespace:default locationConstraint:au-syd-standard mounter:rclone]
I1229 10:35:05.015600       1 controllerserver.go:89] req.GetSecrets() length:	0
I1229 10:35:05.015612       1 controllerserver.go:93] Did not find the secret that matches pvc name. Fetching custom secret from PVC annotations
I1229 10:35:05.036035       1 controllerserver.go:111] pvc annotations:
	map[cos.csi.driver/secret:objectpath-rclone-final-secret2 cos.csi.driver/secret-namespace:default kubectl.kubernetes.io/last-applied-configuration:{"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{"cos.csi.driver/secret":"objectpath-rclone-final-secret2","cos.csi.driver/secret-namespace":"default"},"name":"objectpath-rclone-gafinal-pvc1","namespace":"default"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"256Mi"}},"storageClassName":"ibm-object-storage-standard-rclone"}}
 volume.beta.kubernetes.io/storage-provisioner:cos.s3.csi.ibm.io volume.kubernetes.io/storage-provisioner:cos.s3.csi.ibm.io]
I1229 10:35:05.045515       1 controllerserver.go:493] -parseCustomSecret-
I1229 10:35:05.045551       1 controllerserver.go:133] custom secret parameters parsed successfully, length of custom secret: 11
I1229 10:35:05.045568       1 controllerserver.go:136] volume_id:"pvc-ce088145-7ada-4e65-b895-3cd5c6571eb8" objectPath found in secret: "/data"
I1229 10:35:05.045576       1 controllerserver.go:449] - getObjectStorageCredentialsFromSecret-
I1229 10:35:05.045605       1 controllerserver.go:201] cosEndpoint and locationConstraint getting paased to ObjectStorageSession: https://s3.au-syd.cloud-object-storage.appdomain.cloud, au-syd-standard
I1229 10:35:05.045724       1 controllerserver.go:207] Bucket name provided: psa-addon-object-path-bucket-4
I1229 10:35:05.045747       1 controllerserver.go:208] Check if the provided bucket already exists: psa-addon-object-path-bucket-4
I1229 10:35:05.852634       1 controllerserver.go:268] create volume: pvc-ce088145-7ada-4e65-b895-3cd5c6571eb8
I1229 10:35:05.852665       1 server.go:168] GRPC response: volume:{capacity_bytes:268435456  volume_id:"pvc-ce088145-7ada-4e65-b895-3cd5c6571eb8"  volume_context:{key:"bucketName"  value:"psa-addon-object-path-bucket-4"}  volume_context:{key:"client"  value:"awss3"}  volume_context:{key:"cosEndpoint"  value:"https://s3.au-syd.cloud-object-storage.appdomain.cloud"}  volume_context:{key:"csi.storage.k8s.io/pv/name"  value:"pvc-ce088145-7ada-4e65-b895-3cd5c6571eb8"}  volume_context:{key:"csi.storage.k8s.io/pvc/name"  value:"objectpath-rclone-gafinal-pvc1"}  volume_context:{key:"csi.storage.k8s.io/pvc/namespace"  value:"default"}  volume_context:{key:"locationConstraint"  value:"au-syd-standard"}  volume_context:{key:"mounter"  value:"rclone"}  volume_context:{key:"objectPath"  value:"/data"}  volume_context:{key:"userProvidedBucket"  value:"true"}}

Prachi030510 · 2025-12-30T07:10:10Z

Test results -
https://github.ibm.com/alchemy-containers/armada-storage/issues/8583

pkg/driver/controllerserver.go

Bhagyashreek8 · 2025-12-30T09:09:42Z

pkg/driver/controllerserver.go


+	if bytesVal, ok := secret.Data["objectPath"]; ok {
+		objectPath = string(bytesVal)
+		klog.Infof("parseCustomSecret: Found objectPath: %s", objectPath)


Since we are already logging objectPath in CreateVolume method, no need to log here again.

Removed logging of objectPath in parseCustomSecret

Removed logging of the found objectPath for custom secret parsing.

Bhagyashreek8

Just fix the log messages. rest looks good.

contsto · 2026-01-01T07:43:06Z

🔴 Coverage decreased from 84.80% to 84.78%

mssachan

lgtm

Bhagyashreek8

lgtm

Prachi Shivanand Anure added 2 commits December 26, 2025 18:01

Fix object path trimming for Bucket Object Path mount support

3addddd

Fix object path trimming for Bucket Object Path mount support

addff75

minor canges

059feef

mssachan requested changes Dec 29, 2025

View reviewed changes

pkg/mounter/mounter-rclone.go Outdated Show resolved Hide resolved

pkg/mounter/mounter-s3fs.go Outdated Show resolved Hide resolved

pkg/s3client/s3client_test.go Show resolved Hide resolved

pkg/mounter/mounter-rclone.go Outdated Show resolved Hide resolved

Prachi Shivanand Anure added 4 commits December 29, 2025 12:32

deleted: tests/e2e/test_results.txt

91af80c

minor canges

e3a9168

minor canges

3c0c643

minor canges

ddaca46

Bhagyashreek8 reviewed Dec 30, 2025

View reviewed changes

pkg/driver/controllerserver.go Show resolved Hide resolved

Bhagyashreek8 reviewed Dec 30, 2025

View reviewed changes

Remove logging of objectPath in parseCustomSecret

ecc32cb

Removed logging of the found objectPath for custom secret parsing.

Bhagyashreek8 requested changes Dec 30, 2025

View reviewed changes

Prachi Shivanand Anure added 10 commits December 30, 2025 17:47

minor canges

d956184

minor canges

c6f6629

minor canges

58d1084

minor canges

22a562a

minor canges

da91067

minor canges

e597fe4

minor canges

8e8da26

minor canges

4e9e219

minor canges

27e563d

minor canges

1585477

mssachan approved these changes Jan 1, 2026

View reviewed changes

Bhagyashreek8 approved these changes Jan 1, 2026

View reviewed changes

Bhagyashreek8 merged commit 91ac231 into main Jan 1, 2026
4 checks passed

Fix object path trimming in Bucket Object Path mount support #288

Fix object path trimming in Bucket Object Path mount support #288

Conversation

Prachi030510 commented Dec 26, 2025

Describe your changes

Issue ticket number and link

Checklist before requesting a review

Uh oh!

Prachi030510 commented Dec 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mssachan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Prachi030510 commented Dec 29, 2025

Uh oh!

Prachi030510 commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Bhagyashreek8 Dec 30, 2025

Choose a reason for hiding this comment

Uh oh!

Prachi030510 Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Bhagyashreek8 left a comment

Choose a reason for hiding this comment

Uh oh!

contsto commented Jan 1, 2026

Uh oh!

mssachan left a comment

Choose a reason for hiding this comment

Uh oh!

Bhagyashreek8 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Prachi030510 commented Dec 26, 2025 •

edited

Loading

Prachi030510 commented Dec 30, 2025 •

edited

Loading

Prachi030510 Dec 30, 2025 •

edited

Loading