-
Notifications
You must be signed in to change notification settings - Fork 164
rm override for HNS buckets #742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
472cff6 to
07b7afd
Compare
458df56 to
a61f39f
Compare
0cd4aa8 to
c59eeb6
Compare
|
/gcbrun |
Cache should be updated when find is called with a path containing trailing slash
|
/gcbrun |
| and "No such object" not in str(ex) | ||
| ] | ||
| if errors: | ||
| raise errors[0] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After collecting all errors, why are we only returning first error ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I kept the behavior same as rm() implementation in the GCSFileSystem to be consistent. For regional buckets it is just returning the first error, so I kept it same to make sure the behavior is same for customers.
|
/gcbrun |
This PR introduces an override for the rm method in ExtendedGcsFileSystem to provide native support for deleting files and explicit folder objects within Hierarchical Namespace (HNS) enabled GCS buckets.
Recursive Directory Deletion: When recursive=True is specified for a directory, the implementation first deletes all file objects within the directory structure in batches. It then proceeds to delete the now-empty folder objects, starting from the deepest sub-folders and working its way up.
Non-Recursive Directory Deletion: If rm is called on a non-empty directory without recursive=True, it will raise an OSError, mimicking standard filesystem behavior and preventing accidental data loss
Cache Invalidation:
Testing:
Unit tests and integration tests are added for covering following scenarios: