You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The cleanup_old_files maintenance function currently deletes files one at a time. For cloud storage this results in unnecessary network requests, increasing runtime and cloud service provider costs. If a bulk deletion interface is available on the target file system, it would make sense to use this instead. For example, the S3 API can delete 1000 objects in a single request. This is particularly relevant for applications with high write volumes and frequent compaction, where thousands of small files may be marked for deletion during a compaction step.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
The
cleanup_old_files
maintenance function currently deletes files one at a time. For cloud storage this results in unnecessary network requests, increasing runtime and cloud service provider costs. If a bulk deletion interface is available on the target file system, it would make sense to use this instead. For example, the S3 API can delete 1000 objects in a single request. This is particularly relevant for applications with high write volumes and frequent compaction, where thousands of small files may be marked for deletion during a compaction step.Beta Was this translation helpful? Give feedback.
All reactions