Skip to content

Commit 8a414b2

Browse files
committed
Improve performance of LocalStorage._list_blobs
Current implementation globs over the entire root directory, but needs files only with the given prefix. Globbing over the entire root directory is very slow in case of a large number of files. 'Path.resolve()' is used in an attempt to protect against directory traversal.
1 parent 78e2cf7 commit 8a414b2

File tree

1 file changed

+3
-5
lines changed

1 file changed

+3
-5
lines changed

medusa/storage/local_storage.py

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -51,12 +51,10 @@ def disconnect(self):
5151
async def _list_blobs(self, prefix=None):
5252
if prefix is None:
5353
paths = list(self.root_dir.glob('**/*'))
54+
elif str(prefix).endswith('/'):
55+
paths = list(self.root_dir.joinpath(prefix).resolve().glob('**/*'))
5456
else:
55-
paths = [
56-
p for p in self.root_dir.glob('**/*')
57-
# relative_to() cuts off the base_path and bucket, so it works just like cloud storages
58-
if str(p.relative_to(self.root_dir)).startswith(str(prefix))
59-
]
57+
paths = [p.resolve() for p in self.root_dir.glob(str(prefix) + '*/**/*')]
6058

6159
return [
6260
AbstractBlob(

0 commit comments

Comments
 (0)