Can't find a way to store files inside a directory, instead of the root of the S3 bucket

[NogueraMateo]

Hello, I am working with this library and I have a profile_picture column with an "ImageField" type. I retrieved the S3 bucket and add it to the StoreManager.

The loading of the profile_pictures works fine, but it is storing the images in the root of the bucket, and I want the images to be stored in a directory like "users_pics" or something like that but I can't find the way.

I could use some help. Thanks!

[boaza]

There is a pending PR which addresses this issue.
Until it is merged, I fixed it locally for my own project.

Here is the steps steps I took:

• clone a fresh copy of the fepo (as of writing main was at commit d8b59ad)
• modified get_file() and delete_file() in sqlalchemy_file/storage.py to correctly split upload_storage and file_id. Note that in my case I only needed the 'complex file_id (e.g folder/file) and simple storage name' case discussed in the PR:
• created a specialized File class to include the path (similar to this solution)

Code:

# sqlalchemy_file/storage.py
...
@classmethod
    def get_file(cls, path: str) -> StoredFile:
        """Retrieve the file with `provided` path,
        path is expected to be `storage_name/file_id`.
        """
        upload_storage, file_id = path.split("/", 1) # <-- changed.  assumes <simple_storage_name>/<complex_file_id>
        return StoredFile(StorageManager.get(upload_storage).get_object(file_id))

@classmethod
    def delete_file(cls, path: str) -> bool:
        """Delete the file with `provided` path.

        The path is expected to be `storage_name/file_id`.
        """
        upload_storage, file_id = path.split("/", 1) # <-- changed.  assumes <simple_storage_name>/<complex_file_id>
        obj = StorageManager.get(upload_storage).get_object(file_id)
        if obj.driver.name == LOCAL_STORAGE_DRIVER_NAME:
            """Try deleting associated metadata file"""
            with contextlib.suppress(ObjectDoesNotExistError):
                obj.container.get_object(f"{obj.name}.metadata.json").delete()

        return obj.delete()
...

# my_project/file_ex.py
import uuid
from typing import Any, Optional, Dict
from sqlalchemy_file.stored_file import StoredFile
from sqlalchemy_file import File


class FileEx(File):
    """
    An extension to sqlalchemy_file.File, which allows adding a prefix to the generated stored file name.
    The prefix can also be a remote path, but this requires fixes to the sqlalchemy_file library itself,
    which were not part of the current release (0.6.0).

    To handle remote paths correctly, StorageManager.get_file() and StorageManager.delete_file() should be
    fixed to correctly parse 'arbitrary/path/to/file'.
    """
    def __init__(self,
                 content: Any = None,
                 filename: Optional[str] = None,
                 content_type: Optional[str] = None,
                 content_path: Optional[str] = None,
                 prefix: str = '',
                 **kwargs: Dict[str, Any]
                 ):
        super().__init__(content, filename, content_type, content_path, prefix=prefix, **kwargs)

    def store_content(
            self,
            content: Any,
            upload_storage: Optional[str] = None,
            name: Optional[str] = None,
            metadata: Optional[Dict[str, Any]] = None,
            extra: Optional[Dict[str, Any]] = None,
            headers: Optional[Dict[str, str]] = None,
            content_path: Optional[str] = None,
    ) -> StoredFile:
        """Store content into provided `upload_storage`
        with additional `metadata`. Can be used by processors
        to store additional files.
        """
        name = name or f'{self.prefix}{str(uuid.uuid4())}'
        return super().store_content(
            content,
            upload_storage,
            name,
            metadata,
            extra,
            headers,
            content_path,
        )

# my_project/my_model.py
class SomeFile(SQLModel, table=True):
    id: Optional[PrimaryKeyType] = Field(None, primary_key=True)
    file: Optional[File | UploadFile] = Field(sa_column=Column(FileField(upload_type=FileEx)))

# main.py
some_file = SomeFile(
    file=FileEx(content=b'hello files', filename='foo.txt', prefix='remote/path/to/files')
)
with get_session() as session:
    session.add(some_file )
    session.commit()