Skip to content

[Feature Request] fsspec/universal_pathlib integration #60

@Mithrandir2k18

Description

@Mithrandir2k18

As far as I understand it, this package cannot be used with universal pathlib right away, but all the building block are already there. All that's missing is registering the AbstractFileSystem subclasses with fsspec and creating fsspec/universal_pathlib subclasses and registering them with their registry.

This seems simple enough and seems to work in my testing:

from fsspec import AbstractFileSystem
from upath import UPath
from upath.registry import register_implementation

from morefs.memory import MemFS

# local instance for inspectability during testing
mem_fs = MemFS()


class MorefsMemPath(UPath):
    @classmethod
    def _fs_factory(
        cls, urlpath: str, protocol: str, storage_options: dict
    ) -> AbstractFileSystem:
        return mem_fs  # this should be MemFS(...)


fsname = "customfs"
register_implementation(fsname, MorefsMemPath)


def test_upath():
    testtext = "Hello from MemFS!"
    testfile = "/example.txt"
    p = UPath(f"{fsname}:///{testfile}")
    p.write_text(testtext)
    assert testtext == p.read_text()
    assert mem_fs.ls("/") == [testfile]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions