feat(services/opfs): OPFS backend for OpenDAL#7199
feat(services/opfs): OPFS backend for OpenDAL#7199jccampagne wants to merge 49 commits intoapache:mainfrom
Conversation
65dc6f2 to
225ae90
Compare
|
|
|
It turns out I will probably need the Synch version as well (using FileSystemSyncAccessHandle) for my project. |
|
I looked through your changes. The large size of this PR makes it difficult for the maintainer to review. I suggest adding the deleter, lister, and reader in separate PRs. Also, how about using a buffered writer for OPFS? That way, writes will be faster.
|
Thanks.
Yes... it started small... and then I added more and more...
How about:
as suggested initially: in #5799 ? It's difficult to validate Write without Read (being able to read allows the check what was written).
I will have a look at "buffered writer" approach. |
Which issue does this PR close?
This is for OPFS backend: see #5799
Added write, read, list, delete, stat; tests.
I will extend this PR for the missing features (read, stat, delete), if that's ok.
This is still a draft, so there are debug logs, and comments in the code to help with devs; most of them will be removed later.
Rationale for this change
I needed OPFS for my Rust project targetting WASM32 (in the browser).
I discovered OpenDAL but OPFS support was limited, so I decided to implement it.
I managed to make it work for my project (using OpenDAL as a dependency), I had implemented (read, write, stat, delete...). But the code was more like a prototype quality as I was learning OpenDA architecture and API.
This is my 3rd iteration. Still early, and incomplete, but I would like some early feedback if possible to make sure I am on the right track to follow the spirit of OpenDAL. Do not hesitate to comment.
As I understand it, OPFS as 2 kinds of APIs:
synchronous interface: using
FileSystemSyncAccessHandleto read and write, can only be used in workers;asynchronous interface:
GetFile+FileSystemWritableFileStream, can be using in main JS thread and works, but less efficient.I decided to go for the async version as it suits my use case (for now), and it it the most generic use case and requires less setup to use (no workers).
I saw the
TwoWaystrait - that could be used later to implement the alternative using the synch version of OPFS.Using SendWrapper to make some struct Send and Sync.
What changes are included in this PR?
Added write, read, list, delete, stat.
Added edge tests in
core/edge/opfs_wasm32.I ran the tests on 3 different engine:
Are there any user-facing changes?
Addition of write to the OPFS operator.
AI Usage Statement
Yes, Gemini and Claude to navigate the codebase, documentation; and help with some code.