Skip to content

feat(services/opfs): OPFS backend for OpenDAL#7199

Open
jccampagne wants to merge 49 commits intoapache:mainfrom
jccampagne:wip_opfs_3
Open

feat(services/opfs): OPFS backend for OpenDAL#7199
jccampagne wants to merge 49 commits intoapache:mainfrom
jccampagne:wip_opfs_3

Conversation

@jccampagne
Copy link
Copy Markdown

@jccampagne jccampagne commented Feb 22, 2026

Which issue does this PR close?

This is for OPFS backend: see #5799

Added write, read, list, delete, stat; tests.

I will extend this PR for the missing features (read, stat, delete), if that's ok.

This is still a draft, so there are debug logs, and comments in the code to help with devs; most of them will be removed later.

Rationale for this change

I needed OPFS for my Rust project targetting WASM32 (in the browser).
I discovered OpenDAL but OPFS support was limited, so I decided to implement it.
I managed to make it work for my project (using OpenDAL as a dependency), I had implemented (read, write, stat, delete...). But the code was more like a prototype quality as I was learning OpenDA architecture and API.

This is my 3rd iteration. Still early, and incomplete, but I would like some early feedback if possible to make sure I am on the right track to follow the spirit of OpenDAL. Do not hesitate to comment.

As I understand it, OPFS as 2 kinds of APIs:

  • synchronous interface: using FileSystemSyncAccessHandle to read and write, can only be used in workers;

  • asynchronous interface: GetFile +FileSystemWritableFileStream, can be using in main JS thread and works, but less efficient.

I decided to go for the async version as it suits my use case (for now), and it it the most generic use case and requires less setup to use (no workers).

I saw the TwoWays trait - that could be used later to implement the alternative using the synch version of OPFS.

Using SendWrapper to make some struct Send and Sync.

What changes are included in this PR?

Added write, read, list, delete, stat.
Added edge tests in core/edge/opfs_wasm32.
I ran the tests on 3 different engine:

  • Gecko (firefox)
  • Chromium (Chrome)
  • Webkit (Safari)

Are there any user-facing changes?

Addition of write to the OPFS operator.

AI Usage Statement

Yes, Gemini and Claude to navigate the codebase, documentation; and help with some code.

@jccampagne jccampagne marked this pull request as ready for review February 23, 2026 01:56
@dosubot
Copy link
Copy Markdown

dosubot bot commented Feb 23, 2026

Related Documentation

Checked 0 published document(s) in 1 knowledge base(s). No updates required.

How did I do? Any feedback?  Join Discord

@dosubot dosubot bot added the releases-note/feat The PR implements a new feature or has a title that begins with "feat" label Feb 23, 2026
@jccampagne jccampagne changed the title Wip OPFS backend for OpenDAL OPFS backend for OpenDAL Feb 23, 2026
@jccampagne
Copy link
Copy Markdown
Author

jccampagne commented Feb 23, 2026

Not sure what is happening in failing test...
https://github.com/apache/opendal/actions/runs/22291346986/job/64479069421?pr=7199

It's gone now - just flaky I suppose.

@jccampagne jccampagne changed the title OPFS backend for OpenDAL feat(services/opfs): OPFS backend for OpenDAL Feb 23, 2026
@jccampagne
Copy link
Copy Markdown
Author

It turns out I will probably need the Synch version as well (using FileSystemSyncAccessHandle) for my project.
In terms of architecture, this should probably be a different service (something like opfs_sync)?
What do you suggest @Xuanwo ?

@Eason0729
Copy link
Copy Markdown
Contributor

I looked through your changes.

The large size of this PR makes it difficult for the maintainer to review. I suggest adding the deleter, lister, and reader in separate PRs.

Also, how about using a buffered writer for OPFS? That way, writes will be faster.

The term may be confusing, here is a similar pattern:

let buf = self.buffer.clone().collect();
let length = buf.len() as u64;
self.core.set(&self.path, buf).await?;
let meta = Metadata::new(EntryMode::from_path(&self.path)).with_content_length(length);
Ok(meta)

*do actual writes on close

@jccampagne
Copy link
Copy Markdown
Author

jccampagne commented Mar 29, 2026

I looked through your changes.

Thanks.

The large size of this PR makes it difficult for the maintainer to review.

Yes... it started small... and then I added more and more...

I suggest adding the deleter, lister, and reader in separate PRs.

How about:

  1. Write + Read + tests
  2. List + tests
  3. Delete + tests

as suggested initially: in #5799 ?

It's difficult to validate Write without Read (being able to read allows the check what was written).

Also, how about using a buffered writer for OPFS? That way, writes will be faster.

The term may be confusing, here is a similar pattern:

let buf = self.buffer.clone().collect();
let length = buf.len() as u64;
self.core.set(&self.path, buf).await?;
let meta = Metadata::new(EntryMode::from_path(&self.path)).with_content_length(length);
Ok(meta)

*do actual writes on close

I will have a look at "buffered writer" approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants