Split opendal mono repo in multiple repos

[Xuanwo]

For a long time, OpenDAL has been a mono repo containing the core, bindings, binaries, and integrations. Now that OpenDAL has grown quite large, and many recent changes are just to keep our CI happy, it may be time to split OpenDAL into multiple repositories. This will let the community focus on the parts they find most interesting.

What do you think?

[PsiACE]

I haven't been back to the code for a while. Yesterday, I tried to continue some work on the repo, and it was pretty cool to see hundreds of CI jobs running for me at the same time. However, I also had to spend effort trying to satisfy every part, like wasm and go-bindings, which are parts that I don't really care about personally.

I completely agree that we should try to reduce the maintenance burden.

We could start by splitting off projects that are not currently on the core roadmap but have ambitious goals of their own. These projects might be better suited for independent development, such as oay and ofs.

By doing this, they can become part of the opendal umbrella structure, seeking greater room for growth, much like Arrow, DataFusion and its companion projects (such as ballista or comet). Together, they form a powerful ecosystem while maintaining their individual independence and focus. This would not only alleviate the maintenance pressure on the core project but also allow these independent projects to evolve more freely.

[PsiACE]

As a first experimental step, we could consider splitting off ofs and its related integrations, including fuse3 and cloud-filter.

This project seems to be an ideal candidate because it shares similar concepts with established projects like JuiceFS —that is, providing a file system interface on top of object storage or other storage services. Spinning ofs into its own repository would allow it to have its own roadmap and community, focusing on file system-level features and optimizations without being tied to the CI and release cycles of the core opendal library.

This would not only serve as a great trial for our repo-splitting idea but also create a more focused and high-potential development space for ofs itself.

[Xuanwo]

Looks good. Let’s start by splitting ofs.

[yuchanns]

+1 to splitting off the binaries. On the bindings side, I know there’s a worry that maintainers have to “satisfy” parts they don’t care about, but I actually see the opposite. If something changes outside their area, we can ping the relevant folks to handle it. And the small core tweaks that do break bindings are exactly the things binding maintainers want to hear about immediately. Keeping bindings close to the core is what keeps everyone in the loop and the integrations healthy.