April 27, 2025: This Week in OpenDAL

[github-actions[bot]]

This weekly update is generated by LLMs. You're welcome to join our Discord for in-depth discussions.

---

Overall Activity Summary

The apache/opendal repository has seen significant activity over the past week, with a focus on feature enhancements, refactoring, and expanding support for bindings across multiple languages. A total of 38 pull requests and 26 issues/discussions were actively worked on. Key themes include improving service modularity, enhancing bindings, and addressing bugs related to platform compatibility.

---

Key Ongoing Projects

• Refactoring Services to Core/Backend Convention
  Progress continues on aligning services with the core.rs and backend.rs structure for better maintainability. Notable contributions include:

  • feat(services/hdfs): refactor HdfsBackend to use HdfsCore for client operations #6033 by @uruemu refactors HDFS services.
  • refactor: Introduce IpfsCore for improved service structure #5950 by @miroim restructures IPFS services.

• Bindings Expansion
  Multiple efforts to improve bindings across languages:

  • feat(bindings/java): Add new read API to support passing options  #6081 by @sunheyi6 introduces a new read API for Java bindings.
  • feat(bindings/c): async stat, read, write ,delete #6072 by @asukaminato0721 adds async stat support for C bindings.
  • Discussions like How to design opendal c binding's async api #6082 explore async API design for C bindings.

• Caching Layer Implementation
  feat: Initial foyer layer implementation #5906 by @jorgehermo9 is making progress on a caching layer to improve performance and reduce redundant operations.

• Write Returns Metadata
  Enhancements like feat(services/gcs): implement write returns metadata #5933 by @meteorgan for GCS and feat(services/azblob): implement write returns metadata #5958 by @meteorgan for Azure Blob Storage aim to extract metadata during write operations.

• High-Performance Documentation
  Contributions such as docs(core): Add performance guide for concurrent write #6011 by @Xuanwo focus on documenting best practices for concurrent writes and HTTP optimizations.

---

Priority Items

• Open PRs Needing Immediate Attention:

  • feat: Initial foyer layer implementation #5906 by @jorgehermo9: Initial caching layer implementation. Feedback on scope and use cases is needed.
  • feat(services/b2): Add operation in http context #6066 by @jorgehermo9: Adds HTTP context operations for Backblaze B2. Requires review for alignment with Add http context in for all http based services #5774.
  • feat(bindings/java): Add new read API to support passing options  #6081 by @sunheyi6: Adds a new read API for Java bindings. Needs final testing and review.

• Critical Issues:

  • Need range reader or seek in c binding #6083 by @taoseng: Requests range reader/seek support for C bindings. Needs design input.
  • Nodejs Binding failed to release #5957: Node.js binding release failure. Requires investigation into GitHub integration permissions.

---

Notable Discussions

• How to design opendal c binding's async api #6082: How to Design Async API for C Bindings
  @asukaminato0721 initiated a discussion on designing async APIs for C bindings, exploring options like callbacks, coroutines, and state machines.

• Do we have better names for opendal python's Layer? #6074: Naming Conventions for Python Layers
  @Xuanwo raised concerns about the term "Layer" in Python bindings, suggesting alternatives like "Middleware" for better alignment with Python conventions.

• Native Pagination Support for File Listing in OpenDAL #6040: Pagination Support for File Listing
  @cbisht31 proposed adding native pagination support for efficient iteration over large datasets, sparking discussions on implementation strategies.

---

Emerging Trends

• Service Modularity: The repository is increasingly adopting a standardized structure with core.rs and backend.rs modules, improving code readability and maintainability.
• Focus on Bindings: Significant efforts are being made to enhance bindings for Java, Python, and C, reflecting the project’s commitment to multi-language support.
• Performance Optimization: Contributions like docs(core): Add performance guide for concurrent write #6011 and feat(core): Use Buffer as http_body::Body directly #6026 aim to improve performance through concurrent operations and reduced HTTP overhead.

---

Good First Issues

• Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702: Refactor Services to Use backend.rs and core.rs Convention
  Tasks involve restructuring service implementations, providing a clear and manageable scope for new contributors.

• Implement Write Returns Metadata for all services #5693: Implement Write Returns Metadata for All Services
  Contributors can focus on extracting metadata for specific services, offering a well-defined task with impactful results.

• Need range reader or seek in c binding #6083: Need Range Reader or Seek in C Binding
  A straightforward enhancement for contributors familiar with C bindings.

• new feature: Add start-after support in python binding #6042: Add Start-After Support in Python Binding
  Introduces a simple feature to Python bindings, making it a great entry point for contributors familiar with Python.

• bug: Failing test on 32bit platform #6077: Failing Test on 32-bit Platforms
  A small but impactful task to fix platform-specific test failures, ideal for new contributors.

---

We encourage new contributors to explore these "Good First Issues" and join the community discussions to get started!