April 13, 2025: This Week in OpenDAL

[github-actions[bot]]

This weekly update is generated by LLMs. You're welcome to join our Discord for in-depth discussions.

Overall Activity Summary

The apache/opendal repository has seen vibrant activity over the past week, with contributions spanning feature development, bug fixes, documentation updates, and refactoring efforts. A total of 38 pull requests were created or updated, with several focusing on enhancing service implementations, improving documentation, and addressing bugs. Additionally, discussions around project roadmaps and technical challenges have been active, fostering community engagement.

Key Ongoing Projects

• Refactoring Services to Core/Backend Convention:

  • The refactor effort tracked in Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702 continues to progress, with contributions like refactor(ftp): restructure FtpBackend and introduce FtpCore #5949 by @uruemu restructuring the FTP service and refactor: Introduce IpmfsCore for improved service structure #5980 by @miroim refactoring IPMFS services.

• Write Returns Metadata:

  • The initiative to implement metadata extraction during write operations is advancing, with feat(services/azblob): implement write returns metadata #5958 by @meteorgan adding support for Azure Blob Storage.

• High-Performance OpenDAL Documentation:

  • Documentation updates for performance optimization are underway, with contributions like docs(core): Add performance guide for concurrent write #6011 by @Xuanwo adding a guide for concurrent writes.

• QueryPairsWriter Refactor:

  • The refactor tracked in Tracking issues of QueryPairsWriter refactor #5976 has been completed for several services, including S3 (chore(services/s3): Use QueryPairsWriter for url write #5977 by @Xuanwo) and Azure Blob (chore(service/azblob): use QueryPairsWriter for url write #5979 by @kemingy), improving code readability and performance.

Priority Items

• Open PRs Requiring Attention:

  • feat: Initial foyer layer implementation #5906 by @jorgehermo9: Initial implementation of a caching layer. Feedback is needed on the scope of caching operations.
  • feat:add tls support for memcached #5471 by @ozewr: TLS support for Memcached. Requires review and testing.
  • feat(core): implement delete with recursive #5762 by @meteorgan: Recursive deletion implementation. Needs further refinement and testing.

• Critical Issues:

  • sync check in opendal need work around #5995: Sync check workaround in bindings. Requires investigation into the root cause.
  • Bindings D CI has been disabled #6010: Disabled CI for D bindings due to external dependency issues. Needs resolution.

Notable Discussions

• OSPP 2025 Projects (Call for OSPP 2025 Projects #5793):

  • A call for project ideas for the Open Source Promotion Plan 2025 is underway. Community members are encouraged to propose innovative projects.

• Reader Behavior (Reader may access content from the same object but a different version #5971):

  • Discussion initiated by @meteorgan on ensuring consistent version reads for objects without requiring explicit version IDs.

• Integration with DuckDB and Polars ((rust) How to use opendal to load data into DuckDB or Polars LazyFrame? #5972):

  • @chitralverma raised questions about best practices for loading data from OpenDAL into DuckDB or Polars LazyFrame.

Emerging Trends

• Refactoring for Maintainability:

  • The repository is moving toward a standardized structure with core.rs and backend.rs modules for services, improving code organization and readability.

• Focus on Performance:

  • Several contributions, such as docs(core): Add performance guide for concurrent write #6011 and Improve TracingLayer to provide better exprience #5999, aim to enhance performance through concurrent operations and improved tracing.

• Increased Documentation Efforts:

  • Multiple PRs and issues are focused on improving documentation, making the project more accessible to new contributors.

Good First Issues

• Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702: Refactor services to use the backend.rs and core.rs convention. This task is straightforward and provides a great introduction to OpenDAL's service structure.
• Implement Write Returns Metadata for all services #5693: Implement Write Returns Metadata for all services. Contributors can focus on extracting metadata for specific services.
• Tracking issues of RFC-5479: Context #5480: Context implementation tasks. These are smaller, well-defined tasks related to RFC-5479.

---

We encourage new contributors to explore the "Good First Issues" section and join the community discussions to get started!