May 25, 2025: This Week in OpenDAL

[github-actions[bot]]

This weekly update is generated by LLMs. You're welcome to join our Discord for in-depth discussions.

Overall Activity Summary

The past week has seen robust activity in the Apache OpenDAL repository, with significant contributions across features, bug fixes, and refactoring efforts. A total of 21 pull requests were created or updated, 20 issues were actively discussed or resolved, and 3 discussions engaged the community. Key themes include enhancements to bindings, support for new APIs, and improvements to service integrations.

Key Ongoing Projects

• Glob Support (RFC-6209: Glob Support #6209): @asukaminato0721 contributed an RFC and implementation for matching file paths against Unix shell-style patterns, enabling more flexible file operations.
• Azure Storage Connection Strings (feat(services/azdls): Support parsing Azure Storage configs from connection strings #6212): @DerGut implemented parsing functionality for Azure Storage connection strings, simplifying configuration for Azure Data Lake Storage users.
• Options API (feat(core): Expose xxx_options API #6215): @Xuanwo introduced new APIs for read_options and write_options, improving usability for developers. This aligns with RFC-6213.
• Enhancements to Python Bindings (feat(bindings/python): Enhance Reader and Writer #6086): @chitralverma expanded Python bindings to support advanced reader and writer options, enhancing async operations.
• Refactor for Blocking API Removal (refactor!: Remove native blocking api in opendal #6216): @Xuanwo completed a major refactor to remove native blocking APIs, transitioning to async wrappers as per RFC-6189.

Priority Items

• feat: initial Operator::from_uri implementation #5482: @jorgehermo9 requested feedback on the implementation of Operator::from_uri. This PR has unresolved TODOs and requires guidance on test placement and documentation updates.
• feat(bindings/python): Enhance Reader and Writer #6086: Python bindings enhancement by @chitralverma needs review, especially for the new async features.
• feat(services/azdls): list start from #5242: @alexwilcoxson-rel proposed improvements for Azure Data Lake Storage, including start_after functionality for efficient listing. This PR requires further validation and testing.
• feat(services/azdls): Support parsing Azure Storage configs from connection strings #6212: Parsing Azure Storage connection strings by @DerGut needs final review and testing for integration with Iceberg Rust.

Notable Discussions

• [VOTE] Release Apache OpenDAL 0.53.3 - Vote Round 2 #6208: [VOTE] Release Apache OpenDAL 0.53.3 - Vote Round 2 initiated by @Xuanwo. Community members are encouraged to test and verify the release candidate.
• How to use opendal to implement breakpoint resume upload and Incremental upload ? #5875: @videni discussed implementing breakpoint resume and incremental uploads using OpenDAL APIs, sparking ideas for advanced use cases.
• Tracking issues for RFC-6209: Glob Support #6210: Tracking issues for RFC-6209 (Glob Support) led by @asukaminato0721, focusing on extending glob functionality across bindings.

Emerging Trends

• Binding Enhancements: Significant focus on improving language bindings (Python, C, Go), making OpenDAL accessible to diverse developer communities.
• Azure Integration: Increased contributions around Azure services, including connection strings and authentication mechanisms.
• Refactoring for Usability: Efforts to streamline APIs (e.g., Options API, Blocking API removal) indicate a trend toward simplifying developer experience.

Good First Issues

• new feature: publish the source and lock files for binaries (oli, oay...) #6198: Publish source and lock files for binaries (e.g., oli, oay) to support build-from-source workflows like NixOS packaging.
• Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702: Refactor services to adopt the backend.rs and core.rs convention, improving code maintainability.
• Implement Write Returns Metadata for all services #5693: Implement "Write Returns Metadata" for services like FTP, HDFS, and WebHDFS.
• Tracking issues of RFC-5479: Context #5480: Tasks related to RFC-5479 (Context API), including extending context support across services.

Thank you to all contributors for your efforts this week! Let's continue building a strong and inclusive open-source community.