This week in OpenDAL Demo

[github-actions[bot]]

Overall Activity Summary

The past week has seen significant activity in the apache/opendal repository, with numerous pull requests addressing bug fixes, feature enhancements, refactoring, and documentation updates. Key themes include migrating services to context-based HTTP clients, improving metrics collection, and enhancing support for specific services like OneDrive and OPFS. Discussions have focused on improving usability, production readiness, and expanding contributor engagement.

Key Ongoing Projects

• Context-Based HTTP Client Migration:

  • Several services are being migrated to use context-based HTTP clients, enabling better integration and observability. Notable contributions include feat(services/huggingface): Add operation in http context #5810 by @liugddx for Huggingface and feat(services/lakefs): Add operation in http context #5809 by @liugddx for LakeFS.
  • This initiative is tracked in Add http context in for all http based services #5774.

• Object Versioning:

  • Enhancements for object versioning are ongoing, enabling services like S3 and COS to support version-specific operations. Recent contributions include feat(core): Implement list with deleted and versions for cos #5514 by @Ziy1-Tan for COS.

• OPFS Support:

  • The Origin Private File System (OPFS) service is being developed to support WASM targets. Contributions include feat: Add origin private file system scaffold #5758 by @Eason0729 and OPFS support for OpenDAL (draft PR) #5269 by @LYZJU2019.

• Metrics Enhancements:

  • HTTP-related metrics are being added to observe::MetricsLayer for better monitoring. Contributions include feat(core): Implement http related metrics support for otel #5800 by @leiysky and feat(core): Implement http related metrics support for prom client #5798 by @Xuanwo.

• OneDrive Improvements:

  • Multiple features for OneDrive, including chunked uploads (fix(services/onedrive): chunk PUT upload remove auth header #5812 by @emliunix) and refresh token handling (feat(services/onedrive): add signer to utilize the refresh token #5733 by @erickguan), are being implemented.

Priority Items

• Bug Fixes:

  • bug: onedrive chunked upload fail with unauthenticated error #5811 by @emliunix: Fix unauthenticated error during OneDrive chunked uploads.
  • bug: compfs failed to pass behavior tests #5801 by @Xuanwo: Address failures in behavior tests for compfs.
  • fix(services/onedrive): add scope for the refresh token #5776 by @erickguan: Add scope for OneDrive refresh token to resolve token retrieval issues.

• Refactoring:

  • refactor(core/raw)!: Use AccessorInfo instead of seperate fields #5796 by @Xuanwo: Refactor core/raw to use AccessorInfo for improved code clarity and efficiency.
  • refactor(core/raw)!: Merge blocking and async operations #5789 by @Xuanwo: Merge blocking and async operations to simplify user understanding.

• Documentation Updates:

  • docs(services/onedrive): add how to get tokens #5792 by @emliunix: Add steps for obtaining OneDrive tokens.
  • doc: fix README root wrong for example close issue #5871 by correct README #5782 by @yihong0618: Fix README example for correct execution.

Notable Discussions

• Call for OSPP 2025 Projects:

  • @PsiACE initiated a discussion on potential projects for Open Source Promotion Plan 2025, encouraging community members to propose ideas.

• Remove adapter::kv and adapter::typed_kv:

  • @Xuanwo proposed migrating services to implement Access directly, removing unnecessary abstraction layers.

• OpenDAL 2025 Roadmap:

  • @Xuanwo shared the roadmap, focusing on production readiness and feature completeness. Contributors are encouraged to participate in related discussions.

Emerging Trends

• Context-Based Development:

  • A strong push toward using context-based APIs for services and layers is evident, improving modularity and observability.

• Focus on Production Readiness:

  • Enhancements like object versioning, metrics, and caching are aimed at making OpenDAL more robust for production use.

• Contributor Engagement:

  • Efforts to simplify onboarding for new contributors, such as identifying good first issues and improving documentation, are visible.

Good First Issues

• Add http context for all HTTP-based services: Introduces context for HTTP calls in services, enabling better observability. Requires basic Rust knowledge and familiarity with HTTP concepts.
• Add HTTP-related metrics in observe::MetricsLayer: Adds metrics for HTTP requests and responses. Suitable for contributors interested in monitoring and observability.
• Tracking issues of user metadata support: Implements user-defined metadata for services. Ideal for contributors with Rust experience and interest in storage services.
• Tracking issues of OpenDAL wasm32-unknown-unknown target support: Expands WASM support for services. A good entry point for contributors interested in WebAssembly.

Contributors

Thank you to all contributors for their efforts this week:

• @Xuanwo
• @liugddx
• @emliunix
• @meteorgan
• @miroim
• @Radovenchyk
• @cuichenli
• @yihong0618
• @Berrysoft
• @Ziy1-Tan
• @Eason0729
• @LYZJU2019
• @leiysky
• @erickguan
• @hoslo
• @tisonkun
• @asukaminato0721
• @Frank-III
• @shaonianche
• @zhaohaidao
• @PsiACE
• @messense
• @stormshield-guillaumed
• @QazCetelic
• @George-Miao
• @PsiACE

Your contributions are invaluable to the growth and success of OpenDAL!

[erickguan]

Cool summary!

[Xuanwo]

Let's see what will happen on this Sunday!