v0.21.0

[Xuanwo]

Upgrade to v0.21

v0.21 is an internal refactor version of OpenDAL. In this version, we refactored our error handling and our Accessor APIs. Thanks to those internal changes, we added an object-level metadata cache, making it nearly zero cost to reuse existing metadata continuously.

Let's start with our errors.

Error Handling

As described in RFC-0977: Refactor Error, we refactor opendal error by a new error
called opendal::Error.

This change will affect all APIs that are used to return io::Error.

To migrate this, please replace std::io::Error with opendal::Error:

- use std::io::Result;
+ use opendal::Result;

And the following error kinds should be updated:

• std::io::ErrorKind::NotFound => opendal::ErrorKind::ObjectNotFound
• std::io::ErrorKind::PermissionDenied => opendal::ErrorKind::ObjectPermissionDenied

And since v0.21, we will return errors ObjectIsADirectory and ObjectNotADirectory instead of anyhow::Error.

Accessor API

In v0.21, we refactor the whole Accessor's API:

- async fn write(&self, path: &str, args: OpWrite, r: BytesReader) -> Result<u64>
+ async fn write(&self, path: &str, args: OpWrite, r: BytesReader) -> Result<RpWrite>

Since v0.21, we will return a reply struct for different operations called RpWrite instead of an exact type. We can split OpenDAL's public API and raw API with this change.

ObjectList and ObjectPage

Since v0.21, Accessor will return ObjectPager for List:

- async fn list(&self, path: &str, args: OpList) -> Result<ObjectStreamer>
+ async fn list(&self, path: &str, args: OpList) -> Result<(RpList, ObjectPager)>

And Object will return an ObjectLister which is built upon ObjectPage:

pub async fn list(&self) -> Result<ObjectLister> { ... }

ObjectLister can be used as an object stream as before. It also provides the function next_page to get the underlying pages directly:

impl ObjectLister {
    pub async fn next_page(&mut self) -> Result<Option<Vec<Object>>>;
}

Code Layout

Since v0.21, we have categorized all APIs into public and raw.

Public APIs are exposed under opendal::Xxx; they are user-face APIs that are easy to use and understand.

Raw APIs are exposed under opendal::raw::Xxx; they are implementation details for underlying services and layers.

Please replace all usage of opendal::io_util::* and opendal::http_util::* to opendal::raw::* instead.

With this change, new users of OpenDAL maybe be it easier to get started.

Summary

Sorry for introducing too much breaking change in a single version. This version can be a solid version for preparing OpenDAL v1.0.

What's Changed

• docs: Add greptimedb and mars into projects by @Xuanwo in docs: Add greptimedb and mars into projects #975
• RFC-0977: Refactor Error by @Xuanwo in RFC-0977: Refactor Error #977
• refactor: Use seperate Error instead of std::io::Error to avoid confusing by @Xuanwo in refactor: Use seperate Error instead of std::io::Error to avoid confusing #976
• fix: RetryAccessor is too verbose by @Xuanwo in fix: RetryAccessor is too verbose #980
• refactor: Return ReplyCreate for create operation by @Xuanwo in refactor: Return ReplyCreate for create operation #981
• refactor: Add ReplyRead for read operation by @Xuanwo in refactor: Add ReplyRead for read operation #982
• refactor: Add RpWrite for write operation by @Xuanwo in refactor: Add RpWrite for write operation #983
• refactor: Add RpStat for stat operation by @Xuanwo in refactor: Add RpStat for stat operation #984
• refactor: Add RpDelete for delete operations by @Xuanwo in refactor: Add RpDelete for delete operations #985
• refactor: Add RpPresign for presign operation by @Xuanwo in refactor: Add RpPresign for presign operation #986
• refactor: Add reply for all multipart operations by @Xuanwo in refactor: Add reply for all multipart operations #988
• refactor: Add Reply for all blocking operations by @Xuanwo in refactor: Add Reply for all blocking operations #989
• feat: impl atomic write for fs service by @killme2008 in feat: impl atomic write for fs service #991
• refactor: Avoid accessor in object entry by @Xuanwo in refactor: Avoid accessor in object entry #992
• refactor: Move accessor into raw apis by @Xuanwo in refactor: Move accessor into raw apis #994
• refactor: Move io to raw by @Xuanwo in refactor: Move io to raw #996
• feat: Add OperatorMetadata to avoid expose AccessorMetadata by @Xuanwo in feat: Add OperatorMetadata to avoid expose AccessorMetadata #997
• refactor: Move {path,wrapper,http_util,io_util} into raw modules by @Xuanwo in refactor: Move {path,wrapper,http_util,io_util} into raw modules #998
• refactor: Move ObjectEntry and ObjectPage into raw by @Xuanwo in refactor: Move ObjectEntry and ObjectPage into raw #999
• refactor: Accept Operator intead of Arc by @Xuanwo in refactor: Accept Operator intead of Arc<dyn Accessor> #1001
• feat: Improve display for error by @Xuanwo in feat: Improve display for error #1002
• Bump to version 0.21 by @Xuanwo in Bump to version 0.21 #1003

Full Changelog: v0.20.1...v0.21.0

---

This discussion was created from the release v0.21.0.