v0.55.0

[Xuanwo]

Rust Core Upgrade to v0.55

Public API

Timestamp types now come from jiff

All public metadata APIs that previously exposed chrono::DateTime<Utc> now use jiff::Timestamp. For example, Metadata::last_modified() and related setters return/accept Timestamp values (core/src/types/metadata.rs). Update downstream crates to depend on jiff if they manipulate these timestamps or convert them to other formats.

Scheme handling is string-based

OperatorInfo::scheme() now returns &'static str instead of Scheme, and Operator::via_iter accepts impl AsRef<str> (typically the services::*_SCHEME constants). Additionally, the deprecated constructors Operator::from_map and Operator::via_map have been removed. Migrate any code that relied on the enum variants or the removed constructors to the new string-based constants and from_iter/via_iter.

List APIs only support versions

OpList::with_version()/version() and Capability::list_with_version have been removed after a long deprecation cycle. Use with_versions()/versions() on OpList and read Capability::list_with_versions instead.

S3Builder::security_token removed

S3Builder no longer exposes the deprecated security_token() helper. Use session_token() exclusively when configuring temporary credentials.

KV-style services no longer pretend to support list

Services that never returned meaningful results for Operator::list (such as D1, FoundationDB, GridFS, Memcached, MongoDB, MySQL, Persy, PostgreSQL, Redb, Redis, SurrealDB, TiKV, etc.) now rely on the default Unsupported implementation. Those features will be implemented later.

Raw API

Deprecated KV adapters removed

The legacy opendal::raw::adapters::{kv, typed_kv} modules have been deleted. Services should directly implement Access instead of depending on the adapters. Remove the corresponding imports and shim layers from any out-of-tree services.

Bindings Python Upgrade to v0.47

Breaking change: Module exports are explicit

opendal.__init__ now only re-exports the capability, exceptions, file, layers, services, types, Operator, and AsyncOperator symbols. Imports such as:

from opendal import Metadata, Layer

no longer work. Update them to use the dedicated submodules:

from opendal.types import Metadata
from opendal.layers import Layer

The legacy helper module opendal.__base has also been removed together with _Base.

Breaking change: Capability accessors renamed

Both Operator.full_capability() and AsyncOperator.full_capability() have been renamed to capability(). Adjust your code accordingly:

-caps = op.full_capability()
+caps = op.capability()

Breaking change: Service identifiers now have typed enums

The constructors for Operator / AsyncOperator provide overloads that accept opendal.services.Scheme members. While plain strings are still accepted at runtime, type checkers (pyright/mypy) expect the new enum values. Migrate code bases that relied on importing the old Scheme enum from opendal to from opendal import services and use services.Scheme.<NAME>.

What's Changed

Added

• feat(services/oss): allow users to specify endpoint addressing style by @howardlau1999 in feat(services/oss): allow users to specify endpoint addressing style #6504
• feat(bindings/ruby): publish ruby binding by @erickguan in feat(bindings/ruby): publish ruby binding #6539
• feat(services/s3): add support for HTTP 429 TooManyRequests for S3-compatible services by @justinjoseph89 in feat(services/s3): add support for HTTP 429 TooManyRequests for S3-compatible services #6589
• feat: Add Operator Registry by @Xuanwo in feat: Add Operator Registry #6608
• feat: Add from_uri support for more object storage services by @Xuanwo in feat: Add from_uri support for more object storage services #6665
• feat(services/webdav): Implement write returns metadata by @yunchipang in feat(services/webdav): Implement write returns metadata #6660
• feat: Add from_uri support for http/webdav/ftp/sftp by @Xuanwo in feat: Add from_uri support for http/webdav/ftp/sftp #6666
• feat(services/swift): Implement write returns metadata by @yunchipang in feat(services/swift): Implement write returns metadata #6669
• feat(buffer): add chunk iterator for Buffer by @TennyZhuang in feat(buffer): add chunk iterator for Buffer #6672
• feat(services/dropbox): Implement write returns metadata by @yunchipang in feat(services/dropbox): Implement write returns metadata #6673
• feat: Add from_uri support for all services by @Xuanwo in feat: Add from_uri support for all services #6668
• chore(bindings/python): Add Justfile and format codebase by @chitralverma in chore(bindings/python): Add Justfile and format codebase #6677
• feat: Introduce TailCutLayer by @Xuanwo in feat: Introduce TailCutLayer #6680
• feat(services/gdrive): Implement write returns metadata by @yunchipang in feat(services/gdrive): Implement write returns metadata #6683
• feat(bindings/python): Add stubs for Exception and Capability by @chitralverma in feat(bindings/python): Add stubs for Exception and Capability #6690
• feat(bindings/python): Add stubs for some more types by @chitralverma in feat(bindings/python): Add stubs for some more types #6703
• feat(bindings/python)!: Add stubs for remaining types by @chitralverma in feat(bindings/python)!: Add stubs for remaining types #6720
• feat(bindings/python)!: Stubs fix for operator and version by @chitralverma in feat(bindings/python)!: Stubs fix for operator and __version__ #6728
• RFC-6707: Capability Override Layer by @Xuanwo in RFC-6707: Capability Override Layer #6707
• feat(bindings/python)!: Generate stubs for Operator overloads and Scheme by @chitralverma in feat(bindings/python)!: Generate stubs for Operator overloads and Scheme #6729
• feat(bindings/nodejs): add concurrent limit layer by @kingsword09 in feat(bindings/nodejs): add concurrent limit layer #6739
• feat(bindings/python): Enable ftp/hdfs_native/sftp for python by @chitralverma in feat(bindings/python): Enable ftp/hdfs_native/sftp for python #6745
• feat: implement content-md5 for s3 by @waynexia in feat: implement content-md5 for s3 #6508
• feat(services/compfs): implement IoVectoredBuf for Buffer by @meteorgan in feat(services/compfs): implement IoVectoredBuf for Buffer #6762
• feat(services/lakefs): Implement write returns metadata by @kingsword09 in feat(services/lakefs): Implement write returns metadata #6770
• Add connection_pool_max_size option to memcached and redis builders by @trxcllnt in Add connection_pool_max_size option to memcached and redis builders #6759
• fix:(bindings/python) Gate service-sftp on windows targets by @chitralverma in fix:(bindings/python) Gate service-sftp on windows targets #6777
• feat(binding/nodejs): add TimeoutLayer, LoggingLayer and ThrottleLayer in nodejs binding by @Kilerd in feat(binding/nodejs): add TimeoutLayer, LoggingLayer and ThrottleLayer in nodejs binding #6772

Changed

• refactor: Migrate cloudflare_kv from adapter::kv to Access instead by @kingsword09 in refactor: Migrate cloudflare_kv from adapter::kv to Access instead #6340
• refactor: Centralize date/time handling with Timestamp wrapper by @dami0806 in refactor: Centralize date/time handling with Timestamp wrapper #6650
• refactor: Move Config related back to config.rs instead by @Xuanwo in refactor: Move Config related back to config.rs instead #6667
• refactor: Remove deprecated project virtiofs by @Xuanwo in refactor: Remove deprecated project virtiofs #6682
• refactor(core): remove conditional compilation in PageList trait by @crwen in refactor(core): remove conditional compilation in PageList trait #6715
• refactor: migrate tikv service from adapter::kv to impl Access directly by @koushiro in refactor: migrate tikv service from adapter::kv to impl Access directly #6713
• refactor: migrate memcached service from adapter::kv to impl Access directly by @koushiro in refactor: migrate memcached service from adapter::kv to impl Access directly #6714
• refactor: migrate mysql service from adapter::kv to impl Access directly by @koushiro in refactor: migrate mysql service from adapter::kv to impl Access directly #6716
• refactor: migrate postgresql service from adapter::kv to impl Access directly by @koushiro in refactor: migrate postgresql service from adapter::kv to impl Access directly #6718
• refactor: migrate persy service from adapter::kv to impl Access directly by @koushiro in refactor: migrate persy service from adapter::kv to impl Access directly #6721
• refactor: migrate foundationdb service from adapter::kv to impl Access directly by @koushiro in refactor: migrate foundationdb service from adapter::kv to impl Access directly #6722
• refactor: migrate surrealdb service from adapter::kv to impl Access directly by @koushiro in refactor: migrate surrealdb service from adapter::kv to impl Access directly #6723
• refactor: migrate mongodb service from adapter::kv to impl Access directly by @koushiro in refactor: migrate mongodb service from adapter::kv to impl Access directly #6724
• refactor: migrate d1 service from adapter::kv to impl Access directly by @koushiro in refactor: migrate d1 service from adapter::kv to impl Access directly #6725
• refactor: migrate gridfs service from adapter::kv to impl Access directly by @koushiro in refactor: migrate gridfs service from adapter::kv to impl Access directly #6726
• refactor: Migrate sqlite from adapter::kv to Access instead by @NoxTav in refactor: Migrate sqlite from adapter::kv to Access instead #6328
• refactor: migrate sled service from adapter::kv to impl Access directly by @koushiro in refactor: migrate sled service from adapter::kv to impl Access directly #6731
• refactor: migrate rocksdb service from adapter::kv to impl Access directly by @koushiro in refactor: migrate rocksdb service from adapter::kv to impl Access directly #6732
• refactor: migrate redb service from adapter::kv to impl Access directly by @koushiro in refactor: migrate redb service from adapter::kv to impl Access directly #6733
• refactor: Remove deprecated kv and typed_kv adapters by @kingsword09 in refactor: Remove deprecated kv and typed_kv adapters #6734
• refactor(hdfs): restructure HdfsBackend and introduce HdfsCore by @kingsword09 in refactor(hdfs): restructure HdfsBackend and introduce HdfsCore #6736
• refactor(hdfs-native): restructure HdfsNativeBackend and introduce HdfsNativeCore by @kingsword09 in refactor(hdfs-native): restructure HdfsNativeBackend and introduce HdfsNativeCore #6737
• refactor(services): impl Debug for all service builders by @koushiro in refactor(services): impl Debug for all service builders #6756
• refactor(types)!: use string-based scheme over enum-based approach by @koushiro in refactor(types)!: use string-based scheme over enum-based approach #6765

Fixed

• fix(services/cloudflare_kv): Use DEFAULT_SCHEME constant for Cloudflare KV scheme by @kingsword09 in fix(services/cloudflare_kv): Use DEFAULT_SCHEME constant for Cloudflare KV scheme #6568
• fix(bindings/d): Enhance D bindings build script and tests for OpenDAL by @kingsword09 in fix(bindings/d): Enhance D bindings build script and tests for OpenDAL #6569
• fix(services/obs): send Range header only for non-full reads by @QuakeWang in fix(services/obs): send Range header only for non-full reads #6574
• ci: Fix default branch name in Ruby release workflow by @erickguan in ci: Fix default branch name in Ruby release workflow #6585
• fix: expose ListDyn + WriteDyn by @bonsairobo in fix: expose ListDyn + WriteDyn #6596
• fix(integrations/cloud_filter): upgrade cloud_filter to resolve CI failures after Rust 1.90.0 by @kingsword09 in fix(integrations/cloud_filter): upgrade cloud_filter to resolve CI failures after Rust 1.90.0 #6602
• fix(integrations/unftp-sbe): properly shutdown after write by @choucavalier in fix(integrations/unftp-sbe): properly shutdown after write #6601
• fix(azblob): don't call AzureStorageConfig::from_env on wasm by @bonsairobo in fix(azblob): don't call AzureStorageConfig::from_env on wasm #6594
• fix: censor the "sig" query param for Azure SAS URIs by @bonsairobo in fix: censor the "sig" query param for Azure SAS URIs #6603
• fix(bindings/ruby): publish ruby binding gem by @erickguan in fix(bindings/ruby): publish ruby binding gem #6606
• fix(docs): docs.rs built failed by @FreeChenMou in fix(docs): docs.rs built failed #6658
• fix(layers/throttle): await limiter before throttled writes by @TennyZhuang in fix(layers/throttle): await limiter before throttled writes #6671
• fix:(bindings/python) Minor fix in stubs by @chitralverma in fix:(bindings/python) Minor fix in stubs #6744
• fix: retry on 409 with AWS S3 by @wolfv in fix: retry on 409 with AWS S3 #6742
• fix(bindings/ruby): fix release build error by @erickguan in fix(bindings/ruby): fix release build error #6766
• fix(binding/nodejs): add missing lister methods by @Kilerd in fix(binding/nodejs): add missing lister methods #6769
• fix(bindings/python): Update python/stubgen & core/hdfs-native by @chitralverma in fix(bindings/python): Update python/stubgen & core/hdfs-native #6774

Docs

• doc(bindinds/ruby): add examples and links by @erickguan in doc(bindinds/ruby): add examples and links #6609
• docs: spelling and minor wording change by @carlsonp in docs: spelling and minor wording change #6620
• docs(bin): add badges to bin tools README by @kingsword09 in docs(bin): add badges to bin tools README #6642
• docs(python): add documentation for local usage by @jayceslesar in docs(python): add documentation for local usage #6480
• docs(apps): add migration guide, update links; remove bin/{oli,ofs} (Tracking Issue: Move oli/ofs out; remove oay #6689) by @tao12345666333 in docs(apps): add migration guide, update links; remove bin/{oli,ofs} (#6689) #6705
• docs(core/types): add some comments for Buffer by @meteorgan in docs(core/types): add some comments for Buffer #6746
• docs: Make it clear that bindings have different version with rust core. by @0lai0 in docs: Make it clear that bindings have different version with rust core. #6780

CI

• ci: Switch from --nocapture to --no-capture by @kingsword09 in ci: Switch from --nocapture to --no-capture #6567
• ci: Add patch file to Ruby release workflow by @erickguan in ci: Add patch file to Ruby release workflow #6592
• build: enhance build website by @tisonkun in build: enhance build website #6640
• ci: use shas fix action permissions by @kenwoodjw in ci: use shas fix action permissions #6659
• ci: Bump doc toolchain to allow opendal docs build by @Xuanwo in ci: Bump doc toolchain to allow opendal docs build #6661
• ci: build and test on python3.14t by @kenwoodjw in ci: build and test on python3.14t #6670
• ci(nodejs): enable npm trusted publishing by @kingsword09 in ci(nodejs): enable npm trusted publishing #6685
• ci: remove ofs behavior tests and planning (Tracking Issue: Move oli/ofs out; remove oay #6689) by @tao12345666333 in ci: remove ofs behavior tests and planning (#6689) #6704
• ci: Auto add ML thread link in github discussion by @Xuanwo in ci: Auto add ML thread link in github discussion #6709
• ci: Fix bug in discussion thread link auto append by @Xuanwo in ci: Fix bug in discussion thread link auto append #6711
• ci: Fix GHA CI again by @Xuanwo in ci: Fix GHA CI again #6712
• ci: Remove hdfs-native from arm platform by @Xuanwo in ci: Remove hdfs-native from arm platform #6792

Chore

• ci: switches etcd image to official etcd image by @kingsword09 in ci: switches etcd image to official etcd image #6562
• chore(bindings/nodejs): replace custom sleep with setTimeout in tests by @kingsword09 in chore(bindings/nodejs): replace custom sleep with setTimeout in tests #6563
• chore(layers/prometheus-client): upgrade prometheus-client dependency to v0.24 by @koushiro in chore(layers/prometheus-client): upgrade prometheus-client dependency to v0.24 #6523
• chore(core): typo: update comments on BytesRange::new by @TheR1sing3un in chore(core): typo: update comments on BytesRange::new #6570
• chore(website): update The ASF logo by @suyanhanx in chore(website): update The ASF logo #6575
• chore: fix website build by @tisonkun in chore: fix website build #6579
• chore: make clippy happy when using rust 1.90.0 by @kingsword09 in chore: make clippy happy when using rust 1.90.0 #6590
• chore: apache project provides code under ALv2 by @tisonkun in chore: apache project provides code under ALv2 #6598
• chore: remove dead project openval by @Xuanwo in chore: remove dead project openval #6604
• chore: upgrade opentelemetry to 0.31.0 by @tisonkun in chore: upgrade opentelemetry to 0.31.0 #6623
• chore(deps): bump tempfile from 3.20.0 to 3.23.0 in /bin/ofs by @dependabot[bot] in chore(deps): bump tempfile from 3.20.0 to 3.23.0 in /bin/ofs #6611
• chore(deps): bump tempfile from 3.21.0 to 3.23.0 in /bin/oli by @dependabot[bot] in chore(deps): bump tempfile from 3.21.0 to 3.23.0 in /bin/oli #6610
• chore(deps): update datafusion requirement from 49.0.2 to 50.0.0 in /integrations/object_store by @dependabot[bot] in chore(deps): update datafusion requirement from 49.0.2 to 50.0.0 in /integrations/object_store #6613
• chore(deps): bump @docusaurus/plugin-client-redirects from 3.8.1 to 3.9.1 in /website by @dependabot[bot] in chore(deps): bump @docusaurus/plugin-client-redirects from 3.8.1 to 3.9.1 in /website #6614
• chore: avoid build warning by @tisonkun in chore: avoid build warning #6627
• build: update docusaurus together by @asukaminato0721 in build: update docusaurus together #6626
• chore(deps): bump the github-actions group with 8 updates by @dependabot[bot] in chore(deps): bump the github-actions group with 8 updates #6633
• chore(deps): bump logforth from 0.27.0 to 0.28.1 in /bin/ofs by @dependabot[bot] in chore(deps): bump logforth from 0.27.0 to 0.28.1 in /bin/ofs #6629
• chore(deps): bump the http-serialization-utils group in /core with 7 updates by @dependabot[bot] in chore(deps): bump the http-serialization-utils group in /core with 7 updates #6634
• chore(deps): bump clap from 4.5.38 to 4.5.41 in /bin/oli by @dependabot[bot] in chore(deps): bump clap from 4.5.38 to 4.5.41 in /bin/oli #6630
• chore(deps): bump the logs-errors-checksums group in /core with 2 updates by @dependabot[bot] in chore(deps): bump the logs-errors-checksums group in /core with 2 updates #6635
• chore(deps): bump the pyo3 group in /bindings/python with 2 updates by @dependabot[bot] in chore(deps): bump the pyo3 group in /bindings/python with 2 updates #6631
• chore(deps): bump the third-party-actions group with 2 updates by @dependabot[bot] in chore(deps): bump the third-party-actions group with 2 updates #6636
• chore(deps): bump axum from 0.7.9 to 0.8.6 in /bin/oay by @dependabot[bot] in chore(deps): bump axum from 0.7.9 to 0.8.6 in /bin/oay #6612
• chore(deps): bump the others group in /core with 15 updates by @dependabot[bot] in chore(deps): bump the others group in /core with 15 updates #6637
• chore: OpenDAL MSRV to 1.85 and edition 2024 by @fatelei in chore: OpenDAL MSRV to 1.85 and edition 2024 #6615
• refactor!: migrate chrono to jiff by @tisonkun in refactor!: migrate chrono to jiff #6643
• chore: support more error codes for S3-compatible services by @leiysky in chore: support more error codes for S3-compatible services #6651
• chore: fixup timestamp refactor by @tisonkun in chore: fixup timestamp refactor #6663
• chore: remove oay: delete bin/oay; drop CI; update release/dev tooling, dependabot, docs by @tao12345666333 in chore: remove oay: delete bin/oay; drop CI; update release/dev tooling, dependabot, docs #6691
• chore: cleanup for moving oli/ofs and removing oay (Tracking Issue: Move oli/ofs out; remove oay #6689) by @tao12345666333 in chore: cleanup for moving oli/ofs and removing oay (#6689) #6700
• chore(deps): bump actions/setup-node from 5 to 6 in the github-actions group by @dependabot[bot] in chore(deps): bump actions/setup-node from 5 to 6 in the github-actions group #6695
• chore(deps): bump the http-serialization-utils group across 1 directory with 3 updates by @dependabot[bot] in chore(deps): bump the http-serialization-utils group across 1 directory with 3 updates #6717
• chore(deps): bump the others group in /core with 2 updates by @dependabot[bot] in chore(deps): bump the others group in /core with 2 updates #6698
• chore(deps): bump tokio from 1.47.1 to 1.48.0 in /core in the async-runtime group by @dependabot[bot] in chore(deps): bump tokio from 1.47.1 to 1.48.0 in /core in the async-runtime group #6696
• chore: remove integrations cloud_filter and fuse3; update CI/docs/README by @tao12345666333 in chore: remove integrations cloud_filter and fuse3; update CI/docs/README #6727
• chore: fixup README render and broken links by @tisonkun in chore: fixup README render and broken links #6735
• chore: apply suggestions from clippy 1.91.0 by @kingsword09 in chore: apply suggestions from clippy 1.91.0 #6754
• chore: cleanup some useless code related to services that were removed due to lack of maintenance by @koushiro in chore: cleanup some useless code related to services that were removed due to lack of maintenance #6763
• chore: rename delete module => deleter module; ServiceAccessor => ServiceBackend by @koushiro in chore: rename delete module => deleter module; ServiceAccessor => ServiceBackend #6757
• chore(services): unify scheme usage by @koushiro in chore(services): unify scheme usage #6764
• chore(services): remove empty list operation and update docs of all services by @koushiro in chore(services): remove empty list operation and update docs of all services #6768
• Bump to version 0.55.0 by @Xuanwo in Bump to version 0.55.0 #6771
• chore(types)!: remove deprecated Operator::from_map and Operator::via_map by @koushiro in chore(types)!: remove deprecated Operator::from_map and Operator::via_map #6775
• chore(types)!: remove deprecated list_with_version capability by @koushiro in chore(types)!: remove deprecated list_with_version capability #6781
• chore(services/s3)!: remove deprecated S3::security_token by @koushiro in chore(services/s3)!: remove deprecated S3::security_token #6779
• chore(services/fs): use NotADirectory directly by @meteorgan in chore(services/fs): use NotADirectory directly #6786
• chore: Fix build by @Xuanwo in chore: Fix build #6789
• chore: Don't introduce openssl for python binding by @Xuanwo in chore: Don't introduce openssl for python binding #6790

New Contributors

• @TheR1sing3un made their first contribution in chore(core): typo: update comments on BytesRange::new #6570
• @howardlau1999 made their first contribution in feat(services/oss): allow users to specify endpoint addressing style #6504
• @QuakeWang made their first contribution in fix(services/obs): send Range header only for non-full reads #6574
• @justinjoseph89 made their first contribution in feat(services/s3): add support for HTTP 429 TooManyRequests for S3-compatible services #6589
• @bonsairobo made their first contribution in fix: expose ListDyn + WriteDyn #6596
• @choucavalier made their first contribution in fix(integrations/unftp-sbe): properly shutdown after write #6601
• @carlsonp made their first contribution in docs: spelling and minor wording change #6620
• @FreeChenMou made their first contribution in fix(docs): docs.rs built failed #6658
• @dami0806 made their first contribution in refactor: Centralize date/time handling with Timestamp wrapper #6650
• @yunchipang made their first contribution in feat(services/webdav): Implement write returns metadata #6660
• @jayceslesar made their first contribution in docs(python): add documentation for local usage #6480
• @NoxTav made their first contribution in refactor: Migrate sqlite from adapter::kv to Access instead #6328
• @wolfv made their first contribution in fix: retry on 409 with AWS S3 #6742
• @trxcllnt made their first contribution in Add connection_pool_max_size option to memcached and redis builders #6759
• @sebbASF made their first contribution in No need to copy the whole tree #6783
• @0lai0 made their first contribution in docs: Make it clear that bindings have different version with rust core. #6780

Full Changelog: v0.54.1...v0.55.0

---

This discussion was created from the release v0.55.0.

---

Mailing list thread: https://lists.apache.org/thread/p0crhc7goolgbt3nndjcdh25hd6lt4jm