feat(services/cos): support cosn scheme and STS security token#7418
Open
WangXiaoBao1222 wants to merge 1 commit intoapache:mainfrom
Open
feat(services/cos): support cosn scheme and STS security token#7418WangXiaoBao1222 wants to merge 1 commit intoapache:mainfrom
WangXiaoBao1222 wants to merge 1 commit intoapache:mainfrom
Conversation
- Add 'cosn://' scheme alias alongside 'cos://' for Tencent COS - Add 'security_token' config field for STS temporary credentials - Precedence: explicit config > env (TENCENTCLOUD_TOKEN / TENCENTCLOUD_SECURITY_TOKEN / QCLOUD_SECRET_TOKEN) when config load is enabled - Redact security_token in Debug output - Add unit tests for from_uri parsing (cosn scheme, bucket, root, security_token) and debug redaction
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #7417.
Rationale for this change
This PR adds two related improvements to the Tencent Cloud COS (
services/cos) backend:cosn://scheme alias — In the Hadoop / Spark ecosystem, Tencent COS iscommonly accessed via the
cosn://protocol (see the officialhadoop-cos document). Users migrating Lance / Spark workloads to
OpenDAL expect the same URI style to work out of the box, so we register
cosnas an alias ofcosat the service layer.STS temporary-credential (
security_token) support — Productiondeployments frequently use Tencent Cloud STS (
GetFederationToken/AssumeRole) to issue short-lived credentials consisting of asecret_id,secret_key, and asecurity_token. Previously, onlythe first two could be passed via config; the third one had to be
injected via environment variables, which is awkward in embedded or
multi-tenant scenarios.
What changes are included in this PR?
Register
cosnas an alias ofcosinservices/cos/src/lib.rs.Add
CosConfig::security_tokenconfig field.Add
CosBuilder::security_token(...)setter with doc comments describingthe precedence rules.
In
CosBuilder::build, when bothsecret_idandsecret_keyareprovided, resolve the session token with this precedence:
config.security_tokenset by the user;TENCENTCLOUD_TOKEN,TENCENTCLOUD_SECURITY_TOKEN,QCLOUD_SECRET_TOKEN), only whendisable_config_loadis not set.The resolved token is then forwarded to
StaticCredentialProvider::with_security_tokenso thex-cos-security-tokenheader is attached automatically when signing.Redact
security_tokenin theDebugimpl ofCosConfigto avoidleaking secrets into logs.
Add unit tests covering:
CosConfig::from_uricorrectly parses bothcos://andcosn://URIs, including
bucket,root, andsecurity_token;security_tokenis redacted inDebugoutput.Verified locally:
cargo fmt --all -- --check✅cargo clippy -p opendal-service-cos --all-targets -- -D warnings✅cargo test -p opendal-service-cos✅temporary credentials from
sts.tencentcloudapi.com— read / write /list / delete all succeeded via both
cos://andcosn://schemes.Are there any user-facing changes?
Yes, but fully backward-compatible:
CosBuilder::security_token(...).CosConfig::security_token.cosn://is now accepted wherevercos://was previously accepted.No existing behavior is removed or changed for users who do not set
security_tokenor usecosn://, so nobreaking-changeslabel isrequired.
AI Usage Statement
Parts of this change (code scaffolding, unit tests, and this PR
description) were drafted with the assistance of an AI coding agent
(Anthropic Claude,
claude-4.7-opus). All changes were reviewed, tested,and verified by the author against a real Tencent Cloud COS bucket before
submission.