Skip to content

feat(storage): add endpoint egress policy#19980

Draft
youngsofun wants to merge 2 commits into
databendlabs:mainfrom
youngsofun:fix/storage-endpoint-egress-policy
Draft

feat(storage): add endpoint egress policy#19980
youngsofun wants to merge 2 commits into
databendlabs:mainfrom
youngsofun:fix/storage-endpoint-egress-policy

Conversation

@youngsofun

@youngsofun youngsofun commented Jun 9, 2026

Copy link
Copy Markdown
Member

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Add a configurable storage endpoint egress policy for external storage endpoint_url values. This gives deployments a server-side control to reject high-risk targets such as loopback, private/link-local ranges, metadata endpoints, and configured databend-meta sockets while keeping the default behavior permissive for self-hosted compatibility.

The policy is enforced in two places: early endpoint parsing for storage locations, and the storage HTTP client before requests are sent. The default permissive mode preserves existing S3-compatible/private storage setups, while strict mode enables stronger protection for Cloud or hardened deployments.

This also treats wildcard databend-meta binds such as 0.0.0.0:9191 as protected sockets for local/private targets, and pins hostname requests to the DNS addresses that passed policy validation.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Executed:

  • cargo fmt --check
  • cargo test -p databend-common-storage endpoint_policy --lib
  • cargo clippy -p databend-common-storage --lib -- -D warnings
  • cargo check -p databend-common-config -p databend-common-sql

Also updated configs_table_basic.txt golden output for the new storage endpoint policy config rows.

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions Bot added the pr-feature this PR introduces a new feature to the codebase label Jun 9, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 043ca26b91

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@youngsofun youngsofun marked this pull request as draft June 9, 2026 10:33
@youngsofun youngsofun force-pushed the fix/storage-endpoint-egress-policy branch from 043ca26 to 589f70f Compare June 9, 2026 10:40
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

🤖 CI Job Analysis

Workflow: 27208522360

📊 Summary

  • Total Jobs: 90
  • Failed Jobs: 2
  • Retryable: 0
  • Code Issues: 2

NO RETRY NEEDED

All failures appear to be code/test issues requiring manual fixes.

🔍 Job Details

  • linux / test_unit: Not retryable (Code/Test)
  • linux / test_logs: Not retryable (Code/Test)

🤖 About

Automated analysis using job annotations to distinguish infrastructure issues (auto-retried) from code/test issues (manual fixes needed).

@youngsofun youngsofun force-pushed the fix/storage-endpoint-egress-policy branch from 589f70f to 9448e62 Compare June 9, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant