feat: define external snapshot API contracts (part1) by weiliu1031 · Pull Request #50393 · milvus-io/milvus

weiliu1031 · 2026-06-09T02:18:44Z

issue: #44358

design doc: docs/design-docs/design_docs/20260609-external-snapshot-export-restore.md

Part 1/3 of the external snapshot cross-bucket restore stack.

This PR defines the API contract and entry surfaces for external snapshot export and restore:

Adds the consolidated design document for cross-bucket external snapshot restore.
Adds public gRPC, REST, and Go SDK API surfaces for RestoreExternalSnapshot and ExportSnapshot.
Adds internal DataCoord proto plumbing and generated code needed by later implementation PRs.
Wires Proxy RBAC grouping and database interceptor behavior for the new APIs.
Keeps the request contract on a single external_spec field and keeps db_name for namespace routing rather than permission scoping.

Validation copied from the commit:

GOTOOLCHAIN=go1.25.10 go test -c -tags dynamic,test -gcflags="all=-N -l" -ldflags="-r ${RPATH}" -o /tmp/datacoord-commit1.test github.com/milvus-io/milvus/internal/datacoord
cd client && GOTOOLCHAIN=go1.25.10 go test -c -o /tmp/client-milvusclient-commit1.test ./milvusclient
internal/proxy package compile was blocked locally by missing C++ header internal/core/output/include/segcore/search_result_export_c.h

sre-ci-robot · 2026-06-09T02:19:07Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: weiliu1031
To complete the pull request process, please assign tedxu after the PR has been reviewed.
You can assign the PR to them by writing /assign @tedxu in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mergify · 2026-06-09T02:19:30Z

@weiliu1031 This is a feature PR (feat:). Please provide a design document.

How to resolve:
Add a design document under docs/design-docs/design_docs/ in this PR, or link an existing in-repo design document in the PR description:

design doc: docs/design-docs/design_docs/YYYYMMDD-your_design.md

sre-ci-robot · 2026-06-09T02:19:51Z

[ci-v2-notice]
Notice: New ci-v2 system is enabled for this PR.

To rerun ci-v2 checks, comment with:

/ci-rerun-code-check // for ci-v2/code-check
/ci-rerun-code-check-macos // for Code Checker MacOS (GitHub Actions)
/ci-rerun-build // for ci-v2/build
/ci-rerun-build-all // for ci-v2/build-all (multi-arch builds)
/ci-rerun-buildenv // for ci-v2/build-env (build milvus-env builder images)
/ci-rerun-ut-integration // for ci-v2/ut-integration, will rerun ci-v2/build
/ci-rerun-ut-go // for ci-v2/ut-go, will rerun ci-v2/build
/ci-rerun-ut-cpp // for ci-v2/ut-cpp
/ci-rerun-ut // for all ci-v2/ut-integration, ci-v2/ut-go, ci-v2/ut-cpp, will rerun ci-v2/build
/ci-rerun-e2e-default // for ci-v2/e2e-default
/ci-rerun-e2e-amd // for ci-v2/e2e-amd (e2e pool dispatcher)
/ci-rerun-build-ut-cov // for ci-v2/build-ut-cov (build + unit tests in one pipeline)
/ci-rerun-gosdk // for ci-v2/go-sdk (Go SDK E2E tests, ARM)

If you have any questions or requests, please contact @zhikunyao.

sre-ci-robot · 2026-06-09T03:40:05Z

❌ CI Loop Results `8048e8f`

Stage	Result	Duration	Tests
✅ Build	SUCCESS	15.0min	-
❌ Code-Check	FAILURE	5.5min	-
✅ UT-CPP-Cov	SUCCESS	54.5min	7836 passed

Total: 74min | Pipeline | Artifacts

sre-ci-robot · 2026-06-09T03:50:31Z

❌ CI Loop Results `e14f3c0`

Stage	Result	Duration	Tests

Total: 2min | Pipeline | Artifacts

codecov · 2026-06-09T04:09:14Z

Codecov Report

❌ Patch coverage is 74.31421% with 103 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.96%. Comparing base (fb65266) to head (2a5de0b).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/proxy/snapshot_impl.go	58.33%	39 Missing and 11 partials ⚠️
...nternal/distributed/proxy/httpserver/handler_v2.go	85.61%	15 Missing and 6 partials ⚠️
internal/distributed/mixcoord/client/client.go	0.00%	9 Missing ⚠️
client/milvusclient/snapshot_options.go	85.45%	8 Missing ⚠️
internal/datacoord/services.go	0.00%	7 Missing ⚠️
internal/distributed/proxy/service.go	0.00%	4 Missing ⚠️
internal/coordinator/mix_coord.go	0.00%	2 Missing ⚠️
internal/distributed/mixcoord/service.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #50393       +/-   ##
===========================================
+ Coverage   76.06%   78.96%    +2.89%     
===========================================
  Files        1580     2239      +659     
  Lines      280634   397336   +116702     
===========================================
+ Hits       213478   313754   +100276     
- Misses      57566    73983    +16417     
- Partials     9590     9599        +9

Components	Coverage Δ
Client	`80.48% <91.91%> (+0.11%)`	⬆️
Core	`85.94% <ø> (∅)`
Go	`76.84% <68.54%> (+<0.01%)`	⬆️

Files with missing lines	Coverage Δ
client/milvusclient/snapshot.go	`65.86% <100.00%> (+12.20%)`	⬆️
...nternal/distributed/proxy/httpserver/request_v2.go	`88.59% <100.00%> (+0.07%)`	⬆️
internal/proxy/database_interceptor.go	`86.55% <100.00%> (+0.37%)`	⬆️
pkg/util/constant.go	`14.10% <ø> (ø)`
internal/coordinator/mix_coord.go	`60.68% <0.00%> (-0.15%)`	⬇️
internal/distributed/mixcoord/service.go	`71.72% <0.00%> (-0.27%)`	⬇️
internal/distributed/proxy/service.go	`72.85% <0.00%> (-0.36%)`	⬇️
internal/datacoord/services.go	`81.96% <0.00%> (-0.24%)`	⬇️
client/milvusclient/snapshot_options.go	`76.75% <85.45%> (+3.67%)`	⬆️
internal/distributed/mixcoord/client/client.go	`79.55% <0.00%> (-0.46%)`	⬇️
... and 2 more

... and 693 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sre-ci-robot · 2026-06-09T04:46:35Z

❌ CI Loop Results `1b970d1`

Stage	Result	Duration	Tests
✅ Build	SUCCESS	15.2min	-
❌ Code-Check	FAILURE	7.2min	-
✅ UT-CPP-Cov	SUCCESS	54.8min	7836 passed

Total: 72min | Pipeline | Artifacts

sre-ci-robot · 2026-06-09T04:59:17Z

❌ CI Loop Results `acd84a4`

Stage	Result	Duration	Tests
✅ Build	SUCCESS	9.8min	-
❌ Code-Check	FAILURE	4.0min	-
✅ UT-CPP-Cov	SUCCESS	36.0min	7836 passed

Total: 49min | Pipeline | Artifacts

sre-ci-robot · 2026-06-09T08:01:07Z

✅ CI Loop Results `83764a5`

Stage	Result	Duration	Tests
✅ Build	SUCCESS	11.4min	-
✅ Code-Check	SUCCESS	8.2min	-
✅ UT-GO	SUCCESS	23.0min	1030 passed
✅ UT-Integration	SUCCESS	24.5min	46 passed
✅ UT-CPP-Cov	SUCCESS	37.1min	7836 passed

Total: 76min | Pipeline | Artifacts

Overall Coverage: 71.3%
Diff Coverage: Go 73.2% (224 hit, 82 miss, 306 measurable lines, 3328 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 3328 changed lines were unmeasured.
Total Patch Coverage: 73.2% (224/306 measurable lines, 3328 unmeasured)

issue: milvus-io#44358 Define the design and API contract for external snapshot export and restore. This adds the design record, public gRPC, REST, Go SDK surfaces, internal DataCoord proto plumbing, RBAC grouping, and database interceptor support. The request contract uses a single external_spec carrier and keeps db_name for namespace routing instead of authorization scope. Validation: - GOTOOLCHAIN=go1.25.10 go test -c -tags dynamic,test \ -gcflags="all=-N -l" -ldflags="-r ${RPATH}" \ -o /tmp/datacoord-commit1.test \ github.com/milvus-io/milvus/internal/datacoord - cd client && GOTOOLCHAIN=go1.25.10 go test -c \ -o /tmp/client-milvusclient-commit1.test ./milvusclient - source ~/.profile && cd pkg && GOTOOLCHAIN=go1.25.10 \ go build ./util - source ~/.profile && cd tests/go_client && GOTOOLCHAIN=go1.25.10 \ go list -deps -test ./testcases/... - source ~/.profile && cd pkg && GOTOOLCHAIN=go1.25.10 \ go mod verify - source ~/.profile && cd tests/go_client && GOTOOLCHAIN=go1.25.10 \ go mod verify - internal/proxy package compile blocked by missing local C++ header \ internal/core/output/include/segcore/search_result_export_c.h Signed-off-by: Wei Liu <wei.liu@zilliz.com>

sre-ci-robot · 2026-06-09T11:10:56Z

✅ CI Loop Results `2a5de0b`

Stage	Result	Duration	Tests
✅ Build	SUCCESS	9.3min	-
✅ Code-Check	SUCCESS	6.1min	-
✅ UT-GO	SUCCESS	22.8min	1030 passed
✅ UT-Integration	SUCCESS	24.3min	46 passed
✅ UT-CPP-Cov	SUCCESS	36.4min	7837 passed

Total: 76min | Pipeline | Artifacts

Overall Coverage: 71.3%
Diff Coverage: Go 73.2% (224 hit, 82 miss, 306 measurable lines, 3328 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 3328 changed lines were unmeasured.
Total Patch Coverage: 73.2% (224/306 measurable lines, 3328 unmeasured)

sre-ci-robot added area/dependency Pull requests that update a dependency file area/test sig/testing labels Jun 9, 2026

sre-ci-robot requested review from bigsheeper, congqixia and cydrain June 9, 2026 02:18

sre-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines. label Jun 9, 2026

mergify Bot added dco-passed DCO check passed. do-not-merge/missing-design-doc kind/feature Issues related to feature request from users labels Jun 9, 2026

weiliu1031 force-pushed the restore-snapshot-cross-cluster-part1 branch from 8048e8f to 1b970d1 Compare June 9, 2026 03:09

mergify Bot removed the do-not-merge/missing-design-doc label Jun 9, 2026

weiliu1031 force-pushed the restore-snapshot-cross-cluster-part1 branch 2 times, most recently from e14f3c0 to acd84a4 Compare June 9, 2026 03:33

sre-ci-robot added the low-code-coverage add test-label from zhikun, diff coverage > 80% label Jun 9, 2026

weiliu1031 force-pushed the restore-snapshot-cross-cluster-part1 branch from acd84a4 to 83764a5 Compare June 9, 2026 06:34

sre-ci-robot removed the low-code-coverage add test-label from zhikun, diff coverage > 80% label Jun 9, 2026

sre-ci-robot added the low-code-coverage add test-label from zhikun, diff coverage > 80% label Jun 9, 2026

weiliu1031 force-pushed the restore-snapshot-cross-cluster-part1 branch from 83764a5 to 2a5de0b Compare June 9, 2026 09:36

mergify Bot added the ci-passed label Jun 9, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: define external snapshot API contracts (part1)#50393

feat: define external snapshot API contracts (part1)#50393
weiliu1031 wants to merge 1 commit into
milvus-io:masterfrom
weiliu1031:restore-snapshot-cross-cluster-part1

weiliu1031 commented Jun 9, 2026 •

edited

Loading

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

mergify Bot commented Jun 9, 2026

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

codecov Bot commented Jun 9, 2026 •

edited

Loading

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

weiliu1031 commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

mergify Bot commented Jun 9, 2026

Uh oh!

sre-ci-robot commented Jun 9, 2026

Uh oh!

sre-ci-robot commented Jun 9, 2026

❌ CI Loop Results 8048e8f

Uh oh!

sre-ci-robot commented Jun 9, 2026

❌ CI Loop Results e14f3c0

Uh oh!

codecov Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sre-ci-robot commented Jun 9, 2026

❌ CI Loop Results 1b970d1

Uh oh!

sre-ci-robot commented Jun 9, 2026

❌ CI Loop Results acd84a4

Uh oh!

sre-ci-robot commented Jun 9, 2026

✅ CI Loop Results 83764a5

Uh oh!

sre-ci-robot commented Jun 9, 2026

✅ CI Loop Results 2a5de0b

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

weiliu1031 commented Jun 9, 2026 •

edited

Loading

❌ CI Loop Results `8048e8f`

❌ CI Loop Results `e14f3c0`

codecov Bot commented Jun 9, 2026 •

edited

Loading

❌ CI Loop Results `1b970d1`

❌ CI Loop Results `acd84a4`

✅ CI Loop Results `83764a5`

✅ CI Loop Results `2a5de0b`