fix: load nullable diskann valid data for stream index

[marcelo-cjl]

Summary

• Cherry-pick fix: load nullable diskann valid data for stream index #50283 from 2.6 to master.
• Cache nullable vector valid_data sidecar for stream-loaded disk ANN indexes so offset mapping can be built.
• Add C++ coverage for valid_data slice filtering/cache, multiple valid_data slices, and stream load mapping.

Related PR: #50283

Test plan

• git diff --check upstream/master..HEAD
• make cppcheck
• C++ unittest not run locally: clean workspace has no built internal/core/output/unittest/all_tests; targeted C++ test needs core unittest build.

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: marcelo-cjl
To complete the pull request process, please assign jiaoew1991 after the PR has been reviewed.
You can assign the PR to them by writing /assign @jiaoew1991 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• internal/core/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sre-ci-robot]

[ci-v2-notice]
Notice: New ci-v2 system is enabled for this PR.

To rerun ci-v2 checks, comment with:

• /ci-rerun-code-check // for ci-v2/code-check
• /ci-rerun-code-check-macos // for Code Checker MacOS (GitHub Actions)
• /ci-rerun-build // for ci-v2/build
• /ci-rerun-build-all // for ci-v2/build-all (multi-arch builds)
• /ci-rerun-buildenv // for ci-v2/build-env (build milvus-env builder images)
• /ci-rerun-ut-integration // for ci-v2/ut-integration, will rerun ci-v2/build
• /ci-rerun-ut-go // for ci-v2/ut-go, will rerun ci-v2/build
• /ci-rerun-ut-cpp // for ci-v2/ut-cpp
• /ci-rerun-ut // for all ci-v2/ut-integration, ci-v2/ut-go, ci-v2/ut-cpp, will rerun ci-v2/build
• /ci-rerun-e2e-default // for ci-v2/e2e-default
• /ci-rerun-e2e-amd // for ci-v2/e2e-amd (e2e pool dispatcher)
• /ci-rerun-build-ut-cov // for ci-v2/build-ut-cov (build + unit tests in one pipeline)
• /ci-rerun-gosdk // for ci-v2/go-sdk (Go SDK E2E tests, ARM)

If you have any questions or requests, please contact @zhikunyao.

[sre-ci-robot]

✅ CI Loop Results caffd2b

Stage	Result	Duration	Tests
✅ Build	SUCCESS	14.1min	-
✅ Code-Check	SUCCESS	8.6min	-
✅ UT-GO	SUCCESS	23.7min	1012 passed
✅ UT-Integration	SUCCESS	25.3min	46 passed
✅ UT-CPP-Cov	SUCCESS	56.1min	7839 passed

Total: 76min | Pipeline | Artifacts

Overall Coverage: 71.3%
Diff Coverage: CPP 86.7% (189 hit, 29 miss, 218 measurable lines, 147 unmeasured)
Diff Coverage HTML: view changed lines
Total Patch Coverage: 86.7% (189/218 measurable lines, 147 unmeasured)

[codecov]

Codecov Report

❌ Patch coverage is 97.63314% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.96%. Comparing base (43010a4) to head (904a59d).
⚠️ Report is 8 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/core/src/storage/DiskFileManagerTest.cpp	97.90%	3 Missing ⚠️
...nternal/core/src/index/VectorIndexValidDataUtils.h	95.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #50361      +/-   ##
==========================================
- Coverage   79.01%   78.96%   -0.05%     
==========================================
  Files        2239     2239              
  Lines      396634   396800     +166     
==========================================
- Hits       313391   313342      -49     
- Misses      73703    73868     +165     
- Partials     9540     9590      +50     

Components	Coverage Δ
Client	80.59% <ø> (ø)
Core	85.97% <97.63%> (+0.01%)	⬆️
Go	76.82% <ø> (-0.08%)	⬇️

Files with missing lines	Coverage Δ
internal/core/src/index/VectorDiskIndex.cpp	90.07% <100.00%> (+0.07%)	⬆️
...nternal/core/src/index/VectorIndexValidDataUtils.h	95.83% <95.00%> (-0.33%)	⬇️
internal/core/src/storage/DiskFileManagerTest.cpp	99.06% <97.90%> (-0.18%)	⬇️

... and 45 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sre-ci-robot]

❌ CI Loop Results 904a59d

Stage	Result	Duration	Tests
✅ Build	SUCCESS	9.6min	-
❌ Code-Check	FAILURE	1.9min	-
❌ UT-CPP-Cov	FAILURE	8.2min	-

Total: 21min | Pipeline | Artifacts

Failed Test Logs:

• UT-CPP-Cov: view log

[marcelo-cjl]

/ci-rerun-e2e-default

[marcelo-cjl]

/ci-rerun-gosdk

[marcelo-cjl]

/ci-rerun-ut-go

[marcelo-cjl]

/ci-rerun-code-check

[marcelo-cjl]

/ci-rerun-build-ut-cov

[sre-ci-robot]

✅ CI Loop Results 904a59d

Stage	Result	Duration	Tests
✅ Build	SUCCESS	9.2min	-
✅ Code-Check	SUCCESS	4.8min	-
✅ UT-GO	SUCCESS	14.4min	1012 passed
✅ UT-Integration	SUCCESS	23.6min	46 passed
✅ UT-CPP-Cov	SUCCESS	39.4min	7839 passed

Total: 58min | Pipeline | Artifacts

Overall Coverage: 71.3%
Diff Coverage: CPP 96.3% (184 hit, 7 miss, 191 measurable lines, 88 unmeasured)
Diff Coverage HTML: view changed lines
Total Patch Coverage: 96.3% (184/191 measurable lines, 88 unmeasured)

[marcelo-cjl]

/ci-rerun-gosdk

[marcelo-cjl]

/ci-rerun-e2e-default

[marcelo-cjl]

/ci-rerun-gosdk