Skip to content

enhance: optimize unfiltered search on sealed segments#48272

Open
CLiqing wants to merge 1 commit intomilvus-io:masterfrom
CLiqing:master
Open

enhance: optimize unfiltered search on sealed segments#48272
CLiqing wants to merge 1 commit intomilvus-io:masterfrom
CLiqing:master

Conversation

@CLiqing
Copy link

@CLiqing CLiqing commented Mar 16, 2026

issue: #48279

Optimize the sealed-segment search path when no scalar filter is applied:

  • Three-level MVCC fast path (MvccNode): For sealed segments without collection TTL:
    • Level 1 (no deletes): skip all bitmap operations, reuse thread-local zero bitmap, signal downstream via skip_filter flag
    • Level 2 (has deletes): skip mask_with_timestamps, apply only mask_with_delete
    • Level 3 (default): unchanged original path for filtered / growing / TTL queries
  • VectorSearchNode skip_filter: When skip_filter is set, pass empty BitsetView to Knowhere, triggering native IDSelectorAll instead of iterating over a full zero bitmap
  • Hardware -mpopcnt: Add -mpopcnt compiler flag for x86_64 to use hardware popcnt instruction instead of __popcountdi2 software fallback

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: CLiqing
To complete the pull request process, please assign jiaoew1991 after the PR has been reviewed.
You can assign the PR to them by writing /assign @jiaoew1991 in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot added area/compilation size/L Denotes a PR that changes 100-499 lines. labels Mar 16, 2026
@mergify
Copy link
Contributor

mergify bot commented Mar 16, 2026

@CLiqing Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

@mergify
Copy link
Contributor

mergify bot commented Mar 16, 2026

@CLiqing

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

  1. Title Format: The PR title must begin with one of these prefixes:
  • feat: for introducing a new feature.
  • fix: for bug fixes.
  • enhance: for improvements to existing functionality.
  • test: for add tests to existing functionality.
  • doc: for modifying documentation.
  • auto: for the pull request from bot.
  • build(deps): for dependency updates from Dependabot.
  1. Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, fix, enhance, test or doc.

Example:

enhance: improve search performance significantly 

Please review and update your PR to comply with these guidelines.

@mergify mergify bot added needs-dco DCO is missing in this pull request. do-not-merge/invalid-pr-format labels Mar 16, 2026
@sre-ci-robot
Copy link
Contributor

[ci-v2-notice]
Notice: New ci-v2 system is enabled for this PR.

To rerun ci-v2 checks, comment with:

  • /ci-rerun-code-check // for ci-v2/code-check
  • /ci-rerun-build // for ci-v2/build
  • /ci-rerun-build-all // for ci-v2/build-all (multi-arch builds)
  • /ci-rerun-ut-integration // for ci-v2/ut-integration, will rerun ci-v2/build
  • /ci-rerun-ut-go // for ci-v2/ut-go, will rerun ci-v2/build
  • /ci-rerun-ut-cpp // for ci-v2/ut-cpp
  • /ci-rerun-ut // for all ci-v2/ut-integration, ci-v2/ut-go, ci-v2/ut-cpp, will rerun ci-v2/build
  • /ci-rerun-e2e-arm // for ci-v2/e2e-arm
  • /ci-rerun-e2e-default // for ci-v2/e2e-default
  • /ci-rerun-ciloop // for ci-v2/ciloop (build + unit tests in one pipeline)

If you have any questions or requests, please contact @zhikunyao.

@mergify mergify bot added dco-passed DCO check passed. and removed needs-dco DCO is missing in this pull request. labels Mar 16, 2026
@CLiqing CLiqing force-pushed the master branch 3 times, most recently from a359463 to 6ebac37 Compare March 16, 2026 08:25
@CLiqing CLiqing changed the title perf: optimize unfiltered search on sealed segments enhance: optimize unfiltered search on sealed segments Mar 16, 2026
@mergify mergify bot added kind/enhancement Issues or changes related to enhancement and removed do-not-merge/invalid-pr-format labels Mar 16, 2026
@mergify
Copy link
Contributor

mergify bot commented Mar 16, 2026

@CLiqing Please associate the related issue to the body of your Pull Request. (eg. "issue: #")

@codecov
Copy link

codecov bot commented Mar 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.74%. Comparing base (8900b46) to head (f0d5ead).
⚠️ Report is 12 commits behind head on master.

❌ Your project check has failed because the head coverage (74.74%) is below the target coverage (77.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           master   #48272       +/-   ##
===========================================
- Coverage   77.53%   74.74%    -2.80%     
===========================================
  Files        2099     1480      -619     
  Lines      348958   244980   -103978     
===========================================
- Hits       270576   183104    -87472     
+ Misses      70137    53607    -16530     
- Partials     8245     8269       +24     
Components Coverage Δ
Client 78.62% <ø> (ø)
Core ∅ <ø> (∅)
Go 75.62% <92.58%> (-0.09%) ⬇️
see 673 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sre-ci-robot sre-ci-robot added the low-code-coverage add test-label from zhikun, diff coverage > 80% label Mar 16, 2026
@sre-ci-robot sre-ci-robot added size/M Denotes a PR that changes 30-99 lines. and removed size/L Denotes a PR that changes 100-499 lines. labels Mar 18, 2026
@mergify
Copy link
Contributor

mergify bot commented Mar 18, 2026

@CLiqing Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

@mergify mergify bot added needs-dco DCO is missing in this pull request. and removed dco-passed DCO check passed. labels Mar 18, 2026
@mergify
Copy link
Contributor

mergify bot commented Mar 18, 2026

@CLiqing go-sdk check failed, comment rerun go-sdk can trigger the job again.

@sre-ci-robot sre-ci-robot added size/L Denotes a PR that changes 100-499 lines. and removed size/M Denotes a PR that changes 30-99 lines. labels Mar 18, 2026
@mergify mergify bot added dco-passed DCO check passed. and removed needs-dco DCO is missing in this pull request. labels Mar 18, 2026
Three independent optimizations for the sealed-segment search path:

1. Three-level MVCC fast path (MvccNode): skip bitmap ops when sealed
   + no deletes; apply only delete mask when deletes exist; default
   path for filtered/growing/TTL queries.

2. VectorSearchNode skip_filter: pass empty BitsetView to Knowhere
   (IDSelectorAll) when MvccNode signals no filtering needed.

3. Hardware -mpopcnt for x86_64: enable __builtin_popcountll hardware
   instruction instead of software fallback.

issue: milvus-io#48279

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: CLiQing <2208529306@qq.com>
@mergify
Copy link
Contributor

mergify bot commented Mar 18, 2026

@CLiqing go-sdk check failed, comment rerun go-sdk can trigger the job again.

@sre-ci-robot sre-ci-robot removed the low-code-coverage add test-label from zhikun, diff coverage > 80% label Mar 18, 2026
@CLiqing
Copy link
Author

CLiqing commented Mar 19, 2026

rerun go-sdk

@mergify
Copy link
Contributor

mergify bot commented Mar 19, 2026

@CLiqing go-sdk check failed, comment rerun go-sdk can trigger the job again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/compilation dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement size/L Denotes a PR that changes 100-499 lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants