Skip to content

Support Roaring-backed bitsets - #1732

Draft
hicder-rddt wants to merge 1 commit into
zilliztech:mainfrom
hicder-rddt:hieu/roaring
Draft

Support Roaring-backed bitsets#1732
hicder-rddt wants to merge 1 commit into
zilliztech:mainfrom
hicder-rddt:hieu/roaring

Conversation

@hicder-rddt

@hicder-rddt hicder-rddt commented Jul 21, 2026

Copy link
Copy Markdown

Add BitsetView support for dense, live Roaring, and frozen Roaring filters, including count updates, dense conversion, offsets, and out-id mapping.

Preserve Roaring filters through search setup instead of rebuilding dense views, and add a typed Faiss fast path that expands Roaring filters into valid ID arrays for fp16, bf16, and int8 exhaustive distance paths.

Wire roaring into Conan and CMake dependencies and cover live and frozen Roaring bitsets with unit tests.

Part of the changes for milvus-io/milvus#51367

@sre-ci-robot

Copy link
Copy Markdown
Collaborator

Welcome @hicder-rddt! It looks like this is your first PR to zilliztech/knowhere 🎉

@mergify

mergify Bot commented Jul 21, 2026

Copy link
Copy Markdown

@hicder-rddt 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

  1. If you're fixing a bug, label it as kind/bug.
  2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
  3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
  4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

@mergify

mergify Bot commented Jul 21, 2026

Copy link
Copy Markdown

@hicder-rddt e2e jenkins job failed, comment /run-e2e can trigger the job again.

@mergify

mergify Bot commented Jul 22, 2026

Copy link
Copy Markdown

@hicder-rddt e2e jenkins job failed, comment /run-e2e can trigger the job again.

@alexanderguzhva

Copy link
Copy Markdown
Collaborator

@hicder-rddt Hi, is it good to review the PR?

@hicder-rddt

Copy link
Copy Markdown
Author

@alexanderguzhva thanks for looking. not yet, I'm still working on the corresponding change on Milvus side. That's why it's still in draft.

Once it's up, I think it's better to review together.

@sre-ci-robot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: hicder-rddt
To complete the pull request process, please assign liliu-z after the PR has been reviewed.
You can assign the PR to them by writing /assign @liliu-z in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Add BitsetView support for dense, live Roaring, and frozen Roaring filters, including count updates, dense conversion, offsets, and out-id mapping.

Preserve Roaring filters through search setup instead of rebuilding dense views, and add a typed Faiss fast path that expands Roaring filters into valid ID arrays for fp16, bf16, and int8 exhaustive distance paths.

Wire roaring into Conan and CMake dependencies and cover live and frozen Roaring bitsets with unit tests.

Signed-off-by: Hieu Pham <hieu.pham@reddit.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants