Skip to content

fix: handle zero terms in Canberra distance - #5574

Open
LOGO127 wants to merge 1 commit into
facebookresearch:mainfrom
LOGO127:codex/faiss-5557-canberra-zero
Open

fix: handle zero terms in Canberra distance#5574
LOGO127 wants to merge 1 commit into
facebookresearch:mainfrom
LOGO127:codex/faiss-5557-canberra-zero

Conversation

@LOGO127

@LOGO127 LOGO127 commented Sep 4, 2026

Copy link
Copy Markdown

Fixes #5557

Summary

METRIC_Canberra produces NaN when both vectors contain zero in the same dimension because it evaluates 0 / 0. During IndexFlat search, those NaN distances are rejected by the result heap, leaving FLT_MAX/-1 sentinels.

Skip the division when the denominator is zero so that the dimension contributes zero. The regression covers both pairwise_distances and IndexFlat.search with shared-zero dimensions.

Validation

Built the actual C++ library and SWIG bindings in WSL Ubuntu with GCC 13.3, C++20, Release, GPU disabled, and both generic and AVX2 targets. Verified the loaded module was respectively faiss.swigfaiss or faiss.swigfaiss_avx2 from the local build, with no fallback or installed Faiss wheel involved.

  • On base cfbf4ec with the new regression, the test fails in both builds. Separate native probes also reproduce all-NaN pairwise distances and FLT_MAX/-1 search results.
  • With this PR's source, FAISS_OPT_LEVEL=GENERIC python -m pytest tests/test_extra_distances.py -q: 14 passed.
  • FAISS_OPT_LEVEL=AVX2 python -m pytest tests/test_extra_distances.py -q: 14 passed.
  • An additional local SciPy comparison passed in both builds for dimensions 1, 2, 8, 16, 31, 32, 33, and 65, using signed sparse vectors, shared zeros, and both partial and full-k searches; it checks distance values and their association with returned labels.
  • The two changed native source files matched the submitted files by SHA-256. Python compilation, focused Ruff lint, and git diff --check passed.

GPU and Meta's internal test suite were not run locally. Implementation and local validation were assisted by Codex.

@meta-cla

meta-cla Bot commented Sep 4, 2026

Copy link
Copy Markdown

Hi @LOGO127!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@meta-cla meta-cla Bot added the CLA Signed label Sep 4, 2026
@meta-cla

meta-cla Bot commented Sep 4, 2026

Copy link
Copy Markdown

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

METRIC_Canberra produces NaN for shared-zero dimensions, causing valid neighbors to be dropped

1 participant