fix: handle zero terms in Canberra distance - #5574
Conversation
|
Hi @LOGO127! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
Fixes #5557
Summary
METRIC_CanberraproducesNaNwhen both vectors contain zero in the same dimension because it evaluates0 / 0. DuringIndexFlatsearch, thoseNaNdistances are rejected by the result heap, leavingFLT_MAX/-1sentinels.Skip the division when the denominator is zero so that the dimension contributes zero. The regression covers both
pairwise_distancesandIndexFlat.searchwith shared-zero dimensions.Validation
Built the actual C++ library and SWIG bindings in WSL Ubuntu with GCC 13.3, C++20, Release, GPU disabled, and both generic and AVX2 targets. Verified the loaded module was respectively
faiss.swigfaissorfaiss.swigfaiss_avx2from the local build, with no fallback or installed Faiss wheel involved.cfbf4ecwith the new regression, the test fails in both builds. Separate native probes also reproduce all-NaNpairwise distances andFLT_MAX/-1search results.FAISS_OPT_LEVEL=GENERIC python -m pytest tests/test_extra_distances.py -q: 14 passed.FAISS_OPT_LEVEL=AVX2 python -m pytest tests/test_extra_distances.py -q: 14 passed.git diff --checkpassed.GPU and Meta's internal test suite were not run locally. Implementation and local validation were assisted by Codex.