Why is the recall rate of IP lower than L2 for IVFPQ ?

[shiwanghua]

Why is the recall rate of IP lower than L2 for IVFPQ ?

lower about 5~30% for top1 and top10

[mdouze]

It depends on the data distribution. But indeed it is typical that IP is harder to index than L2. This is because the quantizers are trained with k-means that guarantees to minimize the quantization error only for squared L2.

[shiwanghua]

It depends on the data distribution. But indeed it is typical that IP is harder to index than L2. This is because the quantizers are trained with k-means that guarantees to minimize the quantization error only for squared L2.

But I used HNSWFlatIP as the k-means quantizers for IVF train and PQ train.