IndexShards merges non-L2 distance metrics as similarities, reversing METRIC_L1 result ordering #5503

Bug type

INCORRECT_RESULT

Description

IndexShards merges the search results returned by multiple sub-indexes into one global Top-K result.

For METRIC_L1, returned values are distances, so smaller values must rank before larger values. However, when IndexShards merges shard-local results, it treats every metric other than METRIC_L2 as a similarity metric for which larger values are better.

As a result, when using distance metrics such as METRIC_L1, METRIC_Linf, or METRIC_Lp, vectors with larger distances may be returned before vectors with smaller distances.

A single IndexFlat using METRIC_L1 returns the correct ordering. Splitting the same dataset across multiple IndexFlat(METRIC_L1) indexes and combining them with IndexShards reverses the ordering.

Environment

• FAISS version: 1.14.2
• Python version: 3.13.9

Minimal reproducible example

import platform
import sys

import faiss
import numpy as np


print("faiss_version:", faiss.__version__)
print("python_version:", sys.version.split()[0])
print("platform:", platform.platform())

# Store one vector in each shard
xb0 = np.array([[0.0, 0.0]], dtype=np.float32)
xb1 = np.array([[10.0, 0.0]], dtype=np.float32)

# Query vector
xq = np.array([[1.0, 0.0]], dtype=np.float32)

# Full dataset used as the correctness baseline
xb_all = np.vstack([xb0, xb1])

# Baseline: a single IndexFlat returns the correct result
flat = faiss.IndexFlat(2, faiss.METRIC_L1)
flat.add(xb_all)
D_flat, I_flat = flat.search(xq, 2)

# Reproducer: merge two L1 indexes through IndexShards
shard0 = faiss.IndexFlat(2, faiss.METRIC_L1)
shard1 = faiss.IndexFlat(2, faiss.METRIC_L1)

shard0.add(xb0)
shard1.add(xb1)

shards = faiss.IndexShards(2, False, True)
shards.add_shard(shard0)
shards.add_shard(shard1)

D_shards, I_shards = shards.search(xq, 2)

print("flat_labels:", I_flat.tolist())
print("flat_distances:", D_flat.tolist())
print("shards_labels:", I_shards.tolist())
print("shards_distances:", D_shards.tolist())

expected_labels = [[0, 1]]
expected_distances = [[1.0, 9.0]]

assert I_flat.tolist() == expected_labels
assert D_flat.tolist() == expected_distances

# These assertions fail on affected versions.
# Actual result:
# labels [[1, 0]], distances [[9.0, 1.0]]
assert I_shards.tolist() == expected_labels
assert D_shards.tolist() == expected_distances

Expected behavior

The query vector is:

[1, 0]

The two database vectors are:

id 0: [0, 0], L1 distance = 1
id 1: [10, 0], L1 distance = 9

The correct Top-2 result is therefore:

labels:    [[0, 1]]
distances: [[1.0, 9.0]]

IndexShards should return the same result as a single IndexFlat(METRIC_L1) containing the concatenated dataset.

Actual behavior

A single IndexFlat(METRIC_L1) returns the correct result:

flat_labels: [[0, 1]]
flat_distances: [[1.0, 9.0]]

IndexShards returns:

shards_labels: [[1, 0]]
shards_distances: [[9.0, 1.0]]

The vector with distance 9 is incorrectly ranked before the vector with distance 1.

Impact

This issue produces silently incorrect search results.

For sharded indexes using non-L2 distance metrics, vectors with larger distances may be ranked ahead of closer vectors. The search call succeeds without an error, but the returned Top-K ordering can be incorrect, potentially affecting retrieval, reranking, recommendation, and nearest-neighbor search correctness.