HNSW external-vector fast search calculates incorrect scores and causes severe recall loss

[shirly121]

Description

When HNSW is configured with use_external_vector=true, searches calculate incorrect scores and return results with very low recall once the number of vectors exceeds the default brute-force threshold of 1000.

The issue is reproducible using only the ZVec C++ core API through VectorSource, AddWithSource, and SearchWithSource.

Reproduction configuration

Data type: FP32
Dimension: 64
Metric: Inner Product
Number of vectors: 2000
TopK: 20
M: 16
ef_construction: 200
ef_search: 500
use_external_vector: true

The test uses deterministic, normalized random vectors.

class TestVectorSource : public zvec::core::VectorSource {
 public:
  TestVectorSource(const float* base, uint32_t dim)
      : base_(base), dim_(dim) {}

  const void* get_vector(uint32_t node_id) const override {
    return base_ + static_cast<size_t>(node_id) * dim_;
  }

 private:
  const float* base_;
  uint32_t dim_;
};

auto param = HNSWIndexParamBuilder()
                 .WithMetricType(MetricType::kInnerProduct)
                 .WithDataType(DataType::DT_FP32)
                 .WithDimension(64)
                 .WithIsSparse(false)
                 .WithM(16)
                 .WithEFConstruction(200)
                 .WithUseExternalVector(true)
                 .Build();

auto index = IndexFactory::CreateAndInitIndex(*param);
index->Open(index_path,
            {StorageOptions::StorageType::kMMAP, true});

for (uint32_t id = 0; id < vectors.size(); ++id) {
  VectorData value{
      DenseVector{all_vectors.data() + id * dimension}};
  index->AddWithSource(value, id, source);
}

auto query_param = HNSWQueryParamBuilder()
                       .with_topk(20)
                       .with_fetch_vector(true)
                       .with_ef_search(500)
                       .build();

SearchResult result;
index->SearchWithSource(query, query_param, source, &result);

Actual behavior

The test consistently produces very low recall:

recall@20=2/20

The vectors returned through fetch_vector=true match the original vectors, but most scores calculated by ZVec are incorrect.

For example:

key=647
zvec_score=2.7003e-42
exact_score=0.0175026

original[0:3]=[-0.0631494, -0.190733, 0.151877]
fetched[0:3]=[-0.0631494, -0.190733, 0.151877]

The fetched vector matches the original VectorSource data exactly, while the score calculated during search is incorrect.

More importantly, the exact nearest vector is present when running a diagnostic query that returns all documents, but ZVec assigns it an incorrect score:

exact top-1 diagnostic:
key=181
zvec_score=3.57331e-43
exact_score=0.428171

Because the true nearest vector receives an almost-zero score, it is excluded from the returned Top20.

This directly demonstrates that the recall loss is caused by incorrect score computation in the external-vector fast-search path, rather than missing or corrupted source vectors.

Threshold behavior

The issue appears exactly when HNSW switches from brute-force search to graph search:

N=1000: recall@20=20/20
N=1001: recall@20=0/20
N=5000: recall@20=0/20

This corresponds to the following condition:

if (entity_->doc_cnt() <= ctx->get_bruteforce_threshold()) {
  return search_bf_impl(query, qmeta, count, context);
}

The default brute-force threshold is 1000. Existing tests with at most 1000 vectors do not enter the HNSW fast-search path and therefore do not expose this issue.

Suspected cause

HnswExternalStreamerEntity uses MmapMemoryBlock, so an unfiltered level-0 search enters the mmap fast path:

if constexpr (std::is_same_v<MemBlockType, MmapMemoryBlock>) {
  fast_search_neighbors(...);
}

Inside fast_search_neighbors(), neighbor vectors are retrieved with:

const void* vec_ptr = entity.get_vector_ptr(node);

However, HnswExternalStreamerEntity does not provide an external-vector-specific get_vector_ptr(). It inherits the mmap implementation, which reads from the HNSW node chunk:

const void* get_vector_ptr(node_id_t id) const {
  uint32_t chunk_idx = id >> node_index_mask_bits_;
  uint32_t offset = (id & node_index_mask_) * node_size();
  return get_node_chunk_base(chunk_idx) + offset;
}

In external-vector mode, vector_size is set to zero because vectors are not stored in the HNSW node chunks. Consequently, this pointer refers to the key or graph data instead of the vector returned by VectorSource.

The entry-point score can still be correct because it is calculated through HnswDistCalculator, which uses VectorSource. Scores for neighbors expanded by the level-0 fast path are calculated using the incorrect pointer.

Expected behavior

With use_external_vector=true:

• Every distance calculation should use vectors returned by VectorSource.
• Returned scores should match scores calculated from the source vectors.
• The true nearest vector should not be excluded due to an incorrectly calculated score.
• Recall should not collapse when the document count exceeds 1000.

Possible fix

Provide external-vector-specific direct pointer access:

const void* get_vector_ptr(node_id_t id) const {
  return vec_src_ ? vec_src_->get_vector(id) : nullptr;
}

Alternatively, prevent HnswExternalStreamerEntity from entering the mmap direct-pointer fast path and use the existing get_vector_typed()/VectorSource path instead.

Regression test

A regression test can be added to tests/core/interface/index_interface_test.cc and run with:

cmake --build build --target index_interface_test -j<physical-core-count>

build/bin/index_interface_test \
  --gtest_filter=IndexInterface.ExternalVectorFastSearchRecallRegression

Example output:

recall@20=2/20
exact top-1 key=181 score=0.428171
exact top-1 diagnostic:
key=181 zvec_score=3.57331e-43 exact_score=0.428171