[Bug]: mmap_forward_store.cc:393 : Failed to find target chunk for index x

[saluto]

Description

Hello,

I ran into the following issue with the latest zvec 0.6.0:

import zvec
import numpy as np

col = zvec.create_and_open("demo", zvec.CollectionSchema(name="demo", vectors=[
    zvec.VectorSchema("embedding", zvec.DataType.VECTOR_FP16, 128, index_param=zvec.HnswIndexParam())
]))
for i in range(16112):
    col.insert([zvec.Doc(id=str(i), vectors={"embedding": np.random.randn(128)})])
col.optimize()
res = col.query(queries=zvec.Query("embedding", vector=np.random.randn(128)), topk=1024)
print("IDs:", {d.id for d in res})
# Output:
#   [ERROR ... mmap_forward_store.cc:393] Failed to find target chunk for index 16068
#   IDs: {''}

The error and the reported index are nondeterministic. The error probability seems to increase with topk. For this specific configuration, I had a 100% error probability so far.

Steps to Reproduce

Install e.g. python 3.12.3 and zvec 0.6.0. Then run:

import zvec
import numpy as np

col = zvec.create_and_open("demo", zvec.CollectionSchema(name="demo", vectors=[
    zvec.VectorSchema("embedding", zvec.DataType.VECTOR_FP16, 128, index_param=zvec.HnswIndexParam())
]))
for i in range(16112):
    col.insert([zvec.Doc(id=str(i), vectors={"embedding": np.random.randn(128)})])
col.optimize()
res = col.query(queries=zvec.Query("embedding", vector=np.random.randn(128)), topk=1024)
print("IDs:", {d.id for d in res})

Logs / Stack Trace

[ERROR ... mmap_forward_store.cc:393] Failed to find target chunk for index 16068
IDs: {''}

Operating System

Ubuntu 24.04.4 LTS

Build & Runtime Environment

zvec 0.6.0, python 3.12.3

[zhourrr]

Thanks for reporting this. The issue was caused by incorrect row lookup when the final IPC chunk was larger than the preceding chunks. It has been fixed in #645 and will be included in v0.7.0, which will be released soon.