KNN by doc_id returns too few rows because the source document is removed after LIMIT is applied

[sanikolaev]

Bug Description:

MRE:

mysql> DROP TABLE IF EXISTS t; CREATE TABLE t (f text,  v float_vector knn_type='hnsw' knn_dims='2' hnsw_similarity='l2'); INSERT INTO t (id, f, v) VALUES (1,  'doc1',  (1,0)),(2,  'doc2',  (2,0)),(3,  'doc3',  (3,0)); SELECT id, knn_dist() AS distance FROM t WHERE knn(v, 25, 1) limit 2;
--------------
DROP TABLE IF EXISTS t
--------------

Query OK, 0 rows affected (0.005 sec)

--------------
CREATE TABLE t (f text,  v float_vector knn_type='hnsw' knn_dims='2' hnsw_similarity='l2')
--------------

Query OK, 0 rows affected (0.004 sec)

--------------
INSERT INTO t (id, f, v) VALUES (1,  'doc1',  (1,0)),(2,  'doc2',  (2,0)),(3,  'doc3',  (3,0))
--------------

Query OK, 3 rows affected (0.001 sec)

--------------
SELECT id, knn_dist() AS distance FROM t WHERE knn(v, 25, 1) limit 2
--------------

+------+----------+
| id   | distance |
+------+----------+
|    2 |        1 |
+------+----------+
1 row in set (0.003 sec)

Expected: 2 documents.

Manticore Search Version:

Manticore 25.6.9 1c407206f@26042207 dev (columnar 13.2.3 1e3cb2e@26042021) (secondary 13.2.3 1e3cb2e@26042021) (knn 13.2.3 1e3cb2e@26042021) (embeddings 1.1.1 1e3cb2e@26042021)

Operating System Version:

macos

Have you tried the latest development version?

Yes

Internal Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

Details

• Implementation completed
• Tests developed
• Documentation updated
• Documentation reviewed

[KlimTodrik]

Merged in Buddy #661
Waiting for merge in manticoresearch repo manticoresoftware/manticoresearch#4574

[github-actions]

Thanks for the update! However, the internal checklist is not yet complete. A Manticore team member will close this issue once everything is checked off. 🛠️