search and query latency

[zhu3359]

i delpoy a miluvs cluster who has two querynode, and create two collections with 20k entities each, i test that it takes about more than 400ms to query and search, dose this meet the expections？ i see some data from monitoring service,
Search Latency: p99-milvus-proxy--query: 500+ms
p99-milvus-proxy--search: 2ms
p99-milvus-querynode--query: 500+ms
p99-milvus-querynode--query-Sealed: 500+ms

[yhmo]

Set consistency_level to be "Bounded" or "Eventually".

results = client.search(collection_name=collection_name,
                data=[vector],
                anns_field="vector",
                limit=10,
                consistency_level="Bounded",
                search_params={})

For more info: https://milvus.io/docs/consistency.md#Consistency-Level

[zhu3359]

Set consistency_level to be "Bounded" or "Eventually".

results = client.search(collection_name=collection_name,
                data=[vector],
                anns_field="vector",
                limit=10,
                consistency_level="Bounded",
                search_params={})

For more info: https://milvus.io/docs/consistency.md#Consistency-Level

@yhmo i didn‘t set consistency_level when create collection， and didn't set consistency_level when search the collection also; so the consistency_level is default "Bounded" ?

[yhmo]

Ideally, 20k entities, search latency should be a few milliseconds. 400ms is much longer than expected.
The most common reason is consistency level to be "Strong".
So, you can try to explicitly set consistency_level to be "Bounded"/"Eventually". If the search latency is still 400ms, then we can exclude this reason.

The other possible reasons:

• output_fields contain long-length string field or vector field and the "limit" is a large value.
• super high dimension vector

[zhu3359]

Ideally, 20k entities, search latency should be a few milliseconds. 400ms is much longer than expected. The most common reason is consistency level to be "Strong". So, you can try to explicitly set consistency_level to be "Bounded"/"Eventually". If the search latency is still 400ms, then we can exclude this reason.

The other possible reasons:

• output_fields contain long-length string field or vector field and the "limit" is a large value.
• super high dimension vector

@yhmo the dimension of vector is 1792, is this super high？and in the proxy panel , search latency p99-milvus-proxy--search is just 2ms, but p99-milvus-proxy--query is 500ms+, what is the meaning? vector search cost 2ms but client.query() cost 500ms+?

[xiaofan-luan]

query means it's just filtering in milvus.
search means an ann search.

in your case what is your expression of query and search? are they shared the same expression?

[yhmo]

1792 is not super high.

• search latency
  p99-milvus-proxy--search = 2ms is no problem since proxy simply receives the request and passes the request to query nodes, it doesn't compute.
  p99-milvus-querynode--query-Sealed = 500+ms, it is slower then expected. Possible reasons: output_fields returns too much data, or filtering on json field as @xiaofan-luan mentioned.

• query latency
  p99-milvus-proxy--query: 500+ms
  p99-milvus-querynode--query: 500+ms
  query() is to retrieve data to the client, the latency mainly depends on how much data is returned and the filtering expression also affect latency.

[xiaofan-luan]

My guess is that you are filtering a json filed or dynamic field, causing the performance become bad

[zhu3359]

@yhmo @xiaofan-luan there is no json field, but there is a string field book_info whose max length may reach 4k and has no inverted index，and filter is”book_info like "%第一%“, i think the reason is "like" not suitable for long string field, besides the output field include book_info, thus make it takes longer time to query
res = client.query(
collection_name=collection_name,
filter='book_info like "%第一%"',
output_fields=["book_name", "author_id", "author_name", "book_info"]
)

[xiaofan-luan]

like is probably is the reason of long filtering time.
think of using text match instead of doing match. see https://milvus.io/docs/keyword-match.md#Text-Match