some questions about horizontal scaling and resources

[artesby]

Hello, i have some questions about resource usage and horizontal scaling, thanks in advance for sharing your experience.

1. I am using milvus calculator. There is "number of vectors" parameter. Is there difference whether these vectors belong to single collection or multiple collections? for example, i have 3 collections 15kk vectors each. Is it the same as if i had 1 collection with 45kk vectors? In what direction should i adjust resources depending on distribution between collections?
2. calculator suggests 8\32 instance for proxy|mixcoord|datanode. However, my actual usage is barely 1cpu\3gb. QPS is 60-80 for search (in total, across all collections) and 30-40 for upsert. Is this qps considered low? is this the reason actual usage is low? Maybe you know what QPS is implied by calculator on that limits?
3. what is correct way of scaling index nodes. Is having 4 index nodes of size 4\16 the same as having 2 nodes 8\32 or 1 node 16\64 in case of performance?
4. same question as 3 for query nodes
5. does it make sense to increase query\index node number without increasing collection replica count?

[xiaofan-luan]

1. Is it the same as if i had 1 collection with 45kk vectors? In what direction should i adjust resources depending on distribution between collections?

1. yes, it's the samilar. unless you have 10K+ collections, it doesn't make too much difference for milvus.
2. 8/32 means 8core 32G? usually only proxy/querynode is related to your peformance. Rest of the mode is not related to your query performance but only related to how much data you have.
3. You can use 8core 32G index node and scale it if index is slow.
4. For querynodes it's based on your total cluster size. I would recommend you to use 8core 32g/64g for cluster < 64core and 16core 64G for cluster > 64core
5. it depends. if you cluster is large and have multiple segments then yes you don't need to increase collection replica. On the other side, if you have relatively small collection, then usually a collection can not be split to multiple segments, you will need to increase replica of collection to make sure the querynode you scaled is actually working

[artesby]

Thank you, that has clarified some things for me.

I have one more question: could you provide any advice on increasing maximum search throughput? Currently, increasing QPS results in higher latency, but throughput remains +- the same. My setup is:

• Milvus 2.5.6
• 3 query nodes (8 CPU, 56 GiB RAM each)
• 3 collections, each with 12 million vectors (vector size = 1024), indexed using IVF_FLAT with nlist = 3162
• Each collection has 1 partition, about 40 segments (1 GB), 1 replica, and Bounded consistency.

I have already found in discussions that one way to increase search performance is to set replicas = 2 and deploy x2 query nodes. However, this seems quite expensive, as each additional 80-100 QPS would require 3 more nodes with 8 CPU / 64 GiB RAM. Are there any other parameters I should consider? For example, I can see in Attu that not all segments are assigned to a query node. Is this expected behavior?

[xiaofan-luan]

Thank you, that has clarified some things for me.

I have one more question: could you provide any advice on increasing maximum search throughput? Currently, increasing QPS results in higher latency, but throughput remains +- the same. My setup is:

• Milvus 2.5.6
• 3 query nodes (8 CPU, 56 GiB RAM each)
• 3 collections, each with 12 million vectors (vector size = 1024), indexed using IVF_FLAT with nlist = 3162
• Each collection has 1 partition, about 40 segments (1 GB), 1 replica, and Bounded consistency.

I have already found in discussions that one way to increase search performance is to set replicas = 2 and deploy x2 query nodes. However, this seems quite expensive, as each additional 80-100 QPS would require 3 more nodes with 8 CPU / 64 GiB RAM. Are there any other parameters I should consider? For example, I can see in Attu that not all segments are assigned to a query node. Is this expected behavior?

1. Easiest way to improve the QPS is to use HNSW index instead of IVF
2. The other way is to tune nprobe. The less the nprobe, the less the recall but faster throughput.
3. For 3 nodes with 40 segments, each node has roughly 10+ segments and it's should be balanced