Regarding Benchmark Performance and Achieving Higher RPS

[dmitriivahrushev]

Hello Qdrant Team,

I hope this message finds you well.

I have a question regarding the benchmarks presented on your website (https://qdrant.tech/benchmarks/), specifically regarding achieving higher RPS values.

I ran the benchmarks locally using the following setup:

Hardware: 8 vCPUs, 16 GiB memory, 233 GiB storage
Command: python3 run.py --engines "qdrant-sq-rps-m-64-ef-512" --datasets "dbpedia-openai-100K-1536-angular"
Configuration: qdrant-sq-rps-m-64-ef-512
Observed Result: With 100 threads, the RPS did not exceed ~305.
I noticed the benchmark results on your website claim significantly higher RPS values (e.g., 1238 RPS in the example table in the FAQ) for similar configurations.

I have a few questions:

Could you please confirm if I am interpreting the benchmark results correctly, or if there's a specific setup or configuration detail I might be missing to achieve the higher RPS figures listed?
Are the published benchmarks run on significantly different hardware than the "average machine" mentioned in the FAQ? Or perhaps there are specific tuning parameters or test conditions (like the exact dataset size or search parameters like ef_search) that are optimized for the RPS test scenario?
I am trying to understand the potential performance ceiling and the best practices for tuning Qdrant for high RPS scenarios, especially with large collections. Any insights or guidance you could provide would be greatly appreciated.

Thank you very much for your time and for developing Qdrant. I look forward to your response.
qdrant-sq-rps-m-64-ef-512.zip