Skip to content

Conversation

@OVI3D0
Copy link
Member

@OVI3D0 OVI3D0 commented Jan 12, 2026

Description

Adds POC for Vespa search engine.

Creates a unified abstraction layer for clients + runners and adds vespa client + runners

Benchmarks can be run on Vespa with appropriate configurations + schema for deploying data. Example for nyc_taxis can be found on my fork here: https://github.com/OVI3D0/opensearch-benchmark-workloads/tree/multi-engine

To tell OSB we're running on vespa, we'll use a new --database-type argument. If not provided, OSB will assume we're running against an opensearch cluster. Otherwise, you can tell OSB to benchmark using vespa client + runners by adding --database-type=vespa to your benchmark command.

Issues Resolved

#1000

Testing

  • New functionality includes testing

Still seems to be some issues with indexing due to 429 errors but I'm able to run a full benchmark E2E:

|         50th percentile latency |       asc_sort_tip_amount | 445.257 |     ms |
|         90th percentile latency |       asc_sort_tip_amount | 451.081 |     ms |
|         99th percentile latency |       asc_sort_tip_amount | 455.942 |     ms |
|        100th percentile latency |       asc_sort_tip_amount | 457.462 |     ms |
|    50th percentile service time |       asc_sort_tip_amount | 443.405 |     ms |
|    90th percentile service time |       asc_sort_tip_amount | 448.819 |     ms |
|    99th percentile service time |       asc_sort_tip_amount | 453.214 |     ms |
|   100th percentile service time |       asc_sort_tip_amount | 455.017 |     ms |
|                      error rate |       asc_sort_tip_amount |       0 |      % |
|                  Min Throughput | desc_sort_passenger_count |       2 |  ops/s |
|                 Mean Throughput | desc_sort_passenger_count |       2 |  ops/s |
|               Median Throughput | desc_sort_passenger_count |       2 |  ops/s |
|                  Max Throughput | desc_sort_passenger_count |       2 |  ops/s |
|         50th percentile latency | desc_sort_passenger_count |  442.39 |     ms |
|         90th percentile latency | desc_sort_passenger_count |   448.9 |     ms |
|         99th percentile latency | desc_sort_passenger_count | 455.895 |     ms |
|        100th percentile latency | desc_sort_passenger_count |  458.85 |     ms |
|    50th percentile service time | desc_sort_passenger_count | 441.691 |     ms |
|    90th percentile service time | desc_sort_passenger_count | 447.881 |     ms |
|    99th percentile service time | desc_sort_passenger_count | 455.601 |     ms |
|   100th percentile service time | desc_sort_passenger_count | 457.506 |     ms |
|                      error rate | desc_sort_passenger_count |       0 |      % |
|                  Min Throughput |  asc_sort_passenger_count |       2 |  ops/s |
|                 Mean Throughput |  asc_sort_passenger_count |       2 |  ops/s |
|               Median Throughput |  asc_sort_passenger_count |       2 |  ops/s |
|                  Max Throughput |  asc_sort_passenger_count |       2 |  ops/s |
|         50th percentile latency |  asc_sort_passenger_count | 444.853 |     ms |
|         90th percentile latency |  asc_sort_passenger_count | 451.438 |     ms |
|         99th percentile latency |  asc_sort_passenger_count | 456.043 |     ms |
|        100th percentile latency |  asc_sort_passenger_count | 456.173 |     ms |
|    50th percentile service time |  asc_sort_passenger_count | 444.131 |     ms |
|    90th percentile service time |  asc_sort_passenger_count | 450.392 |     ms |
|    99th percentile service time |  asc_sort_passenger_count | 454.949 |     ms |
|   100th percentile service time |  asc_sort_passenger_count | 454.996 |     ms |
|                      error rate |  asc_sort_passenger_count |       0 |      % |

[WARNING] Error rate is 15.97 for operation 'index'. Please check the logs.

-------------------------------------
[INFO] ✅ SUCCESS (took 46034 seconds)
-------------------------------------
[ec2-user@ip-172-31-0-197 ~]$

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@coderabbitai
Copy link

coderabbitai bot commented Jan 12, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Comment @coderabbitai help to get the list of available commands and usage tips.

Signed-off-by: Michael Oviedo <[email protected]>
Signed-off-by: Michael Oviedo <[email protected]>
…earchDataBaseClient object for telemetry operations that run synchronously before the benchmark

Signed-off-by: Michael Oviedo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant