Zeppelin Benchmark Workspace

This repo is the consolidation point for Zeppelin benchmark harnesses.

Layout

• benchmarks/httpbench - HTTP API scenarios, including turbopuffer comparison.
• benchmarks/ragbench - real-embedding recall, latency, GET, and byte evaluation.
• benchmarks/qpsbench - closed-loop QPS and latency evaluation on real embeddings.
• benchmarks/recall - synthetic production-path recall evaluator.
• benchmarks/corebench - Criterion microbenchmarks for CPU hot paths.
• benchlib - shared benchmark support code.
• adapters/vector-db-benchmark/zeppelin - Zeppelin adapter files for the external vector-db-benchmark framework.

Common Commands

cargo check --workspace
cargo run -p httpbench --bin httpbench_eval -- --config configs/httpbench/smoke.toml
cargo run -p ragbench --bin ragbench_eval -- --config configs/ragbench/smoke.toml
cargo run -p qpsbench --bin qpsbench_eval -- --config configs/qpsbench/smoke.toml
cargo run -p recallbench --bin recall_eval -- --config configs/recall/example.toml --seed-file /path/to/seed.toml
cargo bench -p corebench

The example configs use data/ as a relative dataset root. Override it with --data-dir <path> or a local config file when datasets live elsewhere.

The real storage benchmarks require MinIO/S3-compatible settings via the same environment variables used by Zeppelin tests: TEST_BACKEND=minio, TEST_S3_BUCKET, MINIO_ENDPOINT, MINIO_ACCESS_KEY, and MINIO_SECRET_KEY.

Hold-out assets stay outside this repository. Reference them by path/name/hash only; do not copy seed files, workload contents, generated hold-out data, or raw sealed values into this repo.