Skip to content

Comments

bench(compaction): bootstrap local Criterion harness + versioned JSON benchmark artifacts#585

Open
belveryin wants to merge 5 commits intomainfrom
feat/bench-compaction-local-criterion-json
Open

bench(compaction): bootstrap local Criterion harness + versioned JSON benchmark artifacts#585
belveryin wants to merge 5 commits intomainfrom
feat/bench-compaction-local-criterion-json

Conversation

@belveryin
Copy link
Collaborator

@belveryin belveryin commented Feb 9, 2026

Closes #584

Summary

This PR delivers a baseline compaction benchmark infrastructure for local filesystem runs.

What changed

  • Added Criterion bench target compaction_local.
  • Implemented two deterministic scenarios: read_baseline and read_post_compaction.
  • Added stable JSON artifact emission under target/tonbo-bench/ per run.
  • Added versioned artifact fields for machine diffing: schema_version, resolved config, scenario id/name, throughput/latency summary.
  • Refactored benchmark layout to Fusio-style split: benches/compaction/common.rs (shared logic) and benches/compaction_local.rs (scenario wiring).
  • Fixed wasm test-target regression by target-gating Criterion to non-wasm dev-dependencies.
  • Fixed key-space shaping bug by replacing rotate-based slot mapping with deterministic multiplicative hashing.
  • Kept benchmark docs minimal in README.md with run commands + pointers to benchmark source files; removed docs/benchmarks/compaction-local.md.

Scope guarantees

  • Criterion-based benchmark only.
  • Local filesystem backend only.
  • Env-only config (TONBO_COMPACTION_BENCH_*).
  • No CI workflow changes.
  • No rstest changes.
  • No production runtime behavior changes.

How to run locally

TONBO_COMPACTION_BENCH_INGEST_BATCHES=768 \
TONBO_COMPACTION_BENCH_ROWS_PER_BATCH=96 \
TONBO_COMPACTION_BENCH_KEY_SPACE=4096 \
TONBO_COMPACTION_BENCH_ARTIFACT_ITERATIONS=64 \
TONBO_COMPACTION_BENCH_CRITERION_SAMPLE_SIZE=30 \
TONBO_COMPACTION_BENCH_WAL_SYNC=always \
cargo bench --bench compaction_local

Example artifact

  • target/tonbo-bench/compaction_local-<run_id>.json
  • Key fields: schema_version, config, scenarios[]

@belveryin belveryin requested a review from ethe February 9, 2026 13:37
@belveryin belveryin marked this pull request as ready for review February 9, 2026 14:30
Copy link
Member

@ethe ethe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The overall direction looks good. The benchmark structure and versioned artifact output are useful. I do see one stability issue that should be fixed before merge.


impl ResolvedConfig {
pub(crate) fn from_env() -> Result<Self, BenchError> {
let ingest_batches = env_usize(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A valid config can panic during scenario setup. TONBO_COMPACTION_BENCH_INGEST_BATCHES is parsed in here, but there is no > 0 validation. Later, latest_version_summary errors when list_versions(1) is empty (benches/ compaction/common.rs:429), and that bubbles into a panic at benches/compaction_local.rs:27.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Added explicit > 0 validation for TONBO_COMPACTION_BENCH_INGEST_BATCHES in benches/compaction/common.rs:102, so invalid config fails fast with BenchError::InvalidEnv.
  • Kept the downstream guard in latest_version_summary (benches/compaction/common.rs:435) as defense-in-depth for unexpected empty version lists.

@belveryin belveryin requested a review from ethe February 18, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bench(compaction): bootstrap Criterion harness + stable metrics schema/json output (local fs)

2 participants