Skip to content

tpch.md

Latest commit

 

History

History
76 lines (52 loc) · 2.57 KB

tpch.md

File metadata and controls

76 lines (52 loc) · 2.57 KB
Raw

TPC-H Benchmarks

TPC-H supplies the paper's end-to-end AS performance and DP utility workloads. The AS and DP experiments use separate standalone executables.

AS Performance

The main benchmark compares DuckDB, SIMD-AS at m=64, a naive sampling implementation, and a simple-hash control.

cmake --build build/release --target as_tpch_benchmark

build/release/extension/privacy/as_tpch_benchmark \
  --config benchmark/configs/utility/as_tpch_sf30.json --dry-run

build/release/extension/privacy/as_tpch_benchmark \
  --config benchmark/configs/utility/as_tpch_sf30.json

The runner generates the configured TPC-H database when it is absent. Handwritten query variants live in:

  • benchmark/tpch/tpch_as_queries/
  • benchmark/tpch/tpch_as_simple_hash_queries/
  • benchmark/tpch/tpch_naive_as_queries/

The variable-m experiment uses the same runner and database:

build/release/extension/privacy/as_tpch_benchmark \
  --config benchmark/configs/utility/as_tpch_sf30_m_sweep.json

Each query and mode receives one untimed warmup immediately before its measured executions.

DP Utility and Runtime

dp_benchmark_runner evaluates DuckDB, bounded DP, elastic DP, and SAA over Q01, Q05, Q06, Q14, and Q19.

cmake --build build/release --target dp_benchmark_runner

build/release/extension/privacy/dp_benchmark_runner \
  --config benchmark/configs/utility/dp_tpch_sf30_utility.json --dry-run

build/release/extension/privacy/dp_benchmark_runner \
  --config benchmark/configs/utility/dp_tpch_sf30_utility.json

The customer is the privacy unit. lineitem reaches it through orders. The runner first records private-query runtime, then computes exact references and diagnostics in an untimed pass. Timing rows are checkpointed after each dataset entry.

The Q01 AVG configurations isolate output columns and grouping choices:

build/release/extension/privacy/dp_benchmark_runner \
  --config benchmark/configs/utility/dp_tpch_sf30_q01_avg_columns.json

build/release/extension/privacy/dp_benchmark_runner \
  --config benchmark/configs/utility/dp_tpch_sf30_q01_avg_groups.json

Plot Results

Rscript --vanilla benchmark/tpch/plot_tpch_results.R \
  benchmark/results/utility/as_tpch_sf30.csv benchmark/results/figures

Rscript --vanilla benchmark/tpch/plot_tpch_as_m_sweep.R \
  benchmark/results/utility/as_tpch_sf30.csv \
  benchmark/results/utility/as_tpch_sf30_m_sweep.csv \
  benchmark/results/figures

See Utility evaluation for bound and metric definitions. See repro.md for every paper config and plotting command.