Skip to content

BASELINE.md

Latest commit

 

History

History
193 lines (140 loc) · 5.5 KB

BASELINE.md

File metadata and controls

193 lines (140 loc) · 5.5 KB
doc_id THR-PERF-007
doc_title Baseline Performance Metrics
doc_version 1.0.0
doc_date 2026-04-04
doc_status Draft
project thread_system
category PERF

Baseline Performance Metrics

SSOT: This document is the single source of truth for Baseline Performance Metrics.

Document Version: 1.0 Created: 2025-10-07 System: thread_system Purpose: Establish baseline performance metrics for regression detection

Overview

This document records baseline performance metrics for the thread_system. These metrics serve as reference points for detecting performance regressions during development.

Regression Threshold: <5% performance degradation is acceptable. Any regression >5% should be investigated and justified.

Test Environment

Hardware Specifications

  • CPU: To be recorded on first benchmark run
  • Cores: To be recorded on first benchmark run
  • RAM: To be recorded on first benchmark run
  • OS: macOS / Linux / Windows

Software Configuration

  • Compiler: Clang/GCC/MSVC (see CI workflow)
  • C++ Standard: C++20
  • Build Type: Release with optimizations
  • CMake Version: 3.16+

Benchmark Categories

1. Thread Pool Benchmarks

1.1 Task Submission Latency

Metric: Time to submit a task to the thread pool Test File: thread_pool_benchmarks/submission_bench.cpp

Statistic Target Value Unit Notes
Mean TBD μs Average submission time
Median TBD μs 50th percentile
P95 TBD μs 95th percentile
P99 TBD μs 99th percentile
Min TBD μs Best case
Max TBD μs Worst case

Status: ⏳ Awaiting initial benchmark run

1.2 Task Execution Throughput

Metric: Number of tasks executed per second Test File: thread_pool_benchmarks/throughput_bench.cpp

Statistic Target Value Unit Notes
Mean TBD tasks/s Average throughput
Median TBD tasks/s 50th percentile
P95 TBD tasks/s 95th percentile
P99 TBD tasks/s 99th percentile

Status: ⏳ Awaiting initial benchmark run

1.3 Thread Pool Scaling

Metric: Performance scaling with different thread counts Test File: thread_pool_benchmarks/scaling_bench.cpp

Thread Count Throughput (tasks/s) Efficiency (%) Notes
1 TBD 100% (baseline) Single thread
2 TBD TBD
4 TBD TBD
8 TBD TBD
16 TBD TBD

Status: ⏳ Awaiting initial benchmark run

2. Typed Thread Pool Benchmarks

2.1 Typed Task Submission

Metric: Overhead of type-safe task submission Test File: typed_thread_pool_benchmarks/typed_submission_bench.cpp

Statistic Target Value Unit Notes
Mean TBD μs Average submission time
Overhead vs untyped TBD % Compared to basic thread pool

Status: ⏳ Awaiting initial benchmark run

3. Thread Base Benchmarks

3.1 Thread Creation Overhead

Metric: Time to create and start a thread Test File: thread_base_benchmarks/creation_bench.cpp

Statistic Target Value Unit Notes
Mean TBD μs Average creation time
Median TBD μs 50th percentile

Status: ⏳ Awaiting initial benchmark run

Concurrent Access Benchmarks

4. Data Race Prevention

Metric: Performance impact of synchronization primitives Test File: data_race_benchmark.cpp

Test Case Baseline (ns) With Mutex (ns) With Atomic (ns) Overhead (%)
Single Writer TBD TBD TBD TBD
Multiple Readers TBD TBD TBD TBD
Mixed Access TBD TBD TBD TBD

Status: ⏳ Awaiting initial benchmark run

How to Run Benchmarks

Building Benchmarks

cd thread_system
cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCHMARKS=ON
cmake --build build --target benchmarks

Running All Benchmarks

cd build/benchmarks
./thread_pool_benchmarks
./typed_thread_pool_benchmarks
./thread_base_benchmarks

Recording Results

  1. Run each benchmark 10 times
  2. Record min, max, mean, median, p95, p99
  3. Update this document with actual values
  4. Commit updated BASELINE.md

Regression Detection

Automated Checks

The benchmarks.yml workflow runs benchmarks on every PR and compares results against this baseline.

Manual Review Process

  1. If regression >5% detected, review PR carefully
  2. Check for algorithmic changes that justify regression
  3. Update baseline if new optimization reduces performance intentionally (e.g., added thread safety)
  4. Document any approved regressions in PR description

Historical Changes

Date Version Change Impact Approved By
2025-10-07 1.0 Initial baseline document created N/A Initial setup

Notes

  • All benchmarks use Google Benchmark framework
  • Results may vary based on hardware and system load
  • For accurate comparisons, run benchmarks on same hardware
  • CI environment results are used as primary baseline
  • Performance improvements >10% should also be reviewed to ensure correctness

Status: 📝 Template created - awaiting initial benchmark data collection