Skip to content

runtime-baseline.md

Latest commit

 

History

History
108 lines (84 loc) · 4.86 KB

runtime-baseline.md

File metadata and controls

108 lines (84 loc) · 4.86 KB
Raw

Runtime Baseline

Status: 1.6.0 Pingora-exit baseline

Fluxheim's 1.6.x line removes Pingora from the normal build graph. That work must be measurable before it becomes behavioral. The runtime baseline records what the current implementation builds, depends on, and costs before the replacement runtime ships.

Release Evidence

Release gates write baseline output to:

target/release-evidence/runtime-baseline/

The baseline directory is generated by:

scripts/capture-runtime-baseline.sh check
scripts/capture-runtime-baseline.sh release

check mode captures dependency graphs and Pingora presence for the official profiles without forcing a release build. release mode also builds the default release binary and records its size.

The generated files are:

File Purpose
metadata.env Package version, git commit, dirty flag, timestamp, host, rustc, and Cargo versions.
cargo-tree/*.txt Locked dependency trees for default, full, cache-edge, proxy-edge, load-balancer-edge, native-web-tls, native-web-openssl, PHP, and privacy profiles.
pingora-dependency-surface.tsv Pingora dependency proof report; this should be empty for every supported profile.
binary-size.tsv Default release binary size when release/build mode is used.
startup-time.tsv Milliseconds from process spawn to first successful local HTTP response.
process-idle.tsv Idle RSS, file-descriptor count, sample counts, and TLS availability.
http-latency.tsv Local static HTTP request timing samples.
cache-latency.tsv Local static-cache MISS/HIT timing samples and cache status headers.
load-balancer-latency.tsv Local round-robin route timing samples and observed backend labels.
keepalive-throughput.tsv Single-connection HTTP keep-alive throughput samples.
tls-handshake-latency.tsv HTTPS request timing samples that create a fresh TLS connection per sample when OpenSSL is available to generate the local certificate.

The stricter release policy is tracked in Pingora Dependency Exceptions and checked by:

scripts/validate-pingora-dependency-policy.sh check

That gate fails when a Pingora crate re-enters a supported profile, when an exception is stale, or when the current Fluxheim version has reached the exception's removal_target. cargo test also runs tests/pingora_dependency_policy.rs, which checks the same removal targets against Cargo.lock so deadline drift is visible outside the release gate. Maintainers can test an upcoming release bump before editing Cargo.toml with:

FLUXHEIM_PINGORA_POLICY_VERSION=1.6.2 scripts/validate-pingora-dependency-policy.sh check

Release mode invokes the performance baseline automatically. To skip it for a local emergency run, set:

FLUXHEIM_RUNTIME_BASELINE_PERFORMANCE=0 scripts/capture-runtime-baseline.sh release

Comparison Rules

The first 1.6.0 baseline is not a pass/fail performance benchmark. It is the reference point for later cutovers.

Later 1.6.x releases should compare against this baseline as follows:

  • dependency surface: once a release claims a Pingora crate is gone for a profile, that profile's cargo tree check must fail if the crate returns;
  • binary size: default and focused profile binaries should not grow without a release-note explanation;
  • behavior: smoke tests and fixture parity are stronger than raw speed numbers;
  • performance: startup time, idle memory, file-descriptor count, HTTP/1.1 latency, keep-alive throughput, cache HIT/MISS latency, load-balancer route timing, and TLS fresh-connection timing are captured by the 1.6.0 baseline; container image size should be added before container/runtime cutover work;
  • environment: every measurement must record enough host/toolchain information to explain differences between local developer machines, CI runners, and release builders.

Pingora Surface Policy

1.6.0 deliberately records Pingora dependencies instead of failing on them. The final 1.6.x release must make this a hard gate for every official profile: default, full, cache, proxy, load-balancer, PHP, privacy, FIPS/ISO candidates, and macOS developer builds.

Starting in 1.6.20, the baseline also records the native-web-tls and native-web-openssl proof profiles. Those are not production static-site profiles yet, but their web,tls-rustls and web,tls-openssl dependency trees should already emit no rows in pingora-dependency-surface.tsv.

Temporary compatibility shims are acceptable only while an old and new runtime path are being compared. New internal APIs should not introduce fresh Pingora types unless the same release documents the exception and target removal release. If a removal slips, the exception file must be updated explicitly with a new target and release-note rationale; the gate must not stay green by accident.