perf: bound streaming memory and offload command processing - #527
perf: bound streaming memory and offload command processing#527Mohamed Mansour (mohamedmansour) wants to merge 2 commits into
Conversation
Move owned CLI start/resume state and JSON command preparation onto the existing blocking renderer. Enforce transport chunk and idle pool capacity limits, and reuse bounded scratch for watcher hashing. Add regression tests, before/after benchmark cases, and documentation for the memory and throughput tradeoffs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CLI benchmark reproduction appendixThis is the transient, Linux x86_64 measurement harness used for the CLI tables in the PR description. It uses the real private response pipeline and existing test fixtures; it is not a new public benchmark API or part of the product build.
WEBUI_BENCH_ITERATIONS=180 WEBUI_BENCH_OUTPUT=before-1 \
taskset -c 8,9 ./before-test measured_cli_render_pipeline --nocapture --test-threads=1
WEBUI_BENCH_ITERATIONS=180 WEBUI_BENCH_OUTPUT=after-1 \
taskset -c 8,9 ./after-test measured_cli_render_pipeline --nocapture --test-threads=1The CSVs contain every timed response. For the PR's aggregate tables, pool all three files per variant, group by The Linux clock IDs are 2 for process CPU and 3 for current-thread CPU. The ticker phase is deliberately separate because its yielding spinner changes CPU consumption and scheduling. No TCP, full server, allocator counter, or timing assertion is involved. Full measurement harness// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.
#[allow(unsafe_code)]
mod pipeline_benchmark_clock {
#[repr(C)]
struct Timespec {
seconds: std::os::raw::c_long,
nanos: std::os::raw::c_long,
}
unsafe extern "C" {
fn clock_gettime(clock: std::os::raw::c_int, time: *mut Timespec) -> std::os::raw::c_int;
}
pub(super) fn nanos(clock: i32) -> u64 {
let mut time = Timespec { seconds: 0, nanos: 0 };
// SAFETY: Linux clock_gettime writes a valid repr(C) timespec to this live pointer.
assert_eq!(unsafe { clock_gettime(clock, &mut time) }, 0);
(time.seconds as u64) * 1_000_000_000 + time.nanos as u64
}
}
fn pipeline_benchmark_config() -> RenderConfig {
let mut fragments = vec![
WebUIFragment::raw("<html><head>"),
structural("head_start"),
structural("head_end"),
WebUIFragment::raw("</head><body>"),
structural("body_start"),
WebUIFragment::raw("<pre>"),
WebUIFragment::signal("rows.0.name", false),
WebUIFragment::raw("</pre>"),
WebUIFragment::boundary(0, "index.html", "content", None),
WebUIFragment::raw("<main>"),
WebUIFragment::signal("rows.0.name", false),
WebUIFragment::raw("</main>"),
WebUIFragment::boundary_end(0),
WebUIFragment::raw("<footer>"),
WebUIFragment::signal("rows.0.name", false),
WebUIFragment::raw("</footer>"),
structural("body_end"),
WebUIFragment::raw("</body></html>"),
];
fragments.shrink_to_fit();
let mut document = WebUIProtocol::new(HashMap::from([(
"index.html".to_owned(),
FragmentList { fragments, contains_boundary: true },
)]));
document.initial_state_strategy = webui_protocol::InitialStateStrategy::Components as i32;
render_config(document)
}
fn pipeline_benchmark_config_copy(source: &RenderConfig) -> RenderConfig {
RenderConfig {
protocol: Arc::clone(&source.protocol),
entry: source.entry.clone(),
route_path: source.route_path.clone(),
plugin: None,
body_inject: None,
chunk_pool: Arc::clone(&source.chunk_pool),
}
}
fn pipeline_benchmark_input(target_bytes: usize) -> (Vec<Bytes>, usize, usize) {
let row = serde_json::json!({
"name": "row",
"payload": "x".repeat(64),
"nested": { "enabled": true, "ids": [1, 2, 3, 4] }
});
let row_bytes = serde_json::to_vec(&row).unwrap().len() + 1;
let count = (target_bytes / row_bytes).max(1);
let mut state = serde_json::json!({
"rows": { "0": { "name": "start" }, "items": vec![row; count] }
});
let mut start = serde_json::to_vec(&serde_json::json!({
"type": "start", "version": VERSION, "state": state
})).unwrap();
state["rows"]["0"]["name"] = serde_json::json!("resume");
state["rows"]["items"][0]["name"] = serde_json::json!("changed");
let resume = serde_json::to_vec(&serde_json::json!({
"type": "resume",
"boundary": { "owner": "index.html", "name": "content" },
"state": state
})).unwrap();
assert!(start.len() < MAX_RECORD_BYTES && resume.len() < MAX_RECORD_BYTES);
start.push(b'\n');
start.extend_from_slice(&resume);
start.push(b'\n');
let total_bytes = start.len();
let chunks = start.chunks(16 * 1024).map(Bytes::copy_from_slice).collect();
(chunks, count, total_bytes)
}
fn pipeline_benchmark_percentile(values: impl Iterator<Item = f64>, percentile: usize) -> f64 {
let mut values: Vec<_> = values.collect();
values.sort_by(f64::total_cmp);
values[(values.len() - 1) * percentile / 100]
}
#[actix_web::test]
async fn measured_cli_render_pipeline() {
use std::cell::Cell;
use std::fmt::Write as _;
use std::rc::Rc;
use std::time::Instant;
let output_prefix = std::env::var("WEBUI_BENCH_OUTPUT").unwrap();
let measure_count = std::env::var("WEBUI_BENCH_ITERATIONS")
.ok().map(|value| value.parse::<usize>().unwrap()).unwrap_or(400);
let ticker_count = (measure_count / 2).max(1);
let mut raw = String::from(
"phase,case,iteration,wall_us,process_cpu_us,event_cpu_us,max_tick_gap_us,tick_count\n"
);
let config = pipeline_benchmark_config();
for (name, target) in [("small", 256), ("64k", 65_536), ("1m", 1_048_576)] {
let (chunks, rows, bytes) = pipeline_benchmark_input(target);
let response = render(
tokio_stream::iter(chunks.iter().cloned().map(Ok::<_, String>).collect::<Vec<_>>()),
pipeline_benchmark_config_copy(&config),
defaults(),
).await;
assert_eq!(response.status(), actix_web::http::StatusCode::OK);
let expected = to_bytes(response.into_body()).await.unwrap();
let html = std::str::from_utf8(&expected).unwrap();
assert!(html.contains("<pre>start</pre>"), "{html}");
assert!(html.contains("<main>resume</main>"), "{html}");
assert!(html.contains(",4,0,{}]"), "{html}");
assert!(expected.len() < 4096, "output must remain small");
std::fs::write(format!("{output_prefix}-{name}.html"), &expected).unwrap();
println!("CASE name={name} rows={rows} input_bytes={bytes} output_bytes={}", expected.len());
for phase in ["latency", "ticker"] {
let stop = Rc::new(Cell::new(false));
let max_gap = Rc::new(Cell::new(0_u64));
let ticks = Rc::new(Cell::new(0_u64));
let ticker = if phase == "ticker" {
let stop = Rc::clone(&stop);
let gap = Rc::clone(&max_gap);
let ticks = Rc::clone(&ticks);
Some(actix_web::rt::spawn(async move {
let mut last = Instant::now();
while !stop.get() {
tokio::task::yield_now().await;
let now = Instant::now();
gap.set(gap.get().max(now.duration_since(last).as_nanos() as u64));
ticks.set(ticks.get() + 1);
last = now;
}
}))
} else {
None
};
let count = if phase == "latency" { measure_count } else { ticker_count };
let warmups = if measure_count > 10 { 40 } else { 2 };
let mut samples = Vec::with_capacity(count);
for iteration in 0..warmups + count {
let stream = tokio_stream::iter(
chunks.iter().cloned().map(Ok::<_, String>).collect::<Vec<_>>()
);
let request_config = pipeline_benchmark_config_copy(&config);
let request_defaults = defaults();
tokio::task::yield_now().await;
max_gap.set(0);
let tick_start = ticks.get();
let cpu_start = pipeline_benchmark_clock::nanos(2);
let thread_start = pipeline_benchmark_clock::nanos(3);
let start = Instant::now();
let response = render(stream, request_config, request_defaults).await;
assert_eq!(response.status(), actix_web::http::StatusCode::OK);
let body = to_bytes(response.into_body()).await.unwrap();
let wall = start.elapsed().as_nanos() as f64 / 1000.0;
let thread_cpu = (pipeline_benchmark_clock::nanos(3) - thread_start) as f64 / 1000.0;
let cpu = (pipeline_benchmark_clock::nanos(2) - cpu_start) as f64 / 1000.0;
tokio::task::yield_now().await;
let gap = max_gap.get() as f64 / 1000.0;
let tick_count = ticks.get() - tick_start;
assert_eq!(body, expected, "output changed at {name}/{phase}/{iteration}");
if iteration >= warmups {
samples.push((wall, cpu, thread_cpu, gap));
writeln!(
raw, "{phase},{name},{},{wall:.3},{cpu:.3},{thread_cpu:.3},{gap:.3},{tick_count}",
iteration - warmups
).unwrap();
}
}
stop.set(true);
if let Some(ticker) = ticker { ticker.await.unwrap(); }
println!(
"BENCH phase={phase} case={name} n={count} wall_p50_us={:.3} wall_p95_us={:.3} cpu_p50_us={:.3} cpu_p95_us={:.3} event_cpu_p50_us={:.3} event_cpu_p95_us={:.3} tick_gap_p50_us={:.3} tick_gap_p95_us={:.3}",
pipeline_benchmark_percentile(samples.iter().map(|sample| sample.0), 50),
pipeline_benchmark_percentile(samples.iter().map(|sample| sample.0), 95),
pipeline_benchmark_percentile(samples.iter().map(|sample| sample.1), 50),
pipeline_benchmark_percentile(samples.iter().map(|sample| sample.1), 95),
pipeline_benchmark_percentile(samples.iter().map(|sample| sample.2), 50),
pipeline_benchmark_percentile(samples.iter().map(|sample| sample.2), 95),
pipeline_benchmark_percentile(samples.iter().map(|sample| sample.3), 50),
pipeline_benchmark_percentile(samples.iter().map(|sample| sample.3), 95),
);
}
}
std::fs::write(format!("{output_prefix}.csv"), raw).unwrap();
} |
Follow-up:
|
| Item | Value |
|---|---|
| WebUI | e274c8b80cb54ecc562ab4528f4ca4d93dd1d9a4, clean checkout |
| Benchmark repository | microsoft/webui-benchmarks at 1d83eddf5573cc28fe32b7cd56cdbc6bf98563cd, plus the one-line adapter correction below |
| Benchmark source digest | 22bf026652097229cb852f6cfa52f00f2ce96c8ca723fbb44d4285d7b530686e |
| Benchmark dirty state | Explicitly recorded as true; no metadata was hidden or falsified |
| Dependency mode | Supported local-source mode, WebUI 0.0.28; local handler/protocol/state/expressions and rebuilt npm packages/native addon |
| Hardware | Intel Core Ultra 9 285K, 24 logical CPUs, approximately 62.53 GiB RAM |
| OS | Linux x86_64, WSL2 6.6.114.1-microsoft-standard-WSL2 |
| Node / Bun / Deno | 24.19.0 / 1.4.0 / 2.9.6 |
| Rust / Cargo / pnpm | 1.98.0 / 1.98.0 / 11.5.3 |
| Browser | Headed Chromium 151.0.7922.34, WSL display |
| Full workflow | 2026-09-11 22:38:17-23:09:52 UTC, 31m 35s |
| Progressive report timestamp | 2026-09-11T22:49:49.677Z |
| Complete report timestamp | 2026-09-11T23:00:59.800Z |
The benchmark adapter supplied serialized JSON to the parsed-state API. Its sole local source correction preserves the existing serialization and selects the appropriate existing API:
--- todo/servers/rust-actix/src/lib.rs
+++ todo/servers/rust-actix/src/lib.rs
@@
- return match state.protocol.render_partial(
+ return match state.protocol.render_partial_json(
&state_json,
ENTRY_ID,
request_path,The adapter correction is uncommitted in the benchmark repository, not part of this framework PR. No workload, timing, dependency pins, or correctness rules were changed. Existing Rust workspace tests passed after the alignment.
Local-source setup used the benchmark repository's supported pnpm run webui:local -- --path <webui-checkout> mechanism. Cargo metadata resolved the local crates with source: null; all 170 npm distribution files matched their checkout outputs. The measured JS hosts reported the checkout's target/release/libwebui_node.so, not a released addon. The user's default Node version was not changed; the pinned version was selected per command.
| Measured artifact | SHA-256 |
|---|---|
| Actual checkout addon | e498cac4669cdc7466ba9353828630bf8d876160f72517f8e8534531cea9e6dc |
| CLI | e6ce8b6a46ea48d1a6c60179fb53b85cb8e46e6aff7d0abff261136a766c6367 |
| Rust benchmark server | ff4d5006845901d9f0547a2ebe75b1d2c61b94c7815c2a3837dd13c262d66ada |
The source-snapshot addon has a different hash (3ffd092636618f25c2a13c86796960d302c8c4cb6b95e94a4dde41a39a1f85d1) because the supported build first builds CLI+addon together, then builds the addon alone. Both exact hashes were reproduced from their corresponding Cargo feature-unification builds. The table identifies the actually observed measured binary.
Methodology
The existing pnpm run bench:official workflow was invoked unfiltered, with its standard settings and sequential execution:
pnpm run build
pnpm run bench:officialIts SSR steps were:
node todo/runner/cli.mjs --runtime-mode official --response-mode progressive \
--save-baseline official --allow-load-errors
node todo/runner/cli.mjs --runtime-mode official --response-mode complete \
--save-baseline official-complete --allow-load-errorsEach case uses 100 todos and all 32 structural variants, per-request timestamps, natural unpadded response bytes, and the canonical semantic/response contract. One server worker is pinned to CPU 0; two load workers drive 64 measured connections. Each row gets two 4-second saturation probes at 64 and 128 connections, the normal warmup/restart sequence, a 3-second measured-server warmup, and one 20-second measurement.
Peak RSS is server RSS inside the measured RPS window, sampled every 200 ms. It is not startup RSS, total machine memory, or browser memory. No measured workloads or builds were intentionally overlapped. This single-round-per-case methodology supplies no multi-round confidence interval.
All 32 rows have complete case coverage, available telemetry, metricsStatus: "ok", applied CPU affinity, and exactly one measured attempt. Raw report status is still "partial" because of overload/saturation warnings. A complete matrix is not the same as a clean publication result.
Progressive/native-streaming SSR: full matrix
Latency is milliseconds; RSS is MiB. Error and timeout counts overlap: do not add them. Every measured row recorded zero non-2xx responses.
| Case | Completed RPS | P50 ms | P95 ms | Peak load RSS MiB | Errors / timeouts | Load status |
|---|---|---|---|---|---|---|
| webui-rust-actix | 1,487.60 | 34.01 | 75.75 | 133.26 | 0 / 0 | ok |
| webui-shadowdom-rust-actix | 1,106.35 | 56.83 | 65.86 | 167.05 | 0 / 0 | ok |
| webui-fast-rust-actix | 607.25 | 102.71 | 119.80 | 163.50 | 0 / 0 | ok |
| webui-bun-serve | 325.50 | 168.37 | 333.73 | 279.14 | 0 / 0 | ok |
| webui-deno-serve | 510.50 | 125.27 | 192.84 | 419.55 | 0 / 0 | ok |
| webui-node-http | 480.05 | 125.88 | 161.28 | 679.93 | 0 / 0 | ok |
| sveltekit | 167.95 | 308.94 | 452.60 | 362.33 | 7 / 7 | overloaded |
| nuxt | 75.10 | 442.29 | 523.13 | 567.48 | 44 / 44 | overloaded |
| next | 128.80 | 451.78 | 631.94 | 463.72 | 0 / 0 | ok |
| astro | 60.25 | 834.24 | 1,071.33 | 482.25 | 14 / 14 | overloaded |
| vue | 106.60 | 374.70 | 430.37 | 385.50 | 20 / 20 | overloaded |
| lit | 8.35 | 1,348.25 | 2,207.70 | 383.63 | 82 / 82 | overloaded |
| preact | 283.75 | 170.79 | 238.57 | 350.50 | 0 / 0 | ok |
| react | 213.65 | 271.05 | 496.25 | 591.54 | 0 / 0 | ok |
| react-tanstack | 202.35 | 277.78 | 389.37 | 442.45 | 1 / 1 | overloaded |
| solid | 239.00 | 231.39 | 348.68 | 687.52 | 0 / 0 | ok |
168 measured errors, all also recorded as timeouts; six overloaded cases. Unsaturated-probe warnings: vue, react, webui-node-http, react-tanstack. The prescribed fixed measured connection count was not changed.
Complete-response SSR: full matrix
Same units and error interpretation. Every measured row recorded zero non-2xx responses.
| Case | Completed RPS | P50 ms | P95 ms | Peak load RSS MiB | Errors / timeouts | Load status |
|---|---|---|---|---|---|---|
| webui-rust-actix | 1,632.95 | 38.85 | 43.59 | 128.15 | 0 / 0 | ok |
| webui-shadowdom-rust-actix | 1,281.60 | 49.33 | 55.77 | 140.73 | 0 / 0 | ok |
| webui-fast-rust-actix | 1,009.95 | 63.12 | 67.94 | 130.26 | 0 / 0 | ok |
| webui-bun-serve | 642.40 | 97.76 | 115.07 | 162.79 | 0 / 0 | ok |
| webui-deno-serve | 606.80 | 103.24 | 126.11 | 291.87 | 0 / 0 | ok |
| webui-node-http | 607.10 | 99.16 | 133.88 | 398.80 | 0 / 0 | ok |
| sveltekit | 202.65 | 287.69 | 345.68 | 415.59 | 1 / 1 | overloaded |
| nuxt | 75.05 | 480.78 | 743.45 | 1,066.48 | 48 / 48 | overloaded |
| next | 142.40 | 414.92 | 493.24 | 452.96 | 0 / 0 | ok |
| astro | 71.50 | 736.11 | 901.41 | 520.98 | 8 / 8 | overloaded |
| vue | 226.50 | 246.41 | 334.95 | 422.65 | 0 / 0 | ok |
| lit | 74.25 | 470.69 | 538.80 | 359.53 | 44 / 44 | overloaded |
| preact | 313.70 | 190.52 | 227.84 | 373.32 | 0 / 0 | ok |
| react | 277.25 | 208.99 | 290.25 | 363.02 | 0 / 0 | ok |
| react-tanstack | 224.60 | 263.93 | 324.87 | 429.09 | 0 / 0 | ok |
| solid | 271.80 | 214.97 | 280.19 | 567.18 | 0 / 0 | ok |
101 measured errors, all also recorded as timeouts; four overloaded cases. Unsaturated-probe warnings: vue, lit, next, sveltekit. All 64 saturation probes across both modes had zero errors/timeouts, and every measured warmup had status ok.
Headed browser: attempted, but no publishable numeric report
The workflow ran its unchanged official command:
node todo/browser/cli.mjs --official --rounds 10 --headed \
--output results/ssr-todo-browser-outcomes-official.jsonIt used ten deterministic randomized rounds per product, fresh headed Chromium processes/contexts/pages, 1440x900 viewport, two unmeasured foreground animation frames, HTTPS/HTTP2, production hydration, and the canonical 29-step interaction sequence. Only one product server ran at a time.
All 160 rounds were attempted. Successful counts below are inferred from the runner's exhaustive final failure list, not from a saved numeric report.
| Product | Attempts | Inferred successful outcomes | Failures |
|---|---|---|---|
| webui-rust-actix | 10 | 10 | 0 |
| webui-shadowdom-rust-actix | 10 | 10 | 0 |
| webui-fast-rust-actix | 10 | 10 | 0 |
| webui-bun-serve | 10 | 10 | 0 |
| webui-deno-serve | 10 | 9 | 1 |
| webui-node-http | 10 | 10 | 0 |
| sveltekit | 10 | 10 | 0 |
| nuxt | 10 | 10 | 0 |
| next | 10 | 10 | 0 |
| astro | 10 | 10 | 0 |
| vue | 10 | 10 | 0 |
| lit | 10 | 0 | 10 |
| preact | 10 | 10 | 0 |
| react | 10 | 10 | 0 |
| react-tanstack | 10 | 10 | 0 |
| solid | 10 | 10 | 0 |
| Total | 160 | 149 | 11 |
All ten Lit rounds failed the timestamp/DOM correctness condition: lit did not render one request timestamp across 100 todos. Narrow diagnosis using the runner's own shadow-aware query found exactly 100 title elements in raw SSR HTML, but 300 title elements after hydration, including values outside the required title/timestamp form. The count remained 300 after two seconds. This is a post-hydration correctness discrepancy, not the Rust adapter compile issue.
Deno round 2 failed because its page/context/browser closed. Chromium logged GPU initialization failures and an X connection error. A separate, clearly labeled quick headed diagnostic subsequently passed Deno and reproduced the Lit failure. That diagnostic was not promoted into official evidence or substituted for a failed round.
The official browser runner throws before saving results if any round fails. Consequently no canonical LCP, FCP, hydration/readiness, interaction/navigation, JS transfer/heap, renderer-memory, or other numeric browser metrics survived this attempt, even for successful products. They are unavailable, not zero, and were not reconstructed.
Outputs, publication blockers, and limits
Raw reports remain in the local benchmark checkout:
| Output | Result |
|---|---|
results/ssr-todo-official.json |
Saved; 16 measured rows; schema-valid, warning-bearing report |
results/ssr-todo-official-complete.json |
Saved; 16 measured rows; schema-valid, warning-bearing report |
results/ssr-todo-browser-outcomes-official.json |
Not saved because official browser rounds failed |
results/benchmark-summary.json |
Not generated: required browser report is absent |
results/benchmark-summary-complete.json |
Not generated: exporter requires a clean benchmark source revision |
The complete-only export correctly rejected the uncommitted adapter correction. No clean-source flag was forged, diff hidden, summary fabricated, correctness test weakened, or browser product removed.
Full raw reports, logs, source diff, runtime/dependency provenance, measured binaries, and SHA-256 manifests are retained locally with this run's evidence. The earlier 13-row runs are preserved separately and are not merged into these tables.
These results characterize the current integration checkout, not specifically every changed path: the Rust benchmark adapter uses its own ChannelWriter, rather than webui::streaming::StreamingWriter, and does not benchmark the CLI NDJSON parser or watcher hashing. The focused before/after measurements in the PR description remain the evidence for those changes.
Remaining limitations: one measured SSR round per case/mode; no telemetry-overhead control run; disclosed load/saturation warnings and load-runner negative-duration timer warnings; and the suite's dedicated async data-boundary workload remains unimplemented (available-pending-benchmark-implementation). No older cross-host comparison or whole-server improvement claim is made.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Final result: keep #527 draft. None of the hard 4/16/64 KiB policies wins as a general default. The isolated pool fix in #532 bounds idle retention, but recurring oversized attributes have a real CPU/latency cost. The independent CLI and watcher changes remain #529 and #530. The reproducible benchmark source and instructions are now in this PR at e337eec9. This follow-up commit adds the harness only; it does not change production code. All measured cells, including controls and excluded initial HTTP results, are embedded below. Raw per-request JSONL, execution manifests, archived binaries, and machine-specific drivers remain local session artifacts, not public download links. StreamingWriter experiment resultsMerge recommendationDo not merge hard 4, 16, or 64 KiB as a general/default replacement for the
Setup and interpretationCommon code/dependencies: umbrella Rust 1.98.0, release profile with thin LTO, one codegen unit, panic=abort; Each cell has three separate process runs with rotated/interleaved variant The channel has four slots. A persistent OS consumer drains concurrently; The 4 KiB path preserves default construction. Larger targets use the existing Primary unpaced resultsPooled, concurrent transport; latency in microseconds:
The real SSR cell renders For pooled raw output, process CPU/request was 37.0, 646.4, 189.9, 63.6 us The large-value result is also present without pooling: raw response medians Full results: Equal-target controlsPooled, unpaced, identical configured target and pool bytes within each pair;
Thus a larger soft target already obtains most of the mixed/SSR throughput Slow consumersPacing uses cumulative bytes from first-body arrival, including the final For a pooled 1 MiB raw response at 8 MiB/s, completion medians were all
Those CPU figures include the paced consumer's wakeups; they must not be For real SSR at 8 MiB/s, completion stayed around 286.6-286.8 ms. First-body Actual HTTP confirmation and the socket confoundThe first HTTP run, retained in The benchmark-only HTTP server was then changed to Corrected HTTP, pooled raw 1 MiB, unpaced:
Hard 64's completion ranges overlap soft-default; there is no convincing These are local HTTP/1, non-TLS, one-worker server/client measurements -- not Allocation and memory accountingAccounting is performed separately without recording latency. It verifies The allocator counts
For warmed pooled raw output, allocation-call/request counts were Bounds exclude active producer buffers, a pending send, consumer-held chunks, RSS caution: the Python-launched accounting processes all reported the same Isolated pool-only result and publication caveatBoth sides use a 4 KiB soft target, 5120-byte pool buffers, 16 pool slots
Do not sell the small apparent wins as guaranteed improvements. The material Separate warmed accounting observed raw allocation calls 1->3, and attribute This tradeoff is compatible with merging an explicitly documented idle-retention Full measured-cell tablesTimes below are microseconds. Brackets show the min/max of three process estimates, not confidence intervals. Latency and first body are medians of process request medians; CPU is the median of process CPU/request means. Rate 0 means unpaced; 8,388,608 means 8 MiB/s. Target/pool sizes and output sizes are bytes. The primary target-sized pools and the isolated 5120-byte pool pair are separate experiments. There were 360 timed processes and 143,550 measured requests across all phases, including the 48 confounded initial HTTP processes that are excluded from the recommendation. No timed subprocesses overlapped each other or our builds. Five warmups per process are additional, not measured requests. The non-HTTP comparisons use the archived v1 harness; corrected HTTP uses v2 with TCP_NODELAY enabled and verified. The v2 change affects only HTTP setup/reporting, its regression coverage, and documentation. The public harness is v2. Source/binary fingerprints follow the tables; a fingerprint identifies the measured artifact, not a claim that local raw files are downloadable. Isolated pool-only: all 12 measured cells
Primary transport: all 60 measured cells
Equal-target controls: all 16 measured cells
Corrected HTTP with TCP_NODELAY: all 16 measured cells
Excluded initial HTTP: all 16 confounded cells; DO NOT rank variants using these
Full allocation-accounting tablesThese are separate untimed observations, not throughput measurements or live heap. Counts include growing realloc calls and harness allocations; requested bytes use realloc growth deltas. Cold and warmed results must not be conflated. Idle counts do not measure private Vec capacities. The launcher-associated RSS floor makes the primary accounting RSS unusable for fine-grained rankings, so it is not repeated as a comparative column. Primary warmed accounting: all 60 measured cells
Matched 5 KiB pool accounting: all 16 cold/warmed rows
All seven output hashes and semantic flush offsets
Dataset and measured binary fingerprints
v1 benchmark patch SHA-256: The CLI-only results and their memory uncertainty remain in #529. These transport measurements do not establish a CLI RSS improvement, and the separate external SSR/browser run is not a before/after comparison. |
1 participant
Draft: do not merge this combined change
The measurements justify splitting this work rather than merging the combined
patch. The independently reviewable CLI improvement is #529, and bounded
watcher hashing is #530. The isolated pool-retention fix is #532, now with
completed allocation/CPU measurements and explicit tradeoffs: recurring
1 MiB attributes are about 50% slower, and paced 1 MiB raw responses use
about 38% more combined producer/consumer CPU despite unchanged rate-limited
completion time.
The 4/16/64 KiB transport experiments are complete. None is recommended as
a new general default. Hard 64 KiB is the least costly bounded option, but
large pooled values still regress about 2-3 times in transport. Equal-target
soft controls obtain most of the mixed/SSR benefit without that hard-cap
penalty. The corrected local HTTP results do not establish a universal winner.
The follow-up harness commit is benchmark-only; it does not select a new
production policy. This PR still contains the original held hard-4-KiB proposal.
Completed transport/pool report: all 120 timed cells, 76 allocation-accounting
rows, controls, output hashes, and limitations.
The reproducible harness is published in benchmark-only commit
e337eec9.Initial Nagle/delayed-ACK-confounded HTTP rows are retained but excluded from
the recommendation; corrected HTTP explicitly enables and verifies TCP_NODELAY.
The hard-4-KiB transport change remains experimental because the recorded
large-single-write workload regresses approximately 34-45 times. This draft
retains the original code and complete measurement history; neither the
external SSR results nor the CLI-only measurements establish a net benefit
for the hard-chunking implementation.
Original combined summary
Apply the relevant lessons from astral-sh/uv#21372: keep synchronous processing together on an existing blocking worker, transfer ownership instead of copying large values, bound the bytes between producer and consumer, and reuse bounded scratch storage.
StreamingWriter::with_chunk_sizea hard byte limit for raw and attribute output, reject oversized idle pool returns, and retain consumer-owned recycling. Keep the tiny-write fast path and try a nonblocking send before waiting.Update production pool construction, examples, the living specification, public Rust integration/performance documentation, and benchmark documentation. Add bounded transport/tiny-write and watcher-hashing benchmarks, and register watcher hashing in
cargo xtask bench all.This is not a uniform throughput improvement. The isolated CLI processing benchmark improves substantially for large state, but enforcing 4 KiB transport chunks makes a single 1 MiB write roughly 34-45 times slower in the bounded concurrent microbenchmark because one oversized send becomes 256-257 sends. Watcher hashing also regresses for the 1 MiB and small-file-burst cases. These tradeoffs are included below, not hidden behind the faster cases. No whole-server speedup or process-RSS improvement is claimed.
External canonical suite follow-up
The requested local-source run of
microsoft/webui-benchmarksis reported in full here: all 16 SSR cases in both modes, latency/RPS/peak RSS, overload counts, source/runtime provenance, and browser coverage. The 160-round headed browser run had 11 failures (ten Lit correctness failures and one Deno display failure), so no official browser numeric report or homepage summary was produced. A necessary one-line benchmark-adapter API alignment remains uncommitted and is disclosed in the measured source metadata. These current-checkout integration results are not a before/after comparison.Behavioral and memory contracts
Veccapacity is bounded bymax_pool.max(1) * chunk_size. Oversized returns are dropped, not retained or shrunk. A buffer returns only after the finalBytesconsumer reference is released.Bytes::from_ownerstill allocates ownership metadata.StreamingSessionstill returns a complete semantic-stepVec, not fixed-size transport chunks.No public API, protocol field, or production dependency is added. Criterion is an existing workspace dependency added only to the dev-server's dev-dependencies.
Benchmark environment and scope
Baseline:
f43db65067d48581ed7f99acc4b8d68f201c8bb9.Recorded host: Intel Core Ultra 9 285K, x86_64 Linux
6.6.114.1-microsoft-standard-WSL2; Rust/Cargo 1.98.0; optimized release/bench builds with thin LTO. These are local measurements, not cross-platform guarantees. The Criterion measurements ran in a shared environment and are sensitive to load, cache state, and CPU frequency.All measured cases are reported below, including the repeated tiny-write run and the unchanged-handler comparison that originally motivated the ownership change. CLI P50/P95, transport Criterion estimates, and watcher Criterion medians are different statistics and are labeled accordingly.
1. Real CLI streaming pipeline: owned state plus worker-side parsing
Method
Measure actual
render()throughto_bytes()using prebuilt NDJSON and an in-memory backend stream with 16 KiB chunks. This is the CLI response pipeline, not TCP or a full-server benchmark. Start/resume each retain a largerowssubtree; resume changes that subtree. Components projection keeps rendered output small to isolate state processing.Three sequential before/after pairs were pinned to CPUs 8 and 9. Each run/case used 40 warmups and 180 timed latency responses: 540 timed responses per variant/case. A separate responsiveness phase used 40 warmups and 90 timed responses per run: 270 timed responses per variant/case.
Only the CLI owned-state/worker-processing patch differs between the measured source snapshots. Both use the original transport, dependencies, and watcher implementation. The later test-helper pool-size alignment was excluded. The two optimizations were not independently ablated.
Input serialization, configuration construction, stream-chunk clones, output-byte comparison, and CSV writing are outside the timing window. Process and current-event-thread CPU use Linux
CLOCK_PROCESS_CPUTIME_IDandCLOCK_THREAD_CPUTIME_ID.Each individual record is below the 2 MB cap. All output bodies and checkpoints matched byte-for-byte across every case, variant, warmup, and measured response.
Latency and CPU
All entries are microseconds, median / P95, pooled across the three runs. P50 is the sample median; P95 is sorted sample index
floor((n - 1) * 0.95).For the 1 MiB case, pooled median wall latency falls from about 39.60 ms to 21.58 ms, and median event-thread CPU falls from about 11.95 ms to 0.379 ms. This supports the combined ownership/offloading change for this workload, not a general end-to-end server throughput claim.
Cooperative event-loop responsiveness
The separate ticker phase runs a continuously yielding task on the current thread. Each response records its maximum inter-tick gap. Entries below are microseconds, median / P95 of those per-response maxima; they are not percentiles of all individual ticks.
Per-pair median changes provide an additional view of run variability. Negative values mean lower latency or shorter gaps.
Full ticker-phase wall/CPU measurements (spinner-affected; not throughput evidence)
The spinner changes scheduling and contributes CPU usage. Do not substitute this table for the no-spinner latency/CPU table above. All entries are microseconds, median / P95.
RSS observations: inconclusive
These high-water marks include setup, warmups, the ticker phase, and allocator retention. They establish neither an improvement nor a regression in request memory. No CLI allocation-count measurement was collected.
Reproduction and artifact identity
The Linux-only measurement harness and reproduction steps are provided in the reproduction appendix so the transient CLI benchmark is reproducible without adding a production benchmark API.
Build isolated baseline snapshots, apply only the CLI patch to the after snapshot, and include the identical harness inside both existing
streaming_api::testsmodules. Build optimized CLI test executables withcargo test --release -p microsoft-webui-cli --no-run.When sharing a Cargo target across archived snapshots, force a CLI rebuild with
cargo clean --release -p microsoft-webui-clibefore building the second variant: same-mtime snapshots can otherwise be treated as fresh. Separate target directories also avoid this hazard. Distinct executable hashes confirmed the measured variants.Repeat for pairs 2 and 3. Pool raw per-response CSV samples by phase/case/variant using the percentile definition above; compute per-pair medians separately.
8e492c406cd76e9af07a03a1b6bd9ad1a2685190646975a1052e0896665e7cf547348210a1432b33dc2d856f1d237a93bbaa1e98b581fef95c66f3b3dc6836e2263e3ada3df1757302f55546644a1e8864a7513310037e1e090a25e3b58df260Executable hashes identify the measured binaries; they are not a promise of reproducible binary hashes on another toolchain/path.
2. Bounded streaming transport
Method
Extend the existing Criterion harness before implementing the transport changes, then run the same cases before/after. Optimized builds; 20 samples, 500 ms warmup, 1 s target measurement.
transportuses a persistent concurrent consumer and a four-slot channel. Timing includes backpressure and acknowledgment of final consumption, but excludes thread startup.transport_tinyremoves cross-thread scheduling to isolate small writes.These are Criterion point estimates with their printed confidence intervals, not extracted P50 values or measured CPU time. Short runs show shared-host/frequency variability, so both after tiny-write runs are retained.
Concurrent transport
Entries are microseconds, point estimate [lower, upper].
Tiny writes
Entries are nanoseconds, point estimate [lower, upper].
Interpretation and memory bounds
The old 1 MiB raw write emitted one giant chunk; the new hard default maximum emits 256 chunks, or 257 for the quoted-attribute case. The large-single-value slowdown is a real bounded-memory/backpressure tradeoff. Hosts can choose a larger bounded maximum through the existing
with_chunk_sizesetting when throughput matters more than a small queued-byte ceiling.The default unpooled active-buffer capacity request falls from 5,120 to 4,096 bytes (-20%). The queue payload is now bounded by 16 KiB. A 1 MiB-capacity return to a 4 KiB pool is rejected instead of retained; idle capacity obeys the formula above. These are source-derived buffer/queue bounds, not measured RSS or complete allocation elimination.
The older
writer_pathsgroup and resource example buffer an entire response before draining their channels. They are not evidence of slow-consumer memory bounds.Reproduction
Use the same new benchmark harness against baseline and changed implementations, retaining identical benchmark settings.
3. Watcher hashing
Method
Optimized Criterion builds, 50 samples, 1 s warmup, 3 s target measurement. Baseline runs the original metadata +
fs::read+DefaultHasher::writeimplementation through the same private module/harness. After runs the incremental 8 KiB implementation.File opening and metadata are timed; fixture creation is not. These are hot-page-cache microbenchmarks in a shared environment, not end-to-end rebuild measurements. The table uses extracted median/P50 estimates and median 95% confidence intervals, not Criterion's printed regression/mean estimates.
The 1 MiB file and small-file burst regress in this run. Additional bounded read syscalls are a real potential CPU/latency cost; this is not a claim of uniformly faster hashing.
Allocation/storage change
These are source-derived, not allocator/RSS measurements.
Vecallocation per hashPath, event, and hash-map allocations remain. Allocators can retain freed buffers, so these bounds must not be presented as process-RSS reductions.
Reproduction
Use identical fixtures and the private hashing-module benchmark entry point for both implementations.
Read the
median.point_estimateandmedian.confidence_intervalvalues from each case's Criterionestimates.json; convert nanoseconds to microseconds. The PR adds this benchmark tocargo xtask bench all.4. Existing handler API comparison: context only
This benchmark ran on the unchanged baseline before the CLI modifications. It compares existing APIs; it is not a before/after result for this patch, and the owned-state and parsing-placement effects are not independently isolated by it.
128 rows, eight boundaries; 100 Criterion samples, 3 s warmup, 5 s target measurement. Buffered and streaming output sizes differ. The three streaming modes share the streaming output shape.
Coverage and validation
Regression coverage includes raw-record ownership and channel backpressure; worker-side defaults and initial command errors; parse failure while a backend stalls; exact transport byte caps and UTF-8 splits; attributes across chunk boundaries; timeout/disconnect propagation and cached flush errors; pool capacity and final-consumer lifetime; slow consumers; whole-file digest equivalence; arbitrary short/interrupted reads; exact-cap and growing files; and retry/recreation after failed reads.
xtaskbenchmark registry testscargo xtask checkThe complete gate included license headers, formatting, production Clippy, protobuf drift, dependency audit, workspace tests/builds, WASM, examples, benchmark smoke, and docs. A session-local official
protocwas supplied throughPROTOCbecause the system executable was absent; no repository toolchain workaround was committed.Exploratory CLI test-target Clippy also exposed pre-existing
unwrap/expectlint debt outside the changed streaming module. That stricter invocation is not claimed to pass; the repository's standard production-Clippy and workspace-test gate passes.