22
33Date: 2026-03-30
44Source metrics: ` reports/launch_8k_batch_validation_2026-03-28.md `
5+ Latest stress parity run: 2026-03-30 (1024/512/5)
56
67## 1) GitHub Release
78
@@ -31,10 +32,18 @@ What ships:
3132 - Decode delta: -0.53% .. +1.58%
3233 - Latency delta: -1.33% .. +0.48%
3334
35+ Latest long-output stress parity (1024/512/5):
36+ - 4k baseline: latency 32248 ms, throughput 15.85 tok/s, decode 17.21 tok/s, TTFT 2393 ms
37+ - 8k custom: latency 32873 ms, throughput 15.54 tok/s, decode 16.87 tok/s, TTFT 2408 ms
38+ - Delta (8k vs 4k): latency +1.94%, throughput -1.96%, decode -1.98%, TTFT +0.63%
39+
3440Functional context gate:
3541- 8k build passes >4k retrieval prompt checks
3642- 4k baseline overflows at 5813 prompt tokens (context window size 4096)
3743
44+ Stability note:
45+ - In long-output stress mode, one exit(1) runtime event was observed and auto-recovered by unload/reload retry; run completed.
46+
3847Important limitation:
3948- Browser WebGPU does not provide exact live GPU VRAM counters. VRAM numbers are model metadata/proxy signals.
4049
@@ -48,6 +57,8 @@ Report:
4857
4958Hosted preset artifacts:
5059- https://huggingface.co/Ar5en1c/Qwen2.5-1.5B-Instruct-q4f16_1-MLC-ctx8192
60+ Live app:
61+ - https://ar5en1c.github.io/webllm-bench/bench.html
5162```
5263
5364## 2) X Thread (Exact Copy)
@@ -70,7 +81,7 @@ New in this release: one-click hosted 8k preset in the UI.
7081
7182### Post 2
7283```
73- 8k validation on fixed profile (1024 prompt / 128 output / 10 iterations ), using 8 exported runs:
84+ 8k validation on fixed profile (1024/ 128/10 ), using 8 exported runs:
7485
7586Median delta (8k custom vs 4k baseline):
7687- Decode TPS: +0.11%
@@ -90,22 +101,34 @@ Interpretation: parity band, no material regression on this profile.
90101
91102### Post 4
92103```
104+ Latest long-output stress parity (1024/512/5):
105+ - 4k throughput: 15.85 tok/s
106+ - 8k throughput: 15.54 tok/s
107+ - delta: -1.96%
108+
109+ Other deltas:
110+ - latency: +1.94%
111+ - decode: -1.98%
112+ ```
113+
114+ ### Post 5
115+ ```
93116Functional gate result:
94117- ctx8192 build handles >4k retrieval prompts
95118- ctx4096 baseline fails at 5813 prompt tokens with context overflow
96119
97120So this is not only a config change; it is validated long-context behavior.
98121```
99122
100- ### Post 5
123+ ### Post 6
101124```
102125Limitation (explicit):
103126Browser WebGPU does not expose exact live VRAM usage counters.
104127
105128Published VRAM numbers are model metadata / proxy signals, not hardware telemetry.
106129```
107130
108- ### Post 6
131+ ### Post 7
109132```
110133Everything is reproducible from repo artifacts:
111134- benchmark JSON exports
@@ -118,6 +141,15 @@ npm run report:8k:batch
118141npm run launch:draft
119142```
120143
144+ ### Post 8
145+ ```
146+ Links:
147+ - Live: https://ar5en1c.github.io/webllm-bench/bench.html
148+ - Repo: https://github.com/Ar5en1c/webllm-bench
149+ - 8k model: https://huggingface.co/Ar5en1c/Qwen2.5-1.5B-Instruct-q4f16_1-MLC-ctx8192
150+ - 8k workflow package: https://github.com/Ar5en1c/qwen2.5-1.5b-ctx8192-mlc
151+ ```
152+
121153## 3) LinkedIn Post (Exact Copy)
122154
123155```
@@ -137,10 +169,23 @@ Range across runs:
137169- Decode delta: -0.53% to +1.58%
138170- Latency delta: -1.33% to +0.48%
139171
172+ Latest long-output stress parity run (1024 prompt, 512 output, 5 iterations):
173+ - 4k baseline throughput: 15.85 tok/s
174+ - 8k custom throughput: 15.54 tok/s
175+ - Delta: -1.96%
176+
177+ Other stress deltas:
178+ - Latency: +1.94%
179+ - Decode: -1.98%
180+ - TTFT: +0.63%
181+
140182Functional context check also passed:
141183- ctx8192 build handled prompts beyond 4k context
142184- ctx4096 baseline overflowed at 5813 prompt tokens
143185
186+ Stress stability note:
187+ - One exit(1) runtime event occurred and was auto-recovered by unload/reload retry; run completed.
188+
144189One limitation to state clearly: browser WebGPU does not expose exact live VRAM counters, so VRAM values are metadata/proxy signals.
145190
146191All claims are backed by exported JSON reports and a reproducible validation script in the repository.
0 commit comments