|
| 1 | +# Browser framework comparison |
| 2 | + |
| 3 | +This suite compares the checked-out Radi source with pinned React, Preact, Vue, |
| 4 | +and vanilla JavaScript implementations in real Chromium. It complements the |
| 5 | +older `deno task bench` microbenchmarks; those are not used for this comparison. |
| 6 | +Benchmark dependencies are isolated here and do not ship with Radi. |
| 7 | + |
| 8 | +## Run |
| 9 | + |
| 10 | +Requires Node 22 or later and a machine that can run Playwright Chromium. |
| 11 | + |
| 12 | +```sh |
| 13 | +cd benchmarks |
| 14 | +npm ci |
| 15 | +npx playwright install chromium |
| 16 | +npm run smoke |
| 17 | +npm run bench |
| 18 | +# More repetitions: |
| 19 | +npm run bench -- --samples=30 --warmup=5 |
| 20 | +``` |
| 21 | + |
| 22 | +On Linux, install the browser's system dependencies if Playwright requests them |
| 23 | +(`npx playwright install-deps chromium`). The runner builds production bundles, |
| 24 | +serves them on an ephemeral loopback port, launches headless Chromium, and closes |
| 25 | +the server and browser on completion or failure. No hosted service is needed. |
| 26 | +An incorrect DOM result or browser error fails the command; no new report is |
| 27 | +written for a failed run. Existing results must be checked by their timestamp. |
| 28 | + |
| 29 | +Outputs are `results/results.json` (all samples and metadata) and |
| 30 | +`results/report.md` (a comparison table). They are ignored by Git because they |
| 31 | +are machine-specific. Save both elsewhere before another run overwrites them. |
| 32 | +The smoke command validates functionality; one sample is not a performance claim. |
| 33 | + |
| 34 | +## What is measured |
| 35 | + |
| 36 | +| Dimension | Measurement | |
| 37 | +| --- | --- | |
| 38 | +| Delivery size | Minified application bundle bytes, gzip bytes, Brotli bytes | |
| 39 | +| Startup | Navigation start to imported runtime and committed empty table, in a fresh browser context | |
| 40 | +| Rendering and scale | Create 1,000 and 10,000 rows; replace 1,000 rows with new keys | |
| 41 | +| Updates | Update every tenth label; update all 1,000 labels | |
| 42 | +| Keyed reconciliation | Append 1,000 rows; swap distant rows; reverse; remove one row | |
| 43 | +| Events | Dispatch a button click and commit its selected row state | |
| 44 | +| Cleanup | Clear 1,000 rows; unmount a populated root | |
| 45 | +| Frame latency | Operation start through forced layout and two animation frames | |
| 46 | +| Memory | Post-GC JavaScript heap for empty, 10,000-row, and ten mount/unmount-cycle states | |
| 47 | + |
| 48 | +Timing results include the median, nearest-rank p95, min/max, and raw samples. |
| 49 | +The default is three discarded warmups and ten recorded samples per operation. |
| 50 | +Startup and memory each have ten recorded samples without discarded warmups. |
| 51 | +There is no pass/fail speed target: this is a comparative baseline, not an |
| 52 | +optimization or a performance gate. Correctness is the pass/fail criterion. |
| 53 | + |
| 54 | +## Fairness and interpretation |
| 55 | + |
| 56 | +- Every implementation renders the same table, cells, buttons, labels, selection |
| 57 | + classes, and CSS. Deterministic input arrays are prepared before timing. |
| 58 | + Runtime-specific bookkeeping is timed. Input data generation is not. |
| 59 | +- All lists use stable row IDs. Every sample checks every row's ID, text, order, |
| 60 | + selection, and retained DOM identity. Unmount must remove all root content. |
| 61 | + Setup and validation run outside measured intervals. |
| 62 | +- The adapters use public rendering APIs: React `createRoot` (waiting for a |
| 63 | + layout effect), Preact `render`, Vue's runtime `render` with VNodes, Radi's |
| 64 | + `createRoot`/`createList`/`update`, and a handwritten keyed DOM reconciler. |
| 65 | + React is not forced through `flushSync`. Vue uses render functions rather than |
| 66 | + compiled templates. These are equivalent rendering workloads, not equivalent |
| 67 | + scheduler implementations or each framework's most optimized application. |
| 68 | + Commit time is wall-clock latency, including React's scheduling delay and |
| 69 | + promise continuations; it must not be interpreted as pure CPU execution time. |
| 70 | +- Radi keeps a lookup map so retained keyed row closures read the latest immutable |
| 71 | + data. Its construction cost is included. No adapter uses virtualization, |
| 72 | + memoized row components, or framework-specific compiler optimizations. |
| 73 | +- Click latency uses a synthetic DOM click, including each framework's handler. |
| 74 | + It excludes physical input dispatch. Frame latency includes refresh pacing and |
| 75 | + forced layout; it is neither actual paint completion nor Core Web Vitals INP. |
| 76 | +- Production bundle sizes include the adapter and shared benchmark harness, |
| 77 | + not just library code. Each framework is bundled separately with the same |
| 78 | + esbuild configuration. Compression is calculated offline. |
| 79 | +- Startup uses local uncompressed HTTP and fresh contexts with no-store responses. |
| 80 | + It excludes browser process launch and does not simulate a slow network or |
| 81 | + fully cold OS/browser code caches. It is not LCP or a real-site load metric. |
| 82 | +- Memory is Chromium's `Runtime.getHeapUsage.usedSize` after explicit GC. It |
| 83 | + includes the harness and is not total process RAM or native DOM memory. The |
| 84 | + cycle measurement is a retention signal, not proof of absence of leaks. |
| 85 | +- Frameworks run sequentially. A timestamp-based rotation varies their order; |
| 86 | + the exact order is recorded. Run several complete comparisons on an idle |
| 87 | + machine to check order, thermal, JIT, GC, and background-load effects. Do not |
| 88 | + infer meaningful differences from tiny or overlapping ranges. |
| 89 | +- Metadata records the Radi Git revision and dirty state, package versions, |
| 90 | + browser version, CPU, OS, Node, viewport, and repetition counts. The lockfile |
| 91 | + fixes transitive dependencies. Run from a clean checkout for published results. |
| 92 | + |
| 93 | +This measures client runtime costs shared by all five implementations. It does |
| 94 | +not claim to exhaust every possible performance dimension. SSR, hydration, |
| 95 | +networked applications, mobile hardware, other browsers, compiler strategies, |
| 96 | +and application-specific deep component trees need separate workloads. Vanilla |
| 97 | +is a reference implementation, not a theoretical lower bound. |
| 98 | + |
| 99 | +Rendering semantics: [React DOM commit timing](https://react.dev/learn/manipulating-the-dom-with-refs), |
| 100 | +[Preact rendering](https://preactjs.com/guide/v10/api-reference/#render), and |
| 101 | +[Vue render functions](https://vuejs.org/api/render-function.html). |
0 commit comments