Skip to content

Commit 64562be

Browse files
authored
test(bench): add reproducible framework comparisons
1 parent 1d85a10 commit 64562be

12 files changed

Lines changed: 1623 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -405,6 +405,9 @@ import { createElement, Fragment } from 'radi';
405405
406406
## Contributing
407407
408+
For reproducible browser performance comparisons with React, Preact, Vue, and
409+
vanilla JavaScript, see [the comparison benchmark suite](benchmarks/README.md).
410+
408411
1. Fork and clone
409412
2. Install dependencies
410413
3. Run tests: `deno task test`

‎benchmarks/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
results/

‎benchmarks/README.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Browser framework comparison
2+
3+
This suite compares the checked-out Radi source with pinned React, Preact, Vue,
4+
and vanilla JavaScript implementations in real Chromium. It complements the
5+
older `deno task bench` microbenchmarks; those are not used for this comparison.
6+
Benchmark dependencies are isolated here and do not ship with Radi.
7+
8+
## Run
9+
10+
Requires Node 22 or later and a machine that can run Playwright Chromium.
11+
12+
```sh
13+
cd benchmarks
14+
npm ci
15+
npx playwright install chromium
16+
npm run smoke
17+
npm run bench
18+
# More repetitions:
19+
npm run bench -- --samples=30 --warmup=5
20+
```
21+
22+
On Linux, install the browser's system dependencies if Playwright requests them
23+
(`npx playwright install-deps chromium`). The runner builds production bundles,
24+
serves them on an ephemeral loopback port, launches headless Chromium, and closes
25+
the server and browser on completion or failure. No hosted service is needed.
26+
An incorrect DOM result or browser error fails the command; no new report is
27+
written for a failed run. Existing results must be checked by their timestamp.
28+
29+
Outputs are `results/results.json` (all samples and metadata) and
30+
`results/report.md` (a comparison table). They are ignored by Git because they
31+
are machine-specific. Save both elsewhere before another run overwrites them.
32+
The smoke command validates functionality; one sample is not a performance claim.
33+
34+
## What is measured
35+
36+
| Dimension | Measurement |
37+
| --- | --- |
38+
| Delivery size | Minified application bundle bytes, gzip bytes, Brotli bytes |
39+
| Startup | Navigation start to imported runtime and committed empty table, in a fresh browser context |
40+
| Rendering and scale | Create 1,000 and 10,000 rows; replace 1,000 rows with new keys |
41+
| Updates | Update every tenth label; update all 1,000 labels |
42+
| Keyed reconciliation | Append 1,000 rows; swap distant rows; reverse; remove one row |
43+
| Events | Dispatch a button click and commit its selected row state |
44+
| Cleanup | Clear 1,000 rows; unmount a populated root |
45+
| Frame latency | Operation start through forced layout and two animation frames |
46+
| Memory | Post-GC JavaScript heap for empty, 10,000-row, and ten mount/unmount-cycle states |
47+
48+
Timing results include the median, nearest-rank p95, min/max, and raw samples.
49+
The default is three discarded warmups and ten recorded samples per operation.
50+
Startup and memory each have ten recorded samples without discarded warmups.
51+
There is no pass/fail speed target: this is a comparative baseline, not an
52+
optimization or a performance gate. Correctness is the pass/fail criterion.
53+
54+
## Fairness and interpretation
55+
56+
- Every implementation renders the same table, cells, buttons, labels, selection
57+
classes, and CSS. Deterministic input arrays are prepared before timing.
58+
Runtime-specific bookkeeping is timed. Input data generation is not.
59+
- All lists use stable row IDs. Every sample checks every row's ID, text, order,
60+
selection, and retained DOM identity. Unmount must remove all root content.
61+
Setup and validation run outside measured intervals.
62+
- The adapters use public rendering APIs: React `createRoot` (waiting for a
63+
layout effect), Preact `render`, Vue's runtime `render` with VNodes, Radi's
64+
`createRoot`/`createList`/`update`, and a handwritten keyed DOM reconciler.
65+
React is not forced through `flushSync`. Vue uses render functions rather than
66+
compiled templates. These are equivalent rendering workloads, not equivalent
67+
scheduler implementations or each framework's most optimized application.
68+
Commit time is wall-clock latency, including React's scheduling delay and
69+
promise continuations; it must not be interpreted as pure CPU execution time.
70+
- Radi keeps a lookup map so retained keyed row closures read the latest immutable
71+
data. Its construction cost is included. No adapter uses virtualization,
72+
memoized row components, or framework-specific compiler optimizations.
73+
- Click latency uses a synthetic DOM click, including each framework's handler.
74+
It excludes physical input dispatch. Frame latency includes refresh pacing and
75+
forced layout; it is neither actual paint completion nor Core Web Vitals INP.
76+
- Production bundle sizes include the adapter and shared benchmark harness,
77+
not just library code. Each framework is bundled separately with the same
78+
esbuild configuration. Compression is calculated offline.
79+
- Startup uses local uncompressed HTTP and fresh contexts with no-store responses.
80+
It excludes browser process launch and does not simulate a slow network or
81+
fully cold OS/browser code caches. It is not LCP or a real-site load metric.
82+
- Memory is Chromium's `Runtime.getHeapUsage.usedSize` after explicit GC. It
83+
includes the harness and is not total process RAM or native DOM memory. The
84+
cycle measurement is a retention signal, not proof of absence of leaks.
85+
- Frameworks run sequentially. A timestamp-based rotation varies their order;
86+
the exact order is recorded. Run several complete comparisons on an idle
87+
machine to check order, thermal, JIT, GC, and background-load effects. Do not
88+
infer meaningful differences from tiny or overlapping ranges.
89+
- Metadata records the Radi Git revision and dirty state, package versions,
90+
browser version, CPU, OS, Node, viewport, and repetition counts. The lockfile
91+
fixes transitive dependencies. Run from a clean checkout for published results.
92+
93+
This measures client runtime costs shared by all five implementations. It does
94+
not claim to exhaust every possible performance dimension. SSR, hydration,
95+
networked applications, mobile hardware, other browsers, compiler strategies,
96+
and application-specific deep component trees need separate workloads. Vanilla
97+
is a reference implementation, not a theoretical lower bound.
98+
99+
Rendering semantics: [React DOM commit timing](https://react.dev/learn/manipulating-the-dom-with-refs),
100+
[Preact rendering](https://preactjs.com/guide/v10/api-reference/#render), and
101+
[Vue render functions](https://vuejs.org/api/render-function.html).

‎benchmarks/adapters/preact.js‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
import { h, render } from 'preact';
2+
3+
export function mount(target, select) {
4+
return {
5+
set(rows, selected) {
6+
render(
7+
h(
8+
'table',
9+
null,
10+
h(
11+
'tbody',
12+
null,
13+
rows.map((row) =>
14+
h(
15+
'tr',
16+
{
17+
key: row.id,
18+
'data-id': row.id,
19+
className: selected === row.id ? 'selected' : '',
20+
},
21+
h('td', null, row.id),
22+
h(
23+
'td',
24+
null,
25+
h('button', { onClick: () => select(row.id) }, row.label),
26+
),
27+
)
28+
),
29+
),
30+
),
31+
target,
32+
);
33+
},
34+
unmount: () => render(null, target),
35+
};
36+
}

‎benchmarks/adapters/radi.js‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
import {
2+
createElement as h,
3+
createList,
4+
createRoot,
5+
update,
6+
} from '../../src/client.ts';
7+
8+
export function mount(target, select) {
9+
let rows = [];
10+
let byId = new Map();
11+
let selected = null;
12+
const root = createRoot(target);
13+
const table = h(
14+
'table',
15+
null,
16+
h('tbody', null, () =>
17+
createList((key) =>
18+
rows.map((row) =>
19+
key(() =>
20+
h(
21+
'tr',
22+
{
23+
'data-id': row.id,
24+
className: () => selected === row.id ? 'selected' : '',
25+
},
26+
h('td', null, row.id),
27+
h(
28+
'td',
29+
null,
30+
h(
31+
'button',
32+
{ onclick: () => select(row.id) },
33+
() => byId.get(row.id).label,
34+
),
35+
),
36+
), row.id)
37+
)
38+
)),
39+
);
40+
root.render(table);
41+
return {
42+
set(nextRows, nextSelected) {
43+
rows = nextRows;
44+
byId = new Map(rows.map((row) => [row.id, row]));
45+
selected = nextSelected;
46+
update(table);
47+
},
48+
unmount: () => root.unmount(),
49+
};
50+
}

‎benchmarks/adapters/react.js‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
import { createElement as h, useLayoutEffect } from 'react';
2+
import { createRoot } from 'react-dom/client';
3+
4+
export function mount(target, select) {
5+
const root = createRoot(target);
6+
function Table({ rows, selected, done }) {
7+
useLayoutEffect(done);
8+
return h(
9+
'table',
10+
null,
11+
h(
12+
'tbody',
13+
null,
14+
rows.map((row) =>
15+
h(
16+
'tr',
17+
{
18+
key: row.id,
19+
'data-id': row.id,
20+
className: selected === row.id ? 'selected' : '',
21+
},
22+
h('td', null, row.id),
23+
h(
24+
'td',
25+
null,
26+
h('button', { onClick: () => select(row.id) }, row.label),
27+
),
28+
)
29+
),
30+
),
31+
);
32+
}
33+
return {
34+
set: (rows, selected) =>
35+
new Promise((resolve) => {
36+
root.render(h(Table, {
37+
rows,
38+
selected,
39+
done: () => {
40+
resolve();
41+
},
42+
}));
43+
}),
44+
unmount: () => root.unmount(),
45+
};
46+
}

‎benchmarks/adapters/vanilla.js‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
export function mount(target, select) {
2+
const table = document.createElement('table');
3+
const body = table.appendChild(document.createElement('tbody'));
4+
const nodes = new Map();
5+
target.append(table);
6+
return {
7+
set(rows, selected) {
8+
const ids = new Set(rows.map((row) => row.id));
9+
for (const [id, node] of nodes) {
10+
if (!ids.has(id)) {
11+
node.remove();
12+
nodes.delete(id);
13+
}
14+
}
15+
let cursor = body.firstChild;
16+
for (const row of rows) {
17+
let node = nodes.get(row.id);
18+
if (!node) {
19+
node = document.createElement('tr');
20+
node.dataset.id = row.id;
21+
node.appendChild(document.createElement('td')).textContent = row.id;
22+
const button = node.appendChild(document.createElement('td'))
23+
.appendChild(document.createElement('button'));
24+
button.onclick = () => select(row.id);
25+
nodes.set(row.id, node);
26+
}
27+
const button = node.lastChild.firstChild;
28+
if (button.textContent !== row.label) button.textContent = row.label;
29+
const className = selected === row.id ? 'selected' : '';
30+
if (node.className !== className) node.className = className;
31+
if (node !== cursor) body.insertBefore(node, cursor);
32+
cursor = node.nextSibling;
33+
}
34+
},
35+
unmount() {
36+
table.remove();
37+
nodes.clear();
38+
},
39+
};
40+
}

‎benchmarks/adapters/vue.js‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import { h, render } from 'vue';
2+
3+
export function mount(target, select) {
4+
return {
5+
set(rows, selected) {
6+
render(
7+
h('table', null, [h(
8+
'tbody',
9+
null,
10+
rows.map((row) =>
11+
h('tr', {
12+
key: row.id,
13+
'data-id': row.id,
14+
class: selected === row.id ? 'selected' : '',
15+
}, [
16+
h('td', null, String(row.id)),
17+
h('td', null, [
18+
h('button', { onClick: () => select(row.id) }, row.label),
19+
]),
20+
])
21+
),
22+
)]),
23+
target,
24+
);
25+
},
26+
unmount: () => render(null, target),
27+
};
28+
}

0 commit comments

Comments
 (0)