Skip to content

Commit a0adb0f

Browse files
theengineeartheengineear
committed
Improve performance tests.
Changes: * Attempt to test more iterations. * Do better at right-sizing batches per animation frame. * Only measure batches to get around security issues with timestamps. * Show distribution of results for better relative analysis.
1 parent f920e80 commit a0adb0f

File tree

3 files changed

+549
-224
lines changed

3 files changed

+549
-224
lines changed

‎demo/performance/index.css

+61-4
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,74 @@
22
line-height: 20px;
33
}
44

5+
.test + .test {
6+
margin-top: 32px;
7+
}
8+
59
.label {
610
font-weight: bold;
711
}
812

913
.output {
10-
height: calc(20px * 3);
14+
width: 500px;
15+
height: 92px;
16+
}
17+
.output:empty::before {
18+
content: "Loading\2026";
19+
}
20+
21+
.distribution {
22+
height: 20px;
23+
display: flex;
24+
gap: 12px;
25+
font-family: monospace;
26+
}
27+
.distribution + .distribution {
28+
margin-top: 4px;
29+
}
30+
31+
.distribution .left {
32+
display: flex;
33+
}
34+
35+
.distribution .right {
36+
flex-grow: 1;
37+
box-shadow: inset 0 0 0 1px black;
38+
}
39+
40+
.distribution .percentile {
41+
position: absolute;
42+
z-index: 0;
43+
color: lightgray;
44+
transform: translateX(-50%);
45+
}
46+
47+
.distribution .percentile.median {
48+
z-index: 1;
49+
color: black;
50+
}
51+
52+
.distribution .min {
53+
position: absolute;
54+
top: -24px;
55+
left: 4px;
56+
}
57+
.distribution .min::after {
58+
content: " (min)";
59+
}
60+
61+
.distribution .max {
62+
position: absolute;
63+
top: -24px;
64+
right: 4px;
65+
}
66+
.distribution .max::before {
67+
content: "(max) ";
1168
}
1269

13-
#fixture {
14-
height: calc(20px * 9);
15-
color: gray;
70+
.distribution:not(:first-child) .min,
71+
.distribution:not(:first-child) .max {
72+
display: none;
1673
}
1774

1875
p {

‎demo/performance/index.html

+44-44
Original file line numberDiff line numberDiff line change
@@ -6,63 +6,63 @@
66
{
77
"imports": {
88
"lit-html": "https://esm.sh/[email protected]",
9+
"react": "https://esm.sh/[email protected]",
10+
"react-dom/": "https://esm.sh/[email protected]/",
911
"uhtml": "https://esm.sh/[email protected]"
1012
}
1113
}
1214
</script>
1315
<link rel="stylesheet" href="./index.css">
1416
</head>
1517
<body>
16-
<div>
17-
<div class="label">test fixture</div>
18-
<pre id="fixture" class="output"></pre>
18+
<div id="inject" class="test">
19+
<div class="label">inject</div>
20+
<div class="output"></div>
1921
</div>
20-
<div>
21-
<div class="label">default</div>
22-
<pre id="default" class="output"></pre>
22+
<div id="initial" class="test">
23+
<div class="label">initial</div>
24+
<div class="output"></div>
2325
</div>
24-
<div>
25-
<div class="label">lit html</div>
26-
<pre id="lit-html" class="output"></pre>
26+
<div id="update" class="test">
27+
<div class="label">update</div>
28+
<div class="output"></div>
2729
</div>
28-
<div>
29-
<div class="label">µhtml</div>
30-
<pre id="uhtml" class="output"></pre>
31-
</div>
32-
<p>
33-
This tests the performance of <code>html</code> and <code>render</code>.
34-
By testing <em>only</em> these two functions, we isolate time spent by the
35-
templating engine from time spent by the element base class.
36-
</p>
37-
<p>
38-
The term &ldquo;inject&rdquo; refers to the process of taking an array
39-
of template strings, injecting special markup strings, instantiating a
40-
<code>&lt;template&gt;</code> element, mapping DOM elements based on the
41-
special markup previously injected, cloning that template element, and
42-
finally rendering it within a container element. Injection happens only
43-
<em>once per template function declaration</em>. This is the most
44-
time-consuming step of the process, but it also is only typically needed
45-
once per element base class definition.
46-
</p>
47-
<p>
48-
The term &ldquo;initial&rdquo; refers to the process of taking a
49-
template that&rsquo;s <em>already</em> been injected and rendering it into
50-
a new container element. For example, if you render multiple elements in
51-
the page, the templating function is likely shared &mdash; this means that
52-
the engine can skip the <em>injection</em> phase altogether. This happens
53-
whenever a new element of the same type is created (i.e., the same element
54-
appearing in a list over-and-over again).
55-
</p>
30+
<p><b>
31+
For simplicity, these tests do not run in a cross-origin isolated
32+
environment. That means that &ldquo;performance.now&rdquo; timestamps will
33+
only be accurate to roughly &ldquo;100 &micro;s&rdquo;. To combat this,
34+
tests are run in batches so that each time delta is large enough so that
35+
the cross-origin discrepancy would add at most ~2% of error.
36+
</b></p>
37+
<p><b>
38+
Also, consider how developer tools windows or multiple tabs may impact
39+
these tests &mdash; i.e., take all of this information with a major grain
40+
of salt. The goal here is just to show the <em>relative</em> performance
41+
of a handful of engines.
42+
</b></p>
43+
<p><b>
44+
Each asterisk represents a percentile of the underlying data. The median
45+
(p50) is highligted in the chart and printed next to the related
46+
engine&rsquo;s name. The first and last few percentiles are omitted to
47+
reduce jumpiness in the charts from run-to-run.
48+
</b></p>
5649
<p>
57-
The term &ldquo;update&rdquo; refers to the process of changing the values
58-
which are interpolated into a given template. This is the most common
59-
thing the engine needs to do.
50+
The term &ldquo;inject&rdquo; test refers to the process of starting from
51+
some completely-zero state. I.e., new container DOM and a fresh template.
52+
The term &ldquo;initial&rdquo; test is nearly identical to
53+
&ldquo;inject&rdquo; in setup, but it&rsquo;s possible that the package
54+
has cached some computations related to the input template strings (note
55+
that the container here is still created from scratch). The term
56+
&ldquo;update&rdquo; test is the most fine-grained test (given a
57+
pre-existing container that&rsquo;s already been set up, how long does it
58+
take to simply <em>update</em> the managed DOM).
6059
</p>
6160
<p>
62-
Finally, a note on how the tests work — they are batched up and run within
63-
animation frames to guard against any interference that might occur when
64-
an animation frame is skipped due to the main thread being busy. This is
65-
why the tests all take the same amount of time to complete.
61+
Finally, a note on how the tests work &mdash; they are batched up and run
62+
within animation frames to guard against any interference that might occur
63+
when an animation frame is skipped due to the main thread being busy. This
64+
is why the tests all take the same amount of time to complete even though
65+
the actual tasks they measure can differ.
6666
</p>
6767
<script type="module" src="./index.js"></script>
6868
</body>

0 commit comments

Comments
 (0)