Add HTML-in-Canvas APIs #11588

foolip · 2025-08-21T12:51:56Z

Handwavy things that need fleshing out are marked with 👋

At least two implementers are interested (and none opposed):
- Chromium
- …
Tests are written and can be reviewed and commented upon at:
- …
Implementation bugs are filed:
- Chromium: https://issues.chromium.org/u/1/hotlists/6783002
- Gecko: …
- WebKit: …
- Deno (only for timers, structured clone, base64 utils, channel messaging, module resolution, web workers, and web storage): …
- Node.js (only for timers, structured clone, base64 utils, channel messaging, and module resolution): …
Corresponding HTML AAM & ARIA in HTML issues & PRs:
MDN issue is filed: …
The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

/canvas.html ( diff )
/index.html ( diff )
/indices.html ( diff )
/rendering.html ( diff )

Handwavy things that need fleshing out are marked with 👋

Kaiido

Glad to see this being worked on, thanks.

Not quite sure how much discussion should be held at this stage. So to note, this doesn't seem to fully match the latest state of https://github.com/WICG/html-in-canvas. e.g. the rename to drawHTMLElement. The layoutsubtree attribute is also missing along with the implications to the existing fallback contents.
Still, thanks for making this move.

source

Kaiido · 2025-08-25T06:01:25Z

source

+
+   <li><p>If either <var>w</var> or <var>h</var> are zero, then return.</p></li>
+
+   <li><p>👋 Paint <var>element</var> to the specified rectangular area without using any


Why limit what can be painted rather than simply tainting the canvas? I guess many use cases won't need any readback.

https://github.com/WICG/html-in-canvas does have a tainting mode, but we've been discussing whether we should keep it. The issue is that we can't do the same for WebGL and WebGPU, because the pixels can always be exfiltrated using shaders.

source

foolip · 2025-08-29T09:47:43Z

@Kaiido thank you for the review! I've fleshed things out more, renaming to drawHTMLElement() (because drawElement() will probably not be workable in WebGL) and adding the layoutsubtree attribute.

There's still some handwaving going on of course, in particular what causes the subtree to be laid out but not painted.

foolip · 2025-09-02T20:20:35Z

source

+  <span>represents</span> <span>embedded content</span> and has a <code
+  data-x="attr-canvas-layoutsubtree">layoutsubtree</code> attribute specified is additionally
+  expected to be treated as 👋replaced element with subtree layout👋, where children are laid out
+  but not rendered.</p>


Should also say that each child is laid out as if it's the only child.

Should also put contain: layout in the UA style sheet, in that case maybe don't need isolation: isolate.

foolip · 2025-09-17T16:21:14Z

I've fleshed this out some more now, in particular the hit testing.

Kaiido

One common complain with the use of dictionaries in the Canvas2D API is that this makes GC kick in very often during animations which has a non-negligible performance cost.
This API shape makes a big use of such dictionaries with one for the wrapper CanvasElementHitTestRegion and then a nested one for the CanvasHitTestRect, and I guess there will be scenarios where multiple of these will need to be updated at every frame. Since the values are copied over from the passed objects to new internal objects, it's unclear if even a careful author, who would try to reuse the same objects, could avoid GC at all here.

On the other hand, I really like how this API shape enables future additions like using a Path2D, or even a bitmap mask, instead of a CanvasHitTestRect. (btw can we bikeshed on rect for that purpose?)

It's not my area of expertise, but would an actual exposed interface allow for non copy from JS, so that authors can just update the regions instead of setting new ones?

source

Kaiido · 2025-09-23T01:32:28Z

source

+
+  <ol>
+   <li>
+    <p>For each <span>hit test region</span> <var>region</var> in <var>canvas</var>'s <span>hit test regions</span>:</p>


Definitely an edge case, and I suppose it's a bit of a gray area (see whatwg/infra#396) but how is this supposed to work if setHitTestRegions is called during this iteration? E.g.

// Add multiple regions ctx.setHitTestRegions([ { element, rect: { x, y } }, { element: anotherElement, rect: { x: anotherX, y: anotherY } } ]); element.onclick = e => ctx.setHitTestRegions([]); // that was a 'once' handler

Should the anotherElement still perform the hit-test?

This kind of problem is sometimes handled in the spec by making a frozen copy of the thing to iterate before starting iteration. Do you think that'd be OK here?

That would certainly be clearer as to what's supposed to happen yes. Now, whether it's the best behavior or not, I don't know and don't have any strong opinion. Both possibilities might come surprising depending on the case. The fact that the timing of hit-testing w.r.t. events propagation isn't well defined doesn't help...

@szager made a good point here which is that really when we start dispatching events hit testing is already done. So rather than making a copy of the list here, the spec here needs to make clear how the list is used in hit testing and that it all happens before event dispatch.

source

progers · 2025-12-04T23:10:24Z

source

+   is the same for all users.</p></li>
+
+   <li><p>Let <var>T<sub>draw</sub></var> be the scale+translation matrix that transforms the box
+   (0, 0, width of <var>borderBox</var>, width of <var>borderBox</var>,) to


Nit: second width of <var>borderBox</var> should be height of <var>borderBox</var>.

progers · 2025-12-04T23:11:03Z

source

+   <span>sensitive information</span>. Instead, either paint nothing or use static information that
+   is the same for all users.</p></li>
+
+   <li><p>Let <var>T<sub>draw</sub></var> be the scale+translation matrix that transforms the box


This also needs to say that the current transformation matrix is taken into account.

progers · 2025-12-04T23:12:07Z

source


  <pre><code class="css">@namespace "http://www.w3.org/1999/xhtml";

+canvas[layoutsubtree] > * { isolation: isolate !important; contain: strict !important; }


I think we just need contain: paint rather than contain: strict.

progers · 2025-12-04T23:15:19Z

source

+  DOMMatrix <span data-x="dom-context-2d-drawElementImage">drawElementImage</span>(<span>Element</span> element, unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h);
+};
+
+dictionary <dfn dictionary>CanvasHitTestRect</dfn> {


progers · 2025-12-04T23:15:25Z

source

+  double? <dfn dict-member for="CanvasHitTestRect" data-x="dom-CanvasHitTestRect-height">height</dfn>;
+};
+
+dictionary <dfn dictionary>CanvasElementHitTestRegion</dfn> {


progers · 2025-12-04T23:19:08Z

source

+
+   <li><p>If either <var>w</var> or <var>h</var> are zero, then return.</p></li>
+
+   <li><p>Paint <var>element</var> to the specified rectangular area without using any


Also mention that the element's transform is ignored for painting.

progers · 2025-12-04T23:32:57Z

source


+interface mixin <dfn interface>CanvasDrawElementImage</dfn> {
+  // drawing elements
+  DOMMatrix <span data-x="dom-context-2d-drawElementImage">drawElementImage</span>(<span>Element</span> element, unrestricted double x, unrestricted double y);


can you change these to dx and dy (similarly, dw and dh)? This aligns our names with the drawImage API, and will be needed to disambiguate the sx, sy, sw, and sh parameters we are working on now.

progers · 2025-12-04T23:38:05Z

source

+
+   <p>This example paints two elements with hit testing. Since all children are laid out at the
+   top left, the 'transform' CSS property is used to make their position for hit testing match
+   the drawn order. In order redraw when needed, for example if text is selected a ResizeObserver


This sentence is confusing to me. Maybe...
In order redraw when needed, for example if text is selected, a ResizeObserver with the <code data-x="">fireOnEveryPaint</code> parameter is used.</p>?

progers · 2025-12-05T00:21:14Z

source

+  }
+
+  onload = () => {
+    const resizeObserver = new ResizeObserver((entries) => {


Slightly more concise version:

const resizeObserver = new ResizeObserver(([entry]) => { canvas.width = entry.devicePixelContentBoxSize[0].inlineSize; canvas.height = entry.devicePixelContentBoxSize[0].blockSize; draw(); });

progers · 2025-12-05T00:23:29Z

source

+    let transform = ctx.drawElementImage(leftElm, x * devicePixelRatio, y * devicePixelRatio);
+    leftElm.style.transform = transform;
+
+    // purple circle goes in between the elements


I worry this example may be considered an anti-pattern because the purple area isn't hit testable. We could use canvas 2d drawing apis to draw an interesting path that includes both boxes and is painted as the background and which is hard to do with regular css?

Stub out canvas.drawElement()

e181de1

Handwavy things that need fleshing out are marked with 👋

foolip marked this pull request as draft August 21, 2025 12:52

Kaiido reviewed Aug 25, 2025

View reviewed changes

foolip mentioned this pull request Aug 28, 2025

Drawing only part of an element (cropping) WICG/html-in-canvas#37

Open

foolip added 5 commits August 28, 2025 14:25

Fix typ: y->h

8634ebe

Modernize the style somewhat

afc4a4d

Replace handwavy check with real check and drop "update the rendering"

f151c83

Add the layoutsubtree attribute

74d67ae

Rename to drawHTMLElement()

f7d320a

foolip changed the title ~~Stub out canvas.drawElement()~~ Stub out canvas.drawHTMLElement() Aug 29, 2025

foolip mentioned this pull request Sep 2, 2025

HTML-in-Canvas #10650

Open

Fix the layoutsubtree UA style sheet

3cad6d4

foolip commented Sep 2, 2025

View reviewed changes

foolip changed the title ~~Stub out canvas.drawHTMLElement()~~ Stub out canvas.drawElementImage() Sep 16, 2025

foolip added 6 commits September 16, 2025 17:06

Rename to drawElementImage()

4367aac

Define sensitive information

84ec8e6

Expose children to AT

f351c3d

grammar

aed994e

Use containment to make children containing blocks

8c3ce46

Define setHitTestRegions()

ff066c9

foolip changed the title ~~Stub out canvas.drawElementImage()~~ Add HTML-in-Canvas APIs Sep 17, 2025

Kaiido reviewed Sep 23, 2025

View reviewed changes

foolip commented Sep 30, 2025

View reviewed changes

source Outdated Show resolved Hide resolved

foolip added 5 commits October 7, 2025 15:26

Merge remote-tracking branch 'origin/main' into foolip/html-in-canvas

5e3c3d0

Wrap algorithms

f8f119a

Fix typo and reserved word

202adae

Add a note about not using CTM

be22310

Fix reference to hit test regions

9866034

foolip added 2 commits October 7, 2025 17:14

Skip non-positive width/height regions

4e559a9

Use CSS border box

4838669

foolip mentioned this pull request Oct 22, 2025

WHATUP at TPAC 2025: agenda suggestions #11711

Closed

foolip added 5 commits November 11, 2025 12:19

Replace hit testing API with a simpler model

9fa39e9

Add a hit testing example

f79ed95

Add getElementTransform() and related algorithms

0a243be

Update drawElementImage() to return matrix

1f71b9a

Express the transformations in a more approachable way

5aeb03b

progers reviewed Dec 5, 2025

View reviewed changes


		<li><p>If either <var>w</var> or <var>h</var> are zero, then return.</p></li>

		<li><p>👋 Paint <var>element</var> to the specified rectangular area without using any


		<pre><code class="css">@namespace "http://www.w3.org/1999/xhtml";

		canvas[layoutsubtree] > * { isolation: isolate !important; contain: strict !important; }


		<li><p>If either <var>w</var> or <var>h</var> are zero, then return.</p></li>

		<li><p>Paint <var>element</var> to the specified rectangular area without using any

Add HTML-in-Canvas APIs #11588

Are you sure you want to change the base?

Add HTML-in-Canvas APIs #11588

Uh oh!

Conversation

foolip commented Aug 21, 2025 • edited by pr-preview bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kaiido left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

foolip commented Aug 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foolip commented Sep 17, 2025

Uh oh!

Kaiido left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

foolip commented Aug 21, 2025 •

edited by pr-preview bot

Loading

Kaiido left a comment •

edited

Loading