Clarify how canvas fallback content relates to the accessibility tree

[brennanyoung]

I find the spec for canvas wanting clarification for how (or whether) fallback content should be included in the accessibility tree.

Consider these two distinct use cases where an alternative to canvas may be useful:

• The user agent does not support canvas or scripting.
• The user agent does support canvas or scripting, but the user relies on assistive technology (AT) to represent an alternative to the canvas.

Are these compatible? The current spec says

In non-visual media, and in visual media if scripting is disabled for the canvas element or if support for canvas elements has been disabled, the canvas element represents its fallback content instead.

"instead"? This might be interpreted as permission for UA vendors to exclude canvas fallback content from the accessibility tree if the canvas renders just fine. I find that problematic. I notice some ATs handle canvas fallback content quite differently to (say) div content (even though they have the same "generic" semantic).

I'm reminded of how the alt attribute is a key mechanism for accessibility, but is also presented visually when an image fails to load.

Concrete case: A canvas rendering of a stock ticker. (Aria has the log role for such things, so that's a solved problem).

Where does the accessible alternative belong? In the fallback subtree? Before? After?

How do I make sure the AT 'gets' the fallback content if the browser succeeds in rendering it?

The spec also says that canvas (fallback) content should be non-interactive, and then goes on to list most of the HTML interactive elements. (Operable aria roles are, of course excluded). <input type=text> appears to be forbidden amongst canvas fallback content. Why? AT users might need exactly this. Maybe the canvas view is just a fancy way of presenting a more modest, accessible DOM alternative (a kind of progressive enhancement).

Basically the spec is not at all clear (for authors and UA vendors) how accessible HTML alternatives may best be associated with canvas presentations, especially interactive ones.

Edit: I am not the first to make these observations. This excellent article from TPGi covers similar ground. Please pay close attention the section on "fallback content"

[annevk]

cc @whatwg/canvas @whatwg/a11y

[kdashg]

This is mostly an a11y question, I think? I'm happy to answer questions on the canvas side, but I'm out of my depth on a11y subtrees.

[junov]

Many years ago, the spec had a feature called HitRegions that created a link between pixel regions in a canvas and elements in the fallback content. It was a solid design that made it possible to use keyboard input, element focus, aria roles, etc. to interact with custom controls rendered into a canvas. We even had a prototype implementation in Blink. In the end, it was never shipped and the feature was removed from the spec because there was no demand for it from developers. Many apps that use canvas to render interactive UIs (for example the cell grid in Google Sheets) use invisible DOM elements (non-fallback) to implement accessibility and focus. I think this part of the spec still contains remnants of the HitRegions feature that we need to clean up.

[patrickhlauke]

At a high level, the spec should make clear that it's not just "fallback" in the sense that it will be visually rendered if the <canvas> isn't supported, but that even if it is visually rendered, the content inside the element is present in the DOM and and will be exposed to assistive technologies. Allow the presence of interactive controls inside that, but advise authors that if they do this, they're likely to cause accessibility issues (focus disappearing visually, for instance) that they'll need to handle.

[brennanyoung]

@junov I can only imagine that the lack of clarity about the accessibility of the canvas subtree did not help any potential adoption of HitRegions, which BTW is still a good idea...

[brennanyoung]

Just want to mention a significant real world case:
mozilla/pdf.js#6269

pdf.js is used by Firefox and perhaps other browsers for rendering PDF in-browser using a canvas.

It also 'translates' tagged PDF into a DOM tree with ARIA.

The generated DOM is placed inside the 'fallback' for the canvas.

[junov]

I would like to know whether there are existing use cases that require unrendered fallback content to be interactive (focus + keyboard input). Currently this is partially broken in Blink-based browsers, but practically no-one seems to care (since HitRegions were abandoned), so those bugs are not getting any attention.

Unless I am missing something, It seems to me like DOM elements that overlap the canvas (without being children of the canvas element) are largely sufficient for supporting interaction and accessibility use cases. So I would like to suggest removing the requirement that unrendered fallback content be interactive.

[brennanyoung]

That would be completely fine with me, as long as the updated spec does not actually preclude (or dissuade from) interactive content in the canvas subtree.

I'm curious about those Blink bugs you mentioned tho, because I would certainly expect (and desire) keyboard operation to work for any accessible canvas alternative, including alternatives that 'live' in the fallback zone.

If there is apathy about the matter today, it is not safe to assume that content authors don't care to have that feature, it's just that the whole area of canvas subtree seems full of dragons, and accessibility efforts tend to lean towards what is known to work. (Chicken and egg).

It seems we're both attempting to move the goalposts from opposite ends. In the end, I hope we'll have a neat basket-and-backboard instead of a gaping, mysterious maw.

[AutoSponge]

If the APA can help in any way, this might be a topic for TPAC this year.