Add screen-reader support: Orca and VoiceOver#8738
Closed
sideshowbarker wants to merge 13 commits into
Closed
Conversation
sideshowbarker
changed the title
sideshowbarker
force-pushed
the
toolkit-independent-accessibility
branch
2 times, most recently
from
April 2, 2026 10:52
0be146d to
f45a60f
Compare
sideshowbarker
marked this pull request as draft
April 2, 2026 16:38
sideshowbarker
force-pushed
the
toolkit-independent-accessibility
branch
26 times, most recently
from
April 10, 2026 07:09
db792cb to
347f45a
Compare
github-actions
Bot
added
the
conflicts
|
Your pull request has conflicts that need to be resolved before it can be reviewed and merged. Make sure to rebase your branch on top of the latest |
sideshowbarker
force-pushed
the
toolkit-independent-accessibility
branch
2 times, most recently
from
April 10, 2026 08:35
093cb0d to
0824141
Compare
github-actions
Bot
removed
the
conflicts
sideshowbarker
force-pushed
the
toolkit-independent-accessibility
branch
from
April 10, 2026 08:39
0824141 to
11bc2d1
Compare
sideshowbarker
changed the title
Expose the accessibility tree to macOS VoiceOver and Accessibility Inspector by serializing tree data from the WebContent process and presenting it through NSAccessibility wrapper objects in the UI process. - AccessibilityNodeData IPC struct for serializing tree nodes - AccessibilityTreeManager for caching the tree in the UI process - LadybirdAccessibilityElement as the NSAccessibility wrapper - Accessibility dump test mode in test-web, to enable testing - Map document role to AXWebArea for web-content recognition - Make generic elements ignored so children are navigable - Post AXLoadComplete when the page finishes loading - Expose AXLoaded attribute on the web area element - Return first meaningful element from focused UI element - Add custom role descriptions for landmark subroles
Live focus tracking: - Add page_did_change_active_element to PageClient - New IPC endpoint did_accessibility_focus_change - UI process calls NSAccessibilityHandleFocusChanged() VoiceOver integration improvements: - Map document to AXWebArea with AXLoaded/AXLoadingProgress - Post AXLoadComplete on the NSView ancestor - LadybirdWebView role changed to AXScrollArea - Map heading role to AXHeading Tree-structure fixes: - Filter title text nodes from the accessibility tree - Manually flatten ignored elements in children - Walk up past ignored ancestors in accessibilityParent Text markers: - Add AXStartTextMarker, AXEndTextMarker Parameterized attributes for navigation: - Add AXIndexForChildUIElement for child ordering - Add AXElementBusy, AXSelected, AXVisited, AXBlockQuoteLevel Fix VoiceOver forward navigation into nested groups - Advertise AXUIElementsForSearchPredicate - Modern property API override for accessibilityWindow and - Modern property API override for accessibilityTopLevelUIElement - Make accessibilityHitTest not return ignored elements - Set shouldGroupAccessibilityChildren to No Fix VoiceOver double-reading of elements: - Treat leaf-like interactive roles as navigation terminals - Restrict text-marker and loading attributes to document root - Don’t expose AXStartTextMarker/AXEndTextMarker on every element Fix VoiceOver re-traversal of container subtrees - When navigating forward, skip “container descendants” blocks Code cleanup: - Consolidate calls to is_ignored_role() static function - Drop duplicate accessibilityFocusedUIElement) - Drop buggy AXNext|PreviousTextMarkerForTextMarker - Simplify AXIndexForChildUIElement by droppin nodeID matching - Fix text marker memory leaks by using CFBridgingRelease
This routes VoiceOver press/activate actions back to WebContent via a new perform_accessibility_action IPC endpoint. When you press VO+Space on a link, button, or other interactive element, HTMLElement::click() gets called on the corresponding DOM node, looked up by UniqueNodeID. This also adds support for setting focus via run_focusing_steps() when VoiceOver requests it through accessibilitySetValue:forAttribute: with the focused attribute.
Add table-specific accessibility attributes — so VoiceOver can announce row/column position when navigating table cells: - AXRowCount, AXColumnCount, AXRows, AXVisibleRows, AXColumns, AXVisibleColumns, AXHeader on table elements - AXIndex on row elements (zero-based across rowgroups) - AXRowIndexRange, AXColumnIndexRange on cell elements
The accessibility tree was a static snapshot taken at page load. DOM mutations, AJAX content updates, and interactive state changes weren’t reflected until the next full page navigation. This adds a debounced-tree-update mechanism: when the DOM’s mutated (via the existing page_did_mutate_dom callback used by DevTools) — or when an accessibility action is performed — schedule a tree rebuild after 200ms. The debounce timer batches rapid changes to avoid excessive rebuilds. The existing did_get_accessibility_tree IPC is reused for updates, so the UI process handles them identically to the initial tree load: clear element cache, update the manager, post layout-changed notifications.
When content inside an aria-live region changes, VoiceOver now announces the updated text. That covers both explicit aria-live attributes and implicit live roles (alert, status, log, marquee, timer). - Add “live” field to AccessibilityNodeData, populated from element’s aria-live attribute or implicit role default (assertive for alert, polite for status/log/etc.) - AccessibilityTreeManager compares old/new trees during update_tree(), detecting name changes in nodes that are inside live regions - LadybirdWebView posts NSAccessibilityAnnouncementRequested with appropriate priority (High for assertive, Medium for polite)
Text markers now encode both a node ID and a character offset, enabling cursor-level text navigation. (Previously they only stored a node ID, and forward/backward navigation was stubbed.) - AXNextTextMarkerForTextMarker: advance by one character within a text node, or move to the next text leaf - AXPreviousTextMarkerForTextMarker: retreat by one character, or move to the previous text leaf's last character - AXStringForTextMarkerRange: extract text content between two marker positions across multiple text nodes - AXAttributedStringForTextMarkerRange: same but wrapped in NSAttributedString - AXLengthForTextMarkerRange: character count of the range - AXTextMarkerRangeForUIElement: range spanning all text leaves in an element's subtree - AXStartTextMarker/AXEndTextMarker: first/last positions in the document's text content Added text_leaves_in_order() to AccessibilityTreeManager — for efficient DFS-ordered text leaf enumeration.
Fix accessibility-tree exclusion and serialization bugs, image aria-label handling, name whitespace normalization, aria-describedby population, role=none/presentation descendant handling, tree loading on SPA navigations, visibility:hidden element exclusion, and add table cell span attributes and text-marker hit testing.
…lement On macOS, the Qt port reuses the same LadybirdAccessibilityElement (NSAccessibility wrapper) that the AppKit port uses, via an NSView overlay that conforms to LadybirdAccessibilityViewProtocol. A WebContentAccessibilityView (NSView overlay) provides the scroll-area container, coordinate conversion, element cache, and search-predicate delegation. Three swizzles on QMacAccessibilityElement connect Qt’s cocoa bridge to the overlay: - accessibilityRole: returns AXWebArea for WebContentView elements - accessibilityChildren: returns the overlay as the sole child - accessibilityFocusedUIElement: delegates to the overlay A minimal WebContentViewAccessible (QAccessibleWidget with Grouping role) is registered via QAccessible::installFactory so Qt’s bridge creates a QMacAccessibilityElement for the swizzles to act on. VoiceOver in a Qt app navigates through QMacAccessibilityElement objects, not NSViews. Pure-overlay approaches (without swizzles) were tested extensively: makeFirstResponder on the overlay causes VoiceOver to enter an infinite re-entry loop; without makeFirstResponder, VoiceOver never discovers the overlay. The three swizzles are the minimum needed to bridge Qt’s accessibility tree to the shared LadybirdAccessibilityElement objects. Also extracts LadybirdAccessibilityViewProtocol from the AppKit port so both the AppKit LadybirdWebView and the Qt overlay can conform to the same interface.
Add AccessibilityInterface: a QAccessibleInterface implementation that exposes Ladybird’s accessibility tree to Orca through Qt’s built-in AT-SPI2 bridge. Includes: text interface with U+FFFC hypertext model for embedded objects, object attributes (tag, xml-roles, level) via QAccessibleAttributesInterface (Qt 6.8+, guarded), text run attributes, table-cell interface, actions (press, focus) on all elements for GrabFocus/focus-ring support, and document focus events with a 1000ms delayed notification that skips when another widget (e.g. the address bar) has focus. Leaf-like roles (link, button, heading, img, menuitem, tab, checkbox, radio, listitem, list) expose QAccessibleTextInterface — so Orca can read their content and scroll them into view during structural navigation.
Add custom Orca scripts we embed as Qt resources. On startup, Ladybird
installs them to ~/.local/share/orca/orca-scripts/Ladybird/ if missing
or outdated — so Orca’s browse-mode navigation works out of the box.
The script package extends Orca’s web.Script for Ladybird:
- script.py: Script class extending web.Script (and Qt’s toolkit script,
for issubclass priority). Overrides activate() for cache pre-warming
and on_focused_changed() for address-bar focus mode.
- script_utilities.py: Utilities class extending web.Utilities with
overrides for active_document() (EMBEDS-first with tree-search
fallback), get_caret_context() (document-content fallback), and
get_line_contents_at_offset() (fast path building line content
directly from the accessible tree).
Also installs three runtime patches on Orca’s own classes at
script-load time:
♢ Collection fallback patch: wraps find_all_with_role and
find_all_with_role_and_all_states with a DFS tree-search fallback
when Qt’s Collection GetMatches returns empty. Forward-compatible
with Qt 6.11 which fixes Collection upstream.
♢ SayAll monkey patch: redirects SayAllPresenter.say_all to the
first document-content child when locus is outside the document.
♢ scroll_to_center monkey patch: augments scroll_to_center with a
ScrollSubstringTo call for focus-ring display during structural
navigation.
When Orca navigates through web content, move DOM focus to each element, so the existing CSS :focus-visible focus ring is displayed. That works due to three changes: 1. Advertise setFocusAction on all elements (not just text inputs) so Orca’s GrabFocus AT-SPI2 call reaches our doAction handler. 2. In WebContent, walk from text nodes to their parent element, and set tabindex="-1" on non-focusable elements (headings, paragraphs, etc.) to make them programmatically focusable. 3. Use FocusTrigger::Key when focusing for accessibility so the :focus-visible pseudo-class matches and the focus ring is drawn (same visual as keyboard tab navigation).
This documents the design of the Orca/VoiceOver support in Ladybird.
sideshowbarker
force-pushed
the
toolkit-independent-accessibility
branch
from
April 11, 2026 11:21
11bc2d1 to
1074c8d
Compare
github-actions
Bot
added
the
conflicts
|
Your pull request has conflicts that need to be resolved before it can be reviewed and merged. Make sure to rebase your branch on top of the latest |
Member
Author
|
Superseded by #9156 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This makes Orca and VoiceOver work with Ladybird (AppKit UI on macOS, and Qt UI on both Linux and macOS).
The architecture follows the Chromium/Firefox model: the accessibility tree is serialized from WebContent via IPC, cached in a platform-agnostic
AccessibilityTreeManager(in LibWebView), and presented through platform-specific wrapper objects in the UI process. The IPC serialization, tree management, and WebContent-side tree-building are shared across platforms (macOS and Linux) and across UIs (AppKit and Qt).macOS (AppKit and Qt) – VoiceOver
LadybirdAccessibilityElementimplementsNSAccessibilitydirectly. Both the AppKit UI and the (macOS) Qt UI share it: the Qt build uses an NSView overlay with three very-small, simple, targeted swizzles onQMacAccessibilityElementto connect Qt’s bridge to the shared elements.Linux (Qt) – Orca
AccessibilityInterfaceimplementsQAccessibleInterface, exposing the tree through Qt’s built-in AT-SPI2 bridge — with a custom Orca script which adds patches that work around limitations in tht bridge. (The scripts are embedded as Qt resources and auto-installed on Ladybird startup.)tag,xml-roles,level) for landmark and heading identificationBug fixes to existing code
This PR also fixes several bugs in the existing ARIA/ACCNAME infrastructure:
Element::is_actually_disabled()is now used for theis_disabledfieldElement::exclude_from_accessibility_tree()handlesaria-hidden="true"(walks ancestor chain, excludes descendants)Element::exclude_from_accessibility_tree()handlesvisibility:hidden/visibility:collapseElement::is_referenced()checksaria-describedbyin addition toaria-labelledbyNode::accessible_description()usesNameOrDescription::Namewhen computing text alternatives foraria-describedbyreferenced elementsrole=none/presentationbut global ARIA attributes are not excludedNode::build_accessibility_tree()skips descendants of excluded elements entirelyThis PR also adds a
Documentation/Accessibility.mddoc that covers the full design: shared architecture, AppKit and Qt (both Linux and macOS) presentation layers, Linux Qt AT-SPI2 bridge limitations (with tables mapping which limitations are worked around by our Orca script vs. which require patching Qt itself), browser-engine source code analysis (Chromium, Firefox, WebKit), and an appendix with some Qt bridge patches for potentionally upstreaming.