Add screen-reader accessibility support (AppKit + Qt)#8603
Closed
sideshowbarker wants to merge 19 commits into
Closed
Add screen-reader accessibility support (AppKit + Qt)#8603sideshowbarker wants to merge 19 commits into
sideshowbarker wants to merge 19 commits into
Conversation
sideshowbarker
force-pushed
the
macos-accessibility
branch
3 times, most recently
from
March 24, 2026 15:02
2b58463 to
a940ecd
Compare
github-actions
Bot
added
the
conflicts
|
Your pull request has conflicts that need to be resolved before it can be reviewed and merged. Make sure to rebase your branch on top of the latest |
sideshowbarker
force-pushed
the
macos-accessibility
branch
from
March 24, 2026 17:22
a940ecd to
0f0c33e
Compare
github-actions
Bot
added
conflicts
|
Your pull request has conflicts that need to be resolved before it can be reviewed and merged. Make sure to rebase your branch on top of the latest |
sideshowbarker
force-pushed
the
macos-accessibility
branch
from
March 24, 2026 19:49
0f0c33e to
2755cb0
Compare
github-actions
Bot
removed
the
conflicts
sideshowbarker
force-pushed
the
macos-accessibility
branch
3 times, most recently
from
March 24, 2026 20:25
a0d972d to
c0366ec
Compare
sideshowbarker
force-pushed
the
macos-accessibility
branch
from
April 1, 2026 13:55
c0366ec to
bd9a46f
Compare
sideshowbarker
changed the title
sideshowbarker
changed the title
sideshowbarker
force-pushed
the
macos-accessibility
branch
3 times, most recently
from
April 2, 2026 02:22
9e01553 to
d0b00fc
Compare
Expose the accessibility tree to macOS VoiceOver and Accessibility Inspector by serializing tree data from the WebContent process and presenting it through NSAccessibility wrapper objects in the UI process. - AccessibilityNodeData IPC struct for serializing tree nodes - AccessibilityTreeManager for caching the tree in the UI process - LadybirdAccessibilityElement as the NSAccessibility wrapper - Accessibility dump test mode in test-web, to enable testing - Map document role to AXWebArea for web-content recognition - Make generic elements ignored so children are navigable - Post AXLoadComplete when the page finishes loading - Expose AXLoaded attribute on the web area element - Return first meaningful element from focused UI element - Add custom role descriptions for landmark subroles
Live focus tracking: - Add page_did_change_active_element to PageClient - New IPC endpoint did_accessibility_focus_change - UI process calls NSAccessibilityHandleFocusChanged() VoiceOver integration improvements: - Map document to AXWebArea with AXLoaded/AXLoadingProgress - Post AXLoadComplete on the NSView ancestor - LadybirdWebView role changed to AXScrollArea - Map heading role to AXHeading Tree-structure fixes: - Filter title text nodes from the accessibility tree - Manually flatten ignored elements in children - Walk up past ignored ancestors in accessibilityParent Text markers: - Add AXStartTextMarker, AXEndTextMarker Parameterized attributes for navigation: - Add AXIndexForChildUIElement for child ordering - Add AXElementBusy, AXSelected, AXVisited, AXBlockQuoteLevel Fix VoiceOver forward navigation into nested groups - Advertise AXUIElementsForSearchPredicate - Modern property API override for accessibilityWindow and - Modern property API override for accessibilityTopLevelUIElement - Make accessibilityHitTest not return ignored elements - Set shouldGroupAccessibilityChildren to No Fix VoiceOver double-reading of elements: - Treat leaf-like interactive roles as navigation terminals - Restrict text-marker and loading attributes to document root - Don’t expose AXStartTextMarker/AXEndTextMarker on every element Fix VoiceOver re-traversal of container subtrees - When navigating forward, skip “container descendants” blocks Code cleanup: - Consolidate calls to is_ignored_role() static function - Drop duplicate accessibilityFocusedUIElement) - Drop buggy AXNext|PreviousTextMarkerForTextMarker - Simplify AXIndexForChildUIElement by droppin nodeID matching - Fix text marker memory leaks by using CFBridgingRelease
This routes VoiceOver press/activate actions back to WebContent via a new perform_accessibility_action IPC endpoint. When you press VO+Space on a link, button, or other interactive element, HTMLElement::click() gets called on the corresponding DOM node, looked up by UniqueNodeID. This also adds support setting focus via run_focusing_steps() when VoiceOver requests it through accessibilitySetValue:forAttribute: with the focused attribute.
Add table-specific accessibility attributes — so VoiceOver can announce row/column position when navigating table cells: - AXRowCount, AXColumnCount, AXRows, AXVisibleRows, AXColumns, AXVisibleColumns, AXHeader on table elements - AXIndex on row elements (zero-based across rowgroups) - AXRowIndexRange, AXColumnIndexRange on cell elements
The accessibility tree was a static snapshot taken at page load. DOM mutations, AJAX content updates, and interactive state changes weren’t reflected until the next full page navigation. This adds a debounced-tree-update mechanism: when the DOM’s mutated (via the existing page_did_mutate_dom callback used by DevTools) — or when an accessibility action is performed — schedule a tree rebuild after 200ms. The debounce timer batches rapid changes to avoid excessive rebuilds. The existing did_get_accessibility_tree IPC is reused for updates, so the UI process handles them identically to the initial tree load: clear element cache, update the manager, post layout-changed notifications.
When content inside an aria-live region changes, VoiceOver now announces the updated text. That covers both explicit aria-live attributes and implicit live roles (alert, status, log, marquee, timer). - Add “live” field to AccessibilityNodeData, populated from element’s aria-live attribute or implicit role default (assertive for alert, polite for status/log/etc.) - AccessibilityTreeManager compares old/new trees during update_tree(), detecting name changes in nodes that are inside live regions - LadybirdWebView posts NSAccessibilityAnnouncementRequested with appropriate priority (High for assertive, Medium for polite)
Text markers now encode both a node ID and a character offset, enabling cursor-level text navigation. (Previously they only stored a node ID, and forward/backward navigation was stubbed.) - AXNextTextMarkerForTextMarker: advance by one character within a text node, or move to the next text leaf - AXPreviousTextMarkerForTextMarker: retreat by one character, or move to the previous text leaf's last character - AXStringForTextMarkerRange: extract text content between two marker positions across multiple text nodes - AXAttributedStringForTextMarkerRange: same but wrapped in NSAttributedString - AXLengthForTextMarkerRange: character count of the range - AXTextMarkerRangeForUIElement: range spanning all text leaves in an element's subtree - AXStartTextMarker/AXEndTextMarker: first/last positions in the document's text content Added text_leaves_in_order() to AccessibilityTreeManager — for efficient DFS-ordered text leaf enumeration.
- We didn’t have the encoder encoding the “live” field, but the decoder
tried to decode it — causing IPC message parse failures.
- Text leaf nodes (AXStaticText role) returned the same text content in
AXTitle, AXDescription, and AXValue. VoiceOver reads both Title and
Value, causing each text fragment to be spoken twice (“The The”).
- For static text elements, the text content belongs only in AXValue.
AXTitle and AXDescription should return nil, matching how Safari and
Chrome handle static text.
- Fixed VoiceOver focus ring covering entire “main” area
When VoiceOver navigated to text content inside paragraphs, the
focus ring covered the entire main element, rather than just the
paragraph area. Two causes:
1. Text leaf nodes had zero bounds — because only element nodes
got bounding rects during serialization. Text nodes now
inherit their parent element’s bounds as an approximation.
2. Paragraphs were ignored (transparent), so text leaves were
promoted to main as direct children. Paragraphs are now
kept in the accessibility tree as proper containers with
their own bounds, but the search predicate skips them — so
VoiceOver navigates directly to the text content inside.
This change fixes some bugs in the existing code which were encountered while developing the macOS accessibility support: - is_disabled now populated via Element::is_actually_disabled(), so native HTML disabled buttons/inputs correctly report disabled state. - aria-hidden=true elements and their descendants are now excluded from the accessibility tree. The exclusion check walks up the ancestor chain, matching the spec requirement that aria-hidden on a parent overrides descendants. The tree builder now distinguishes between excluded elements (skip descendants entirely) and non-included elements (still process children). - aria-description attribute is now used as a fallback when accessible_description() returns empty, populating the description field in AccessibilityNodeData.
This change fixes some bugs in the existing code which were encountered while developing the macOS accessibility support: - Images with aria-label but no alt attribute were excluded from the accessibility tree because their default role is none/presentation. Per the Presentational Roles Conflict Resolution spec, elements with global ARIA attributes (like aria-label) override the presentational role. The element is now included with role=image. - Accessible names computed via aria-labelledby could have leading/ trailing whitespace from text concatenation. Names are now trimmed after computation.
This change fixes some bugs in the existing code which were encountered while developing the macOS accessibility support: - Table-cell colspan/rowspan are now serialized in AccessibilityNodeData and used in AXColumnIndexRange and AXRowIndexRange, so VoiceOver correctly reports cell spans. - AXTextMarkerForPosition is now implemented: converts a screen point to a text marker by hit-testing the tree, walking up past ignored elements, and finding the nearest text leaf.
This change fixes two bugs that prevented aria-describedby from working: - accessible_description() called name_or_description() with the NameOrDescription::Description target on the referenced element. But the ACCNAME algorithm’s code paths for computing text content (steps D, F) are gated on the Name target. Changed to use the NameOrDescription::Name — matching the spec’s intent to compute the “text alternative” of the referenced elements. - Element::is_referenced() only checked aria-labelledby, not aria-describedby. That meant elements referenced only by aria-describedby weren’t recognized as “referenced” for the “Name From Content” step (step F) in the ACCNAME algorithm — so their text content was never collected.
Document the architecture, design decisions, and implementation details of Ladybird’s macOS accessibility (VoiceOver) support. Includes the IPC data model, NSAccessibility wrapper design, VoiceOver navigation via search predicates, and a section on what was learned from studying the Chromium, WebKit, and Firefox accessibility source code.
The tree builder was skipping descendants for *all* excluded elements, but role=none/presentation elements should still have their children promoted to the parent. Only aria-hidden and display:none elements should skip descendants entirely. Added Element::is_aria_hidden() helper to distinguish among those cases in the tree builder.
The accessibility tree was only requested from on_load_finish, which doesn’t fire for many pages (e.g., SPA navigations). This change adds a debounced tree request that fires from on_load_start, on_url_change, and on_title_change — in addition to on_load_finish. The requests are debounced via performSelector:afterDelay: (500ms), so that rapid callback sequences result in a single tree build after settling. Also fix accessibilityFocusedUIElement to use DFS (document order) instead of BFS. The previous approach returned the first DOM-focused element, which could be anywhere on the page if JavaScript called .focus() on a non-top element. DFS returns the first non-ignored element from the *top* of the document.
Bug: VoiceOver was unexpectedly reading/navigating through the entire contents of collapsed menus — including non-visible menu items. Cause: To determine whether to exclude an element from the accessibility tree, we were just checking whether the element had a layout node. But that caused visibility:hidden elements to be unexpectedly included (because even though they’re not visible, they still have layout nodes). Fix: Exclude visibility:hidden elements from the accessibility tree. The ARIA spec states that elements which are “not displayed” must not be included in the accessibility tree, and visibility:hidden is among the “not displayed” conditions it explicitly cites.
For the Qt UI: Expose the accessibility tree to screen readers via Qt’s QAccessibleInterface. On Linux, Qt bridges to AT-SPI2 (Orca). On macOS, Qt bridges to NSAccessibility (VoiceOver). The new AccessibilityInterface class wraps each AccessibilityNodeData node, implementing QAccessibleInterface, QAccessibleActionInterface, QAccessibleTextInterface, and QAccessibleTableCellInterface. It shares the same IPC infrastructure and AccessibilityTreeManager with the implementation for the AppKit UI. Qt’s cocoa bridge (QMacAccessibilityElement) for macOS lacks the support we need for AXWebArea, landmark subroles, custom role descriptions, and AXUIElementsForSearchPredicate. So, we add those at runtime by swizzling QMacAccessibilityElement methods to check for dynamic QObject properties (_qt_mac_subrole, _qt_mac_roleDescription) and to implement search-predicate DFS traversal.
…and add “Alternative: Toolkit-independent accessibility” section.
sideshowbarker
force-pushed
the
macos-accessibility
branch
from
April 2, 2026 17:27
d0b00fc to
2c134ed
Compare
Orca queries a widget’s children when it first appears. If the accessibility tree hasn’t loaded yet, Orca sees 0 children and never re-queries. So this change causes us to post a QAccessibleEvent after the tree loads — in order to make Orca know to re-query. (On macOS, we already do the equivalent: notify_accessibility_tree_loaded() handles this via AXLoadComplete + NSAccessibilityHandleFocusChanged.)
sideshowbarker
force-pushed
the
macos-accessibility
branch
from
April 2, 2026 17:29
2c134ed to
64e5217
Compare
sideshowbarker
marked this pull request as draft
April 14, 2026 06:55
github-actions
Bot
added
the
conflicts
|
Your pull request has conflicts that need to be resolved before it can be reviewed and merged. Make sure to rebase your branch on top of the latest |
Member
Author
|
Superseded by #9156 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR enables screen-reader support for Ladybird — exposing the internal accessibility tree to platform accessibility APIs. Both the AppKit UI (macOS VoiceOver) and the Qt UI (Orca on Linux, VoiceOver on macOS) are supported.
The architecture follows the Chromium/Firefox model: the accessibility tree is serialized from WebContent via IPC, cached in a platform-agnostic
AccessibilityTreeManager(in LibWebView), and presented through platform-specific wrapper objects in the UI process. The infrastructure — IPC serialization, tree management, and WebContent-side tree building — is shared between both the AppKit UI and Qt UI.What this enables
QAccessibleInterfacebridge exposes the tree to AT-SPI2 on Linux (untested but structurally complete; Qt correctly mapsWebDocumenttoATSPI_ROLE_DOCUMENT_WEB)HTMLElement::click())AXTextMarkerRef(AppKit only)AppKit UI
LadybirdAccessibilityElement(Objective-C) wraps each node, implementing theNSAccessibilityinformal protocol. UsesNSObjectas the base class —NSAccessibilityElementwas tried but its synthesized properties competed with our dynamic getter overrides (Firefox also usesNSObject— perhaps for the same reason). VoiceOver navigates web content viaAXUIElementsForSearchPredicate, implemented as pre-order DFS with leaf-role skipping and container-descendant skipping.Qt UI
AccessibilityInterface(C++) wraps each node as aQAccessibleInterfacesubclass. On Linux, Qt’s bridge handles roles natively. On macOS, Qt’s cocoa bridge lacks web-content support, soWebContentViewAccessibilityMac.mmswizzles seven methods onQMacAccessibilityElementat runtime to addAXWebArearole, landmark subroles, role descriptions, search predicates, focused-element handling, andListItem-ignore behavior.Bug fixes to existing code
This PR also fixes several bugs in the existing ARIA/ACCNAME infrastructure discovered through testing:
Element::is_actually_disabled()is now used for theis_disabledfieldElement::exclude_from_accessibility_tree()handlesaria-hidden="true"(walks ancestor chain, excludes descendants)Element::exclude_from_accessibility_tree()handlesvisibility:hidden/visibility:collapseElement::is_referenced()checksaria-describedbyin addition toaria-labelledbyNode::accessible_description()usesNameOrDescription::Namewhen computing text alternatives foraria-describedbyreferenced elementsrole=none/presentationbut global ARIA attributes are not excludedNode::build_accessibility_tree()skips descendants of excluded elements entirelyDocs
Documentation/Accessibility.mdcovers the full design: shared architecture, AppKit and Qt presentation layers, design constraints discovered through testing, a comparison table of differences between the two UIs, detailed analysis of how Chromium/Firefox/WebKit handle the same concerns, an assessment of AccessKit as a potential alternative, and known limitations (including QtCollectioninterface support requiring Qt 6.11+ for Orca structural navigation).