Implement UIA TextPattern for TextBox on Windows - #21988
Closed
aryanchoudharypro wants to merge 1 commit into
Closed
Implement UIA TextPattern for TextBox on Windows#21988aryanchoudharypro wants to merge 1 commit into
aryanchoudharypro wants to merge 1 commit into
Conversation
TextBox previously only exposed ValuePattern on Windows, so screen readers (e.g. NVDA) had no way to track the caret or selection while editing: focusing the control announced the whole text once, but arrowing through it, selecting, or editing gave no further feedback. Adds a full TextPattern/TextRangeProvider implementation: - Avalonia.Automation.Provider.ITextProvider/ITextRangeProvider: new platform-agnostic contracts, implemented by TextBoxAutomationPeer and its nested TextRange type against TextBox/TextPresenter/ TextLayout (character/word/line navigation, selection, bounding rects). - AutomationPeer gains TextSelectionChanged/TextChanged events, raised by TextBoxAutomationPeer and coalesced so a single edit produces one UIA event rather than one per touched property. - Win32 glue (AutomationNode.Text.cs, Win32TextRangeProvider) wires the above into the existing GeneratedComInterface-based UIA provider, wrapping fresh ITextRangeProvider instances since, unlike AutomationNode, they aren't cached per-peer. Also fixes several bugs in the existing (previously unused/untested) Win32.Automation interop and marshalling code, found while getting TextPattern to work correctly with an out-of-process client: - ITextProvider.RangeFromPoint took two scalar doubles; the real UIA ABI passes a UiaPoint struct by value. - SafeArrayRef.CreateFromObjects used ComWrappers.TryGetComInstance, which only returns a wrapper that already exists rather than creating one. For objects with no prior COM exposure (unlike long-lived AutomationNode instances, which usually already have a wrapper via prior tree navigation) this silently left uninitialized ArrayPool-rented pointer data in the SAFEARRAY, corrupting it. - Interface-typed SAFEARRAYs were built with SafeArrayCreate instead of SafeArrayCreateEx, so they carried no IID (FADF_HAVEIID). Native code marshaling such an array to an out-of-process UIA client has no way to know which interface each element pointer is. - ITextRangeProvider.GetAttributeValue returned a plain -1 for unsupported attributes instead of UIA's reserved "not supported" sentinel (UiaGetReservedNotSupportedValue). At least NVDA reads a plain -1 as a real (truthy) attribute value; for the hyperlink attribute this made a plain TextBox get announced as a link. Adds TextBoxAutomationPeerTests covering the platform-agnostic layer: caret-degenerate selection, forward/reverse selection normalization, character-unit move/expand with clamping, word-unit expansion, bounding rectangles, and coalesced selection-changed events.
|
You can test this PR using the following package version. |
Collaborator
|
Author
|
@cla-avalonia agree |
Contributor
|
This is already part of #20890 |
Member
|
Thank you for your contribution. I'm closing this PR in favor of #20890. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does the pull request do?
TextBoxon Windows previously only exposedValuePattern, so UI Automation clients (screen readers such as NVDA) had no way to track the caret or selection while editing — focusing the control announced the whole text once, but arrowing through it, selecting, or editing gave no further feedback.This adds a full
TextPattern/TextRangeProviderimplementation forTextBoxon Windows:Avalonia.Automation.Provider.ITextProvider/ITextRangeProvidercontracts, implemented byTextBoxAutomationPeerand a nestedTextRangetype againstTextBox/TextPresenter/TextLayout(character/word/line navigation, selection, bounding rectangles).AutomationPeergainsTextSelectionChanged/TextChangedevents, raised byTextBoxAutomationPeerand coalesced so a single edit produces one UIA event rather than one per touched property.AutomationNode.Text.cs,Win32TextRangeProvider) wires the above into the existingGeneratedComInterface-based UIA provider, wrapping freshITextRangeProviderinstances since, unlikeAutomationNode, they aren't cached per-peer.Along the way, found and fixed several real bugs in the existing (previously unused/untested)
Win32.Automationinterop and marshalling code, surfaced only once an out-of-process client (NVDA) actually exercised it:ITextProvider.RangeFromPointtook two scalardoubles instead of theUiaPointstruct the real UIA ABI expects.SafeArrayRef.CreateFromObjectsusedComWrappers.TryGetComInstance(only returns a wrapper that already exists, doesn't create one) instead ofGetOrCreateComInterfaceForObject, silently leaving uninitializedArrayPool-rented pointer data in the SAFEARRAY for objects with no prior COM exposure — this reliably crashed the process (UIAutomationCore!ProtobufHelper::WriteTextRangeArray, a null-pointer dereference) as soon as NVDA queriedGetSelection().SafeArrayCreateinstead ofSafeArrayCreateEx, so they carried no IID (FADF_HAVEIID) — native code marshaling such an array to an out-of-process client has no way to know which interface each element pointer is.ITextRangeProvider.GetAttributeValuereturned a plain-1for unsupported attributes instead of UIA's reserved "not supported" sentinel (UiaGetReservedNotSupportedValue). NVDA reads a plain-1as a real (truthy) attribute value; for the hyperlink attribute this made a plainTextBoxget announced as a link.What is the current behavior?
TextBoxon Windows only implementsValuePattern. Screen readers get a single whole-text announcement on focus and no further caret/selection/edit feedback.What is the updated/expected behavior with this PR?
TextBoxon Windows now implementsTextPatternas well. To test: run a Windows build with NVDA (or another UIA-based screen reader) active, focus aTextBox, and confirm caret position, character/word navigation, and selection are announced correctly while typing and arrowing through text.How was the solution implemented (if it's not obvious)?
Three layers, built and verified in this order:
Avalonia.Controls) —ITextProvider/ITextRangeProviderinterfaces plus theTextBoxAutomationPeer/TextRangeimplementation. Covered by unit tests, no COM involved.Avalonia.Win32.Automation) —AutomationNode.Text.csimplements the existing (previously unused)UIA.ITextProviderinterop interface by delegating to layer 1 via the existingInvokeSyncdispatcher-marshaling helpers (now shared via the extractedWin32DispatcherHelper, sinceWin32TextRangeProviderisn't anAutomationNodeand needs the same marshaling).Win32TextRangeProviderwraps a singleITextRangeProviderinstance as itsUIA.ITextRangeProviderCOM counterpart.SafeArrayRef/ITextProvider's interop declaration, none of which were previously exercised by any existing pattern (ISelectionProvideretc. only ever return already-wrapped, long-livedAutomationNodeinstances, so theTryGetComInstance/missing-IID gaps never surfaced before).Root-caused the crash and the "announced as a link" misbehavior using
cdb(WinDbg's console debugger) attached to the live process, and verified the fix with NVDA actually running against a real build.Checklist
Breaking changes
None.
TextBoxAutomationPeergains new interface implementations (ITextProvider) but its existing public surface (IValueProvider) is unchanged.Obsoletions / Deprecations
None.
Fixed issues