Skip to content

refactor(java): implement strategy pattern for Appium WebView, Native, and Chromium contexts#360

Open
rookieInTraining wants to merge 3 commits into
alumnium-hq:mainfrom
rookieInTraining:appium-java-fixes
Open

refactor(java): implement strategy pattern for Appium WebView, Native, and Chromium contexts#360
rookieInTraining wants to merge 3 commits into
alumnium-hq:mainfrom
rookieInTraining:appium-java-fixes

Conversation

@rookieInTraining
Copy link
Copy Markdown
Collaborator

Summary

  • Refactors AppiumDriver to use the Strategy pattern, dispatching every action and accessibility snapshot to the correct view strategy based on the current Appium context at call-time :
(NATIVE_APP → NativeAppiumViewStrategy, WEBVIEW_* → WebViewAppiumViewStrategy, CHROMIUM → ChromiumAppiumViewStrategy or WebViewAppiumViewStrategy fallback when CDP is unavailable).
  • Introduces WebViewAccessibilityTree, which parses the HTML page source in a WebView context and assigns stable CSS-selector-based locators so the AI can interact with web content via standard WebElement APIs.
  • Fixes several correctness issues uncovered during Android system testing: XML serialization emitting HTML entities, invalid CSS id-selectors for numeric IDs, live input values not reflected in getPageSource(), HTML5 drag-and-drop events not firing from touch gestures, and elements not scrolled into view before typing.

Type of Change

Mark the relevant options.

  • Bug fix
  • New feature (non-breaking changes, test coverage, refactoring)
  • Breaking change (fix, refactoring, or feature that would cause existing functionality to change)
  • Cleanup (documentation, formatting)

Code or UI Demos (if applicable)

N/A

Checklist

Please verify the following before submitting:

  • I have read the Contributing Guidelines
  • I have added or updated relevant documentation
  • I have written or updated necessary tests
  • I have tested the changes locally
  • The changes follow the project's coding style and structure
  • I have not included any sensitive or credential-related information
  • I have ensured these changes maintain or improve accessibility (for UI changes)
  • I have considered security implications of these changes

Additional Notes

Currently untested for iOS devices. Plus drag and drop fallsback to a JS dispatch event script if there are any exceptions thrown during interaction.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this doing? Parses HTML and tries to construct an accessibility tree out of it? Why not rely on iOS/Android accessibility tree returned by Appium pageSource call? That's what we do in Python/TypeScript.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Appium expose CDP commands? Is it only for Android or does it work on iOS as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants