Unable to Access Certain Interactive Elements on Windows 11

[Jeomon]

I am using the uiautomation library on Windows 11 to interact with desktop elements. While the library works well for many tasks, I have encountered significant limitations in accessing certain interactive elements. Specifically:

1. Start Menu:

   • The library does not detect the elements within the Start Menu, such as the icons under the "Pinned" and "Recommended" sections.

2. Dropdown Menus in Applications (e.g., Notepad):

   • When I click on the "File" menu in Notepad, the elements in the dropdown menu are not accessible.

Steps to Reproduce

1. Install and configure the uiautomation library.
2. Attempt to interact with elements in the Windows 11 Start Menu or dropdown menus in applications like Notepad.
3. Observe the inability to access certain elements.

Expected Behavior

The library should be able to identify and provide access to all interactive elements, including those in:

• The Start Menu (e.g., icons in "Pinned" and "Recommended").
• Dropdown menus in applications (e.g., "File" menu in Notepad).

Actual Behavior

The library does not detect the mentioned elements, making them inaccessible for automation tasks.

Environment

• Operating System: Windows 11

Questions

1. Does this library support accessing such elements on Windows 11?
2. If not, is there any plan to enhance support for these elements?
3. Are there alternative approaches or configurations within the library that I might have missed?

[Mibharti28]

I have also faced same issue on windows 11. Library works well with windows 10 but when same code is executed on windows 11, it randomly fails giving comerror.

[yinkaisheng]

I can't reproduce the issue. It works on my Windows 11 Home Edition.

from uiautomation import uiautomation as auto

def startMenuPinnedList():
    auto.SendKeys('{Win}')
    startMenu = auto.WindowControl(searchDepth=1, ClassName='Windows.UI.Core.CoreWindow')
    listControl = startMenu.ListControl(searchDepth=3, AutomationId='PinnedList')
    for c in listControl.GetChildren():
        print(f'{c.Name}    {c.BoundingRectangle}')

if __name__ == '__main__':
    startMenuPinnedList()

def notepadMenu():
    notepadWindow = auto.WindowControl(searchDepth=1, ClassName='Notepad')
    notepadWindow.SendKeys('Notepad Menu Test')
    menubarControl = notepadWindow.MenuBarControl(searchDepth=1)
    menubarControl.MenuItemControl(searchDepth=1, SubName='文件').Click()
    fileMenu = notepadWindow.MenuControl(searchDepth=1)
    for menuItem in fileMenu.GetChildren():
        print(menuItem.Name)

[Jeomon]

Some issues have now been fixed by Microsoft, and this may be because of the update. (Windows 11) like the notepad issue I was talking about.

But there are some other areas like. for example, when you go to powerpoint. and if you try to access the elements inside the powerpoint, for example, let us say the title text box. If I was able to access it, I am. unable to access it actually straightforward like through searching or traversing the tree I cannot do it. I mean, because I believe that is because of there is a pane Control covering it.

So what I am proposing is that is the reason you were to penetrate that barrier and access the control. s behind it. This is 11 case I am saying and all. So if possible, it would be much, much helpful for me. I think it is helpful for everyone else, like.

You know about this live inspect functionality seen in the browser. So can you bring that kind of a similar functionality in here? Just nothing. Just. when you move the cursor around you can see everything like I mean like everything. What I mean is that whatever is hidden will be revealed like even if it is inaccessible by just moving the cursor over it, you can make that accessible just like even the power point. What I am saying.
This is something which I've seen in this inspect .exe like I. like I seen this obstructions. like in Powerpoint For example, this is I seen when I was using this inspect.exe. But when you use this watch cursor and focus cursor every hidden control are revealed.