Follow-up: UITest framework Next - stability improvements (#49242)

This pull request addresses several CI flakiness and reliability issues
in the UI test automation harness, especially around foreground window
handling and build artifact resolution. The main improvements ensure
that UI interactions (like real mouse clicks) reliably target the
correct window, even in complex CI and build environments, and that test
code accurately locates module executables regardless of output layout.

**Foreground window handling and input reliability:**

* Added `Session.EnsureForeground()` and improved
`WindowControl.TryBringToForeground()` to reliably raise the target
window above others, defeating the Win32 foreground lock using
`AttachThreadInput` and related APIs. This prevents coordinate-based
clicks from landing on the wrong window, a common cause of CI test
flakiness. (`src/common/UITestAutomation.Next/Session.cs`,
`src/common/UITestAutomation.Next/WindowControl.cs`,
`src/common/UITestAutomation.Next/Element/Element.cs`,
`.github/skills/ui-tests-migration/references/ci-stability.md`,
`src/modules/MeasureTool/Tests/ScreenRuler.UITests.Next/TestHelper.cs`)
[[1]](diffhunk://#diff-713524f4ead9951000b3578248f2740088729981051b68c3fa3655ebb548f20fR91-R118)
[[2]](diffhunk://#diff-60be6ec24a99f8974cff00414fc47ea1e60e3ab9aee706b5d6845eb8df19165fR59-R76)
[[3]](diffhunk://#diff-60be6ec24a99f8974cff00414fc47ea1e60e3ab9aee706b5d6845eb8df19165fL247-R332)
[[4]](diffhunk://#diff-3c00581cb1d2b6a4302b8378b87c27cdad2f87c2aefc37eec0bdc6c39af9291aR100-R103)
[[5]](diffhunk://#diff-98e5aea12baaac99aba10e7dd341ed3f2efdd24cb80b29306bb500816de09e2fR96-R114)
[[6]](diffhunk://#diff-d8766b48614fe4a99e3f100e69e611ca2464de6ba0be257586c12d51f9278ed0L498-R504)

**Build artifact and executable path resolution:**

* Refactored `ModuleInfo.GetDevelopmentPath()` to dynamically walk up
from the test assembly and find the module executable in various build
output layouts, including CI artifacts and local builds. This eliminates
hardcoded path offsets and makes test launches robust to different build
structures. (`src/common/UITestAutomation/ModuleInfo.cs`)
* Updated `ModuleConfigData.GetModulePath()` to use the improved path
resolution and fall back to installed builds if necessary, ensuring that
test runs can always locate the correct executable.
(`src/common/UITestAutomation/ModuleConfigData.cs`)
* Simplified `SessionHelper` so that `locationPath` is always empty,
leveraging the fact that module paths are now absolute.
(`src/common/UITestAutomation/SessionHelper.cs`)
[[1]](diffhunk://#diff-c001f0fd3432c8b6102c3120597fc48f24902eaa0d755f51adfa7646344b3833L7-L10)
[[2]](diffhunk://#diff-c001f0fd3432c8b6102c3120597fc48f24902eaa0d755f51adfa7646344b3833L46-R54)

These changes collectively improve test reliability, especially in CI
environments, and make the harness more resilient to changes in build
output structure.

---------

Co-authored-by: Boliang Zhang (from Dev Box) <bozhang@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: 3 people

.github/skills/ui-tests-migration/references/ci-stability.md

@@ -93,6 +93,25 @@ navigating with `By.AccessibilityId(...).Click()`** (Recipe 1) — it's race-saf
 reach for a raw `MouseClick`/manual `MouseHelper` when the interaction genuinely needs real mouse input
 (and by then the window has settled).
 
+**The second trap: a background-launched window comes up *behind* the foreground.** A physical click
+lands on whatever window is **topmost at those pixels** — not necessarily your target. When a module's
+overlay/toolbar is shown by a *background* process (the runner, reacting to a hotkey) while another
+window holds the foreground, Windows' **foreground lock** puts it *behind* that window — it's present,
+`IsWindowVisible`-true, and un-cloaked, yet occluded. A coordinate click then hits the covering window
+and looks **exactly** like the interactivity race, but it's occlusion. This is a prime "passes local,
+flakes on CI" cause: on CI the Settings window used to enable the module is still foreground when the
+overlay appears. The harness guards against it — `Element.Click()` calls `Session.EnsureForeground()`
+first, which raises the target with the foreground-lock-defeating `AttachThreadInput` dance
+(`WindowControl.TryBringToForeground`) before the real click, and still falls back to `Invoke()` if the
+raise doesn't take. Diagnose it via winappcli's `isForeground` flag on `list-windows`; UIA `invoke` is
+immune because it never touches coordinates.
+
+**Elevation must match.** Injected input — a synthetic hotkey *or* a real click — from a process at a
+*different* integrity level than the PowerToys runner is blocked by UIPI: a non-elevated host can't
+drive an elevated runner, and an elevated host's foreground window blocks the non-elevated runner's
+hook. Run the test host at the **same** elevation as the runner (the `.Next` harness launches the
+runner non-elevated, so run the tests non-elevated too).
+
 ---
 
 ## Principle 3 — Screen-capture (WGC) modules: cold-start + don't disturb the session

src/common/UITestAutomation.Next/Element/Element.cs

@@ -97,6 +97,10 @@ public virtual void Click(bool rightClick = false, int msPostAction = 200, int t
         {
             if (Width > 0 && Height > 0)
             {
+                // Raise the target window first: a physical click lands on whatever window is topmost
+                // at these pixels, and a background-launched overlay/toolbar is often placed behind the
+                // window that held the foreground when it was triggered (the Win32 foreground lock).
+                Owner!.EnsureForeground();
                 MouseHelper.MoveTo(X + (Width / 2), Y + (Height / 2));
                 Thread.Sleep(50);
                 if (rightClick)

src/common/UITestAutomation.Next/Session.cs

@@ -88,6 +88,34 @@ private Session(PowerToysModule scope, string appNameOrPid, int pid, string proc
         ProcessName = processName;
     }
 
+    /// <summary>
+    /// Best-effort: raise this session's target window to the foreground so a subsequent coordinate
+    /// interaction (a real mouse click/drag) can't land on a window that happens to be covering it. A
+    /// background-launched overlay/toolbar is often placed BEHIND whatever held the foreground when it
+    /// was triggered (the Win32 foreground lock), so <see cref="Element.Click"/> calls this before its
+    /// physical click. Window-scoped sessions raise their exact HWND; process-scoped sessions raise the
+    /// app's first window. Never throws — a foreground failure must not fail the interaction (the click
+    /// still has its coordinate-free <c>Invoke</c> fallback).
+    /// </summary>
+    public void EnsureForeground()
+    {
+        try
+        {
+            if (Scope == TargetScope.Window && WindowHandle != 0)
+            {
+                WindowControl.TryBringToForeground(new IntPtr(WindowHandle));
+            }
+            else if (Scope == TargetScope.Process)
+            {
+                WindowControl.TryFocusByApp(TargetValue);
+            }
+        }
+        catch
+        {
+            // Best-effort foregrounding only.
+        }
+    }
+
     /// <summary>
     /// Build a session scoped to a whole process via <c>winapp ... -a &lt;app&gt;</c>. Cheaper than
     /// resolving a HWND and ideal for the single-window-per-process case (e.g. Settings smoke

src/common/UITestAutomation.Next/WindowControl.cs

@@ -56,6 +56,24 @@ public static class WindowControl
     [return: MarshalAs(UnmanagedType.Bool)]
     private static extern bool GetWindowRect(IntPtr hWnd, out RECT lpRect);
 
+    [DllImport("user32.dll")]
+    private static extern IntPtr GetForegroundWindow();
+
+    [DllImport("user32.dll", SetLastError = true)]
+    [return: MarshalAs(UnmanagedType.Bool)]
+    private static extern bool AttachThreadInput(uint idAttach, uint idAttachTo, [MarshalAs(UnmanagedType.Bool)] bool fAttach);
+
+    [DllImport("user32.dll", SetLastError = true)]
+    [return: MarshalAs(UnmanagedType.Bool)]
+    private static extern bool BringWindowToTop(IntPtr hWnd);
+
+    [DllImport("user32.dll")]
+    [return: MarshalAs(UnmanagedType.Bool)]
+    private static extern bool IsIconic(IntPtr hWnd);
+
+    [DllImport("kernel32.dll")]
+    private static extern uint GetCurrentThreadId();
+
     private delegate bool EnumWindowsProc(IntPtr hWnd, IntPtr lParam);
 
     private const uint WM_CLOSE = 0x0010;
@@ -244,24 +262,74 @@ public static bool TryCloseByApp(string appNameOrPid, Func<WindowsFinder.WindowI
     /// Bring the first window owned by <paramref name="appNameOrPid"/> to the foreground.
     /// If the window is minimized it's first restored. Tolerant.
     /// </summary>
-    public static bool TryFocusByApp(string appNameOrPid)
+    /// <summary>
+    /// Bring <paramref name="hwnd"/> to the foreground RELIABLY, defeating the Win32 foreground lock.
+    /// A bare <c>SetForegroundWindow</c> from a process that isn't already the foreground is silently
+    /// refused (SPI_SETFOREGROUNDLOCKTIMEOUT) — which is exactly what leaves a freshly-shown overlay /
+    /// toolbar sitting BEHIND the window that held the foreground when it was triggered, so a coordinate
+    /// click then lands on the covering window instead of the target. Briefly attaching our input queue
+    /// to the current foreground thread lifts that restriction for the call. Best-effort; tolerant.
+    /// </summary>
+    public static bool TryBringToForeground(IntPtr hwnd)
     {
         try
         {
-            var w = WindowsFinder.ListByApp(appNameOrPid).FirstOrDefault();
-            if (w is null || w.Hwnd == 0)
+            if (hwnd == IntPtr.Zero || !IsWindow(hwnd))
             {
                 return false;
             }
 
-            var hwnd = new IntPtr(w.Hwnd);
-            if (!IsWindow(hwnd))
+            var foreground = GetForegroundWindow();
+            if (foreground == hwnd)
+            {
+                // Already the foreground window — don't touch its show-state or the input queues.
+                return true;
+            }
+
+            var foregroundThread = foreground == IntPtr.Zero ? 0u : GetWindowThreadProcessId(foreground, out _);
+            var currentThread = GetCurrentThreadId();
+
+            // Only un-MINIMIZE. Do NOT SW_RESTORE a maximized window — that un-maximizes it, resizing the
+            // window and invalidating any element coordinates already resolved for the click (the cause of
+            // the arm64 "Settings got resized then the toggle click missed" flakiness).
+            if (IsIconic(hwnd))
+            {
+                ShowWindow(hwnd, SW_RESTORE);
+            }
+
+            var attached = foregroundThread != 0 && foregroundThread != currentThread;
+            if (attached)
+            {
+                AttachThreadInput(currentThread, foregroundThread, true);
+            }
+
+            BringWindowToTop(hwnd);
+            var ok = SetForegroundWindow(hwnd);
+
+            if (attached)
+            {
+                AttachThreadInput(currentThread, foregroundThread, false);
+            }
+
+            return ok;
+        }
+        catch
+        {
+            return false;
+        }
+    }
+
+    public static bool TryFocusByApp(string appNameOrPid)
+    {
+        try
+        {
+            var w = WindowsFinder.ListByApp(appNameOrPid).FirstOrDefault();
+            if (w is null || w.Hwnd == 0)
             {
                 return false;
             }
 
-            ShowWindow(hwnd, SW_RESTORE);
-            return SetForegroundWindow(hwnd);
+            return TryBringToForeground(new IntPtr(w.Hwnd));
         }
         catch
         {

src/common/UITestAutomation/ModuleConfigData.cs

@@ -153,7 +153,17 @@ public string GetModulePath(PowerToysModule scope)
                 }
             }
 
-            return moduleInfo.GetDevelopmentPath();
+            var developmentPath = moduleInfo.GetDevelopmentPath();
+            if (File.Exists(developmentPath))
+            {
+                return developmentPath;
+            }
+
+            // Last resort: an installed build if one is present, so a slim/installed CI layout that didn't
+            // set useInstallerForTest still resolves. Returns the development path either way so a launch
+            // failure names a concrete location.
+            var installedFallback = moduleInfo.GetInstalledPath(GetPowerToysInstallPath());
+            return File.Exists(installedFallback) ? installedFallback : developmentPath;
         }
 
         public string GetWindowsApplicationDriverUrl() => WindowsApplicationDriverUrl;

src/common/UITestAutomation/ModuleInfo.cs

@@ -26,23 +26,69 @@ public ModuleInfo(string executableName, string windowName, string? subDirectory
         }
 
         /// <summary>
-        /// Gets the relative development path for this module
+        /// Resolves the ABSOLUTE path to this module's executable in the build under test by walking up
+        /// from the test assembly's output folder and returning the first ancestor that actually contains
+        /// the exe. This adapts to any build-output nesting — the local flattened
+        /// <c>&lt;root&gt;\&lt;plat&gt;\&lt;cfg&gt;\tests\&lt;project&gt;\&lt;tfm&gt;\</c> layout AND the deeper
+        /// per-project CI artifact layout — instead of hard-coding the walk-up depth. The old fixed
+        /// <c>\..\..\..</c> offset resolved to a non-existent path on CI, so every legacy test failed to
+        /// launch PowerToys there; this mirrors the .Next harness's resolver. When no ancestor holds the
+        /// exe, it returns the historical fixed-depth path (made absolute) so a launch failure still names
+        /// a concrete location.
         /// </summary>
         public string GetDevelopmentPath()
         {
-            // The test assembly normally lives in <buildRoot>\tests\<project>\<tfm>\, so the build
-            // output root that holds the module exe is three levels above it. When a test project is
-            // built with a RuntimeIdentifier (OutputType=Exe for the MTP runner) the output gains an
-            // extra RID subfolder (<tfm>\win-x64\ or \win-arm64\), pushing the root one level further
-            // up. Detect that case so the relative path stays correct in both layouts.
-            string prefix = IsRuntimeIdentifierOutputFolder() ? @"\..\..\..\.." : @"\..\..\..";
+            // 1) Walk up and return the first ancestor that DIRECTLY holds the exe. Covers the flattened
+            //    <root>\<plat>\<cfg>\tests\<project>\<tfm>\ layout and the CI artifact layout where the
+            //    product's output dir is itself an ancestor of the (deeply nested) test assembly.
+            for (var dir = new DirectoryInfo(AppContext.BaseDirectory); dir is not null; dir = dir.Parent)
+            {
+                var candidate = ComposeUnder(dir.FullName);
+                if (File.Exists(candidate))
+                {
+                    return candidate;
+                }
+            }
 
-            if (string.IsNullOrEmpty(SubDirectory))
+            // 2) Per-project NESTED layout: the test builds under <repoRoot>\src\...\<project>\<plat>\<cfg>\
+            //    while the product sits at <repoRoot>\<plat>\<cfg>\ — not a direct ancestor. Probe the
+            //    conventional <plat>\<cfg> output beneath each ancestor, preferring this run's own.
+            foreach (var platCfg in BuildOutputSubdirs())
             {
-                return $@"{prefix}\{ExecutableName}";
+                for (var dir = new DirectoryInfo(AppContext.BaseDirectory); dir is not null; dir = dir.Parent)
+                {
+                    var candidate = ComposeUnder(Path.Combine(dir.FullName, platCfg));
+                    if (File.Exists(candidate))
+                    {
+                        return candidate;
+                    }
+                }
             }
 
-            return $@"{prefix}\{SubDirectory}\{ExecutableName}";
+            // 3) Fallback: the historical fixed-depth walk-up, resolved to an absolute path so a launch
+            //    failure still names a concrete location instead of a bare relative fragment.
+            var prefix = IsRuntimeIdentifierOutputFolder() ? @"..\..\..\.." : @"..\..\..";
+            var baseDir = AppContext.BaseDirectory.TrimEnd(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar);
+            return Path.GetFullPath(Path.Combine(baseDir, prefix, RelativeExe()));
+        }
+
+        // The exe (optionally under its module subdirectory) directly beneath a build-output root.
+        private string ComposeUnder(string root) =>
+            string.IsNullOrEmpty(SubDirectory)
+                ? Path.Combine(root, ExecutableName)
+                : Path.Combine(root, SubDirectory, ExecutableName);
+
+        private string RelativeExe() =>
+            string.IsNullOrEmpty(SubDirectory) ? ExecutableName : Path.Combine(SubDirectory, ExecutableName);
+
+        // Conventional <plat>\<cfg> build-output subdirs, ordered to prefer this test run's own so a
+        // Debug run resolves the Debug product even when a stale Release build is also present.
+        private static string[] BuildOutputSubdirs()
+        {
+            var all = new[] { @"x64\Debug", @"x64\Release", @"ARM64\Debug", @"ARM64\Release" };
+            var baseDir = AppContext.BaseDirectory.Replace('/', '\\');
+            var current = all.FirstOrDefault(pc => baseDir.Contains("\\" + pc + "\\", StringComparison.OrdinalIgnoreCase));
+            return current is null ? all : new[] { current }.Concat(all.Where(pc => pc != current)).ToArray();
         }
 
         // True when the executing assembly sits in a RID-specific output subfolder (e.g. ...\<tfm>\win-x64),

src/common/UITestAutomation/SessionHelper.cs

@@ -4,10 +4,8 @@
 
 using System;
 using System.Diagnostics;
-using System.Diagnostics.CodeAnalysis;
 using System.IO;
 using System.Linq;
-using System.Reflection;
 using System.Threading.Tasks;
 using Microsoft.VisualStudio.TestTools.UnitTesting;
 using OpenQA.Selenium.Appium;
@@ -43,14 +41,17 @@ public class SessionHelper
         /// </summary>
         private bool UseInstallerForTest { get; }
 
-        [UnconditionalSuppressMessage("SingleFile", "IL3000:Avoid accessing Assembly file path when publishing as a single file", Justification = "<Pending>")]
         public SessionHelper(PowerToysModule scope, string[]? commandLineArgs = null)
         {
             this.scope = scope;
             this.commandLineArgs = commandLineArgs;
             this.sessionPath = ModuleConfigData.Instance.GetModulePath(scope);
             UseInstallerForTest = EnvironmentConfig.UseInstallerForTest;
-            this.locationPath = UseInstallerForTest ? string.Empty : Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
+
+            // GetModulePath now returns an ABSOLUTE path (adaptive dev-build walk-up or the installed
+            // path), so no per-assembly prefix is needed; locationPath stays empty and the launch-path
+            // concatenations below resolve to that absolute path.
+            this.locationPath = string.Empty;
 
             CheckWinAppDriverAndRoot();
         }

src/modules/MeasureTool/Tests/ScreenRuler.UITests.Next/TestHelper.cs

@@ -495,7 +495,13 @@ private static void SelectToolAndVerify(Session ruler, string buttonId, string t
             {
                 var btnX = button.X + (button.Width / 2);
                 var btnY = button.Y + (button.Height / 2);
-                Log($"SelectToolAndVerify[{testName}]: attempt {attempt}: located {buttonId} at ({button.X},{button.Y}) {button.Width}x{button.Height} (offscreen={button.IsOffscreen}, toggle={toggleState}); real mouse click at ({btnX},{btnY})");
+                Log($"SelectToolAndVerify[{testName}]: attempt {attempt}: located {buttonId} at ({button.X},{button.Y}) {button.Width}x{button.Height} (offscreen={button.IsOffscreen}, toggle={toggleState}); raising toolbar to foreground then real mouse click at ({btnX},{btnY})");
+
+                // Raise the toolbar to the foreground first so the real click can't land on a window
+                // occluding it (the Win32 foreground lock). Bounds selection uses a manual MouseHelper
+                // click rather than Element.Click, so — unlike spacing — it needs this guard explicitly.
+                // TryBringToForeground only un-minimizes (IsIconic), so a maximized toolbar isn't resized.
+                ruler.EnsureForeground();
                 MouseHelper.MoveTo(btnX, btnY);
                 Thread.Sleep(200);
                 MouseHelper.LeftClick();