|
| 1 | +# Skill: Inspect Windows application UI with lvt |
| 2 | + |
| 3 | +## When to use |
| 4 | + |
| 5 | +Use `lvt` whenever you need to understand the visual content or structure of a running Windows application. Common scenarios: |
| 6 | + |
| 7 | +- **UI verification** — confirm that a UI change was applied correctly (e.g. a button label changed, a dialog appeared) |
| 8 | +- **Finding UI elements** — locate a specific control, menu item, or text field in an app's visual tree |
| 9 | +- **Screenshot capture** — take an annotated screenshot of an app with element IDs overlaid |
| 10 | +- **Framework detection** — determine which UI frameworks an app uses (Win32, ComCtl, XAML, WinUI 3) |
| 11 | +- **Automated UI interaction planning** — get element IDs and bounds to plan mouse clicks or keyboard input |
| 12 | + |
| 13 | +## Prerequisites |
| 14 | + |
| 15 | +`lvt.exe` must be built and available. The default build output location is `build/lvt.exe` in the repository root. |
| 16 | + |
| 17 | +## Usage |
| 18 | + |
| 19 | +### Target an application |
| 20 | + |
| 21 | +You must specify exactly one target. Pick the most convenient option: |
| 22 | + |
| 23 | +```powershell |
| 24 | +# By process name (most common — omit .exe extension if you like) |
| 25 | +lvt --name notepad |
| 26 | +
|
| 27 | +# By window title substring |
| 28 | +lvt --title "Untitled - Notepad" |
| 29 | +
|
| 30 | +# By PID |
| 31 | +lvt --pid 1234 |
| 32 | +
|
| 33 | +# By HWND (hex) |
| 34 | +lvt --hwnd 0x1A0B3C |
| 35 | +``` |
| 36 | + |
| 37 | +### Get the visual tree |
| 38 | + |
| 39 | +```powershell |
| 40 | +# JSON output (default) — best for programmatic parsing |
| 41 | +lvt --name notepad |
| 42 | +
|
| 43 | +# XML output — more compact, easier to read |
| 44 | +lvt --name notepad --format xml |
| 45 | +
|
| 46 | +# Write to a file instead of stdout |
| 47 | +lvt --name notepad --output tree.json |
| 48 | +``` |
| 49 | + |
| 50 | +### Capture a screenshot |
| 51 | + |
| 52 | +```powershell |
| 53 | +# Screenshot only (no tree output) |
| 54 | +lvt --name notepad --screenshot out.png |
| 55 | +
|
| 56 | +# Screenshot + tree output together |
| 57 | +lvt --name notepad --screenshot out.png --dump |
| 58 | +``` |
| 59 | + |
| 60 | +Screenshots are annotated with element IDs (e0, e1, …) overlaid on each element, making it easy to correlate visual positions with tree nodes. |
| 61 | + |
| 62 | +### Scope to a subtree |
| 63 | + |
| 64 | +When the full tree is too large, scope to a specific element: |
| 65 | + |
| 66 | +```powershell |
| 67 | +# Only show element e5 and its descendants, up to 3 levels deep |
| 68 | +lvt --name myapp --element e5 --depth 3 |
| 69 | +``` |
| 70 | + |
| 71 | +### Detect frameworks only |
| 72 | + |
| 73 | +```powershell |
| 74 | +lvt --name notepad --frameworks |
| 75 | +``` |
| 76 | + |
| 77 | +## Interpreting the output |
| 78 | + |
| 79 | +### Element IDs |
| 80 | + |
| 81 | +Every element gets a stable ID like `e0`, `e1`, `e2`, etc., assigned in depth-first order. These IDs are consistent within a single invocation — use them to: |
| 82 | + |
| 83 | +- Reference specific elements in follow-up commands (`--element e5`) |
| 84 | +- Correlate screenshot annotations with tree nodes |
| 85 | +- Identify click targets by combining element ID with its `bounds` |
| 86 | + |
| 87 | +### Key element properties |
| 88 | + |
| 89 | +| Property | Description | |
| 90 | +|----------|-------------| |
| 91 | +| `id` | Stable element ID (e.g. `e0`) | |
| 92 | +| `type` | Element type name (e.g. `Window`, `Button`, `TextBlock`) | |
| 93 | +| `framework` | Which framework owns this element (`win32`, `comctl`, `xaml`, `winui3`) | |
| 94 | +| `className` | Win32 window class name (Win32/ComCtl elements) | |
| 95 | +| `text` | Visible text content or window title | |
| 96 | +| `bounds` | Screen-relative bounding rectangle `{x, y, width, height}` | |
| 97 | +| `children` | Nested child elements | |
| 98 | + |
| 99 | +### JSON example |
| 100 | + |
| 101 | +```json |
| 102 | +{ |
| 103 | + "target": { "hwnd": "0x001A0B3C", "pid": 12345, "processName": "Notepad.exe" }, |
| 104 | + "frameworks": ["win32", "winui3"], |
| 105 | + "root": { |
| 106 | + "id": "e0", |
| 107 | + "type": "Window", |
| 108 | + "framework": "win32", |
| 109 | + "className": "Notepad", |
| 110 | + "text": "Untitled - Notepad", |
| 111 | + "bounds": { "x": 100, "y": 100, "width": 800, "height": 600 }, |
| 112 | + "children": [ ... ] |
| 113 | + } |
| 114 | +} |
| 115 | +``` |
| 116 | + |
| 117 | +### XML example |
| 118 | + |
| 119 | +```xml |
| 120 | +<LiveVisualTree hwnd="0x001A0B3C" pid="12345" process="Notepad.exe" frameworks="win32,winui3"> |
| 121 | + <Window id="e0" framework="win32" className="Notepad" text="Untitled - Notepad" bounds="100,100,800,600"> |
| 122 | + <ContentPresenter id="e1" framework="winui3" bounds="108,140,784,552" /> |
| 123 | + </Window> |
| 124 | +</LiveVisualTree> |
| 125 | +``` |
| 126 | + |
| 127 | +## Recommended workflow |
| 128 | + |
| 129 | +1. **Start the target app** if it isn't already running |
| 130 | +2. **Run `lvt --name <app> --format xml`** to get a quick overview of the UI tree |
| 131 | +3. **Take a screenshot** with `lvt --name <app> --screenshot ui.png` to see the visual layout with element IDs |
| 132 | +4. **Drill into a subtree** with `--element <id> --depth <n>` if the tree is large |
| 133 | +5. **Use element IDs and bounds** to plan any UI interactions (clicks, keyboard input) |
| 134 | + |
| 135 | +## Tips |
| 136 | + |
| 137 | +- Use `--format xml` for human-readable output and `--format json` for programmatic parsing |
| 138 | +- If the tree is very large, use `--depth` to limit traversal depth first, then drill deeper with `--element` |
| 139 | +- Element IDs change between invocations if the UI structure changes — always re-query before acting on stale IDs |
| 140 | +- The tool requires no special permissions beyond being able to read the target process (same user session) |
| 141 | +- For XAML/WinUI 3 apps, lvt injects a helper DLL into the target — this is safe and non-destructive but means `lvt_tap.dll` must be next to `lvt.exe` |
0 commit comments