|
| 1 | +# Automated UI Testing Design |
| 2 | + |
| 3 | +## Goal |
| 4 | + |
| 5 | +Minimize manual UI testing by automating ~74% of TESTING.md checklist items across three layers: pure Rust unit tests, GTK widget state tests, and a Python/dogtail pre-release integration suite. |
| 6 | + |
| 7 | +## Architecture: Three-Layer Testing Strategy |
| 8 | + |
| 9 | +### Layer 1: Pure Rust Unit Tests (every CI run, no display) |
| 10 | + |
| 11 | +Extract pure logic from GTK callbacks into standalone testable functions. Add CLI integration tests and property-based tests. |
| 12 | + |
| 13 | +**State model extraction** — new files `diff_state.rs` and `merge_state.rs`: |
| 14 | + |
| 15 | +| New pure function | Extracted from | Source location | |
| 16 | +|---|---|---| |
| 17 | +| `find_next_chunk(chunks, cursor_line, direction, side)` | `navigate_chunk` | common.rs:1567 | |
| 18 | +| `format_chunk_label(total, current)` | `update_chunk_label` | common.rs:1645 | |
| 19 | +| `find_conflict_markers_in_text(text)` | `find_conflict_markers` | merge_view.rs:181 | |
| 20 | +| `hit_test_gutter_arrows(x, y, width, ...)` | `handle_gutter_click` | common.rs:1358 | |
| 21 | +| `compute_chunk_map_rects(chunks, height, total_lines)` | `draw_chunk_map` | common.rs:1475 | |
| 22 | +| `plan_copy_chunk(chunks, idx, direction, left_text, right_text)` | `copy_chunk` | common.rs:1265 | |
| 23 | +| `find_all_matches(text, needle)` | `highlight_search_matches` | common.rs:1424 | |
| 24 | +| `swap_state(left, right)` involution | swap callbacks | various | |
| 25 | + |
| 26 | +**CLI integration tests** (`tests/cli_integration.rs`): |
| 27 | +- Test all 15 CLI/Launch Mode items via `std::process::Command` |
| 28 | +- Assert exit codes and stderr output for each argument combination |
| 29 | + |
| 30 | +**Extended property-based tests**: |
| 31 | +- Unified diff format validity roundtrip |
| 32 | +- Navigation invariants (next chunk always past cursor) |
| 33 | +- `filter_for_diff` + `remap_chunks` roundtrip |
| 34 | + |
| 35 | +### Layer 2: `#[gtk::test]` Widget State Tests (every CI run, needs Xvfb) |
| 36 | + |
| 37 | +Create GTK widgets programmatically and verify their state: |
| 38 | + |
| 39 | +- `build_diff_view` with test data: verify tag placement, button sensitivity, action groups |
| 40 | +- `build_merge_view` with 3-way data: verify pane read-only state, conflict overlay |
| 41 | +- TextBuffer manipulation: verify save button goes sensitive on edit |
| 42 | +- Keybinding registration: verify action names exist in action groups |
| 43 | +- Identical/binary file detection: verify info bar creation, button disabled states |
| 44 | + |
| 45 | +**File**: `tests/widget_tests.rs` |
| 46 | + |
| 47 | +### Layer 3: Dogtail + Screenshot Suite (pre-release only) |
| 48 | + |
| 49 | +**Dogtail** (Python AT-SPI) tests in `tests/ui_integration/`: |
| 50 | +- File diff workflow: open files, verify chunk count, navigate, copy chunk, save |
| 51 | +- Directory comparison: open dir, expand tree, double-click file, verify tab |
| 52 | +- 3-way merge: copy from left/right gutter, verify middle buffer |
| 53 | +- VCS window: open repo, double-click file, verify diff tab |
| 54 | +- Keyboard shortcuts: Ctrl+F, Ctrl+S, Alt+Up/Down, Ctrl+J/K, etc. |
| 55 | + |
| 56 | +**Screenshot regression**: |
| 57 | +- Capture views via `WidgetPaintable` -> `Renderer` -> `Texture` -> PNG |
| 58 | +- Compare against reference images using `image-compare` crate (SSIM) |
| 59 | +- Fuzzy threshold to tolerate font/rendering differences |
| 60 | + |
| 61 | +## Project Structure |
| 62 | + |
| 63 | +``` |
| 64 | +src/ui/ |
| 65 | + diff_state.rs # NEW - pure diff logic (extracted from common.rs/diff_view.rs) |
| 66 | + merge_state.rs # NEW - pure merge logic (extracted from merge_view.rs) |
| 67 | + common.rs # MODIFIED - extracted fns become thin wrappers |
| 68 | + diff_view.rs # MODIFIED - delegates to diff_state |
| 69 | + merge_view.rs # MODIFIED - delegates to merge_state |
| 70 | +
|
| 71 | +tests/ |
| 72 | + vcs_integration.rs # existing |
| 73 | + cli_integration.rs # NEW - CLI argument tests |
| 74 | + widget_tests.rs # NEW - #[gtk::test] widget state tests |
| 75 | +
|
| 76 | +tests/ui_integration/ # NEW - Python dogtail suite |
| 77 | + conftest.py # pytest fixtures (launch app, AT-SPI connection) |
| 78 | + test_file_diff.py |
| 79 | + test_dir_compare.py |
| 80 | + test_merge.py |
| 81 | + test_vcs.py |
| 82 | + test_shortcuts.py |
| 83 | + fixtures/ # test data files |
| 84 | + left.txt / right.txt / base.txt |
| 85 | + left_dir/ / right_dir/ |
| 86 | + binary_file.bin |
| 87 | + identical_a.txt / identical_b.txt |
| 88 | +
|
| 89 | +tests/screenshots/ # NEW - visual regression |
| 90 | + refs/ # reference PNGs |
| 91 | +``` |
| 92 | + |
| 93 | +## CI Changes |
| 94 | + |
| 95 | +### Existing CI (`.github/workflows/ci.yml`) |
| 96 | + |
| 97 | +Add xvfb and font packages for widget tests: |
| 98 | + |
| 99 | +```yaml |
| 100 | +- name: Install packages |
| 101 | + run: sudo apt-get update && sudo apt-get install -y |
| 102 | + build-essential libgtksourceview-5-dev xvfb fonts-dejavu-core |
| 103 | + |
| 104 | +- name: Run tests |
| 105 | + env: |
| 106 | + GDK_BACKEND: x11 |
| 107 | + GTK_A11Y: none |
| 108 | + run: xvfb-run -a cargo test --all-targets --locked |
| 109 | +``` |
| 110 | +
|
| 111 | +### New pre-release workflow (`.github/workflows/ui-tests.yml`) |
| 112 | + |
| 113 | +Manual dispatch for pre-release validation: |
| 114 | + |
| 115 | +```yaml |
| 116 | +- Install: xvfb, fonts, python3, pip, at-spi2-registryd, dogtail |
| 117 | +- Build: cargo build --release |
| 118 | +- Run: xvfb-run pytest tests/ui_integration/ -v |
| 119 | +- Run: xvfb-run cargo test --test screenshot_tests |
| 120 | +- Upload: screenshot diff artifacts on failure |
| 121 | +``` |
| 122 | + |
| 123 | +### Convenience targets (`Makefile`) |
| 124 | + |
| 125 | +```makefile |
| 126 | +test: cargo test |
| 127 | +test-ui: xvfb-run -a cargo test |
| 128 | +test-release: xvfb-run -a pytest tests/ui_integration/ && xvfb-run -a cargo test --test screenshot_tests |
| 129 | +``` |
| 130 | + |
| 131 | +## Coverage Map |
| 132 | + |
| 133 | +| Section | Items | Automated | Rate | |
| 134 | +|---|---|---|---| |
| 135 | +| CLI / Launch Modes | 15 | 15 | 100% | |
| 136 | +| Welcome Window | 6 | 3 | 50% | |
| 137 | +| File Diff (all subsections) | 45 | 35 | 78% | |
| 138 | +| Navigation | 4 | 4 | 100% | |
| 139 | +| Directory Comparison | 30 | 18 | 60% | |
| 140 | +| 3-Way Merge | 20 | 13 | 65% | |
| 141 | +| VCS Window | 15 | 11 | 73% | |
| 142 | +| Preferences | 12 | 6 | 50% | |
| 143 | +| Global Shortcuts | 20 | 18 | 90% | |
| 144 | +| **Total** | **~167** | **~123** | **~74%** | |
| 145 | + |
| 146 | +### Items that remain manual (~26%) |
| 147 | + |
| 148 | +These are genuinely hard to automate in the GTK4 ecosystem: |
| 149 | +- **File dialog interactions**: OS-native portal dialogs, can't drive from within process |
| 150 | +- **Visual rendering judgment**: Chunk background colors, gutter curve aesthetics, filler line positions, inline highlight colors |
| 151 | +- **Timing-dependent behavior**: File watcher debouncing (500ms poll), save suppression (600ms window), background diff responsiveness |
| 152 | +- **Cross-platform specifics**: macOS dark mode detection, Windows build behavior |
| 153 | +- **Scroll smoothness**: Subjective UX quality that requires human judgment |
| 154 | + |
| 155 | +## Key Dependencies |
| 156 | + |
| 157 | +### Rust (dev-dependencies) |
| 158 | +- `proptest` (existing) - property-based testing |
| 159 | +- `tempfile` (existing) - temporary files/dirs |
| 160 | +- `image-compare` (new) - SSIM screenshot comparison |
| 161 | +- `image` (new) - PNG encoding/decoding |
| 162 | + |
| 163 | +### Python (pre-release only) |
| 164 | +- `dogtail` >= 1.0 - AT-SPI UI testing framework |
| 165 | +- `pytest` - test runner |
| 166 | + |
| 167 | +### System (CI) |
| 168 | +- `xvfb` - virtual X11 display |
| 169 | +- `fonts-dejavu-core` - consistent font rendering |
| 170 | +- `at-spi2-registryd` - accessibility bus (for dogtail) |
| 171 | + |
| 172 | +## Implementation Order |
| 173 | + |
| 174 | +1. **Extract pure logic** into `diff_state.rs` / `merge_state.rs` + unit tests |
| 175 | +2. **CLI integration tests** in `tests/cli_integration.rs` |
| 176 | +3. **Property-based test extensions** for unified diff, navigation, filter/remap |
| 177 | +4. **CI xvfb setup** + `#[gtk::test]` widget state tests |
| 178 | +5. **Dogtail test suite** scaffold + fixture files |
| 179 | +6. **Screenshot regression** harness + initial reference images |
| 180 | +7. **Makefile** + pre-release workflow |
0 commit comments