Skip to content

Commit bd1e24f

Browse files
brianmcgillionbrianmcgillion
committed
cleaning and cleaning
Signed-off-by: Brian McGillion <bmg.avoin@gmail.com>
1 parent 2dc2e1b commit bd1e24f

37 files changed

+1183
-86
lines changed

‎.github/skills/ghaf-hw-test/README.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# SPDX-FileCopyrightText: 2022-2026 TII (SSRC) and the Ghaf contributors
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
# Ghaf Hardware Test Automation Skill
5+
6+
A CLI skill for running Robot Framework hardware tests on Ghaf devices with automatic fix proposals.
7+
8+
## Quick Start
9+
10+
```bash
11+
# Check device connectivity
12+
.github/skills/ghaf-hw-test/ghaf-hw-test status --device darter-pro --ip 192.168.1.100
13+
14+
# Run tests
15+
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip 192.168.1.100
16+
17+
# Analyze failures
18+
.github/skills/ghaf-hw-test/ghaf-hw-test analyze --propose-fixes
19+
20+
# Full test-fix loop
21+
.github/skills/ghaf-hw-test/ghaf-hw-test run --device darter-pro --ip 192.168.1.100 --fix-loop
22+
```
23+
24+
## Supported Devices
25+
26+
- `darter-pro` - System76 Darter Pro (x86_64)
27+
- `Orin-AGX` - NVIDIA Jetson AGX Orin (aarch64)
28+
- `Orin-NX` - NVIDIA Jetson Orin NX (aarch64)
29+
- `Lenovo-X1` - Lenovo ThinkPad X1 (x86_64)
30+
- `dell-7330` - Dell Latitude 7330 (x86_64)
31+
- `NUC` - Intel NUC (x86_64)
32+
33+
## Commands
34+
35+
| Command | Description |
36+
|---------|-------------|
37+
| `status` | Check device connectivity |
38+
| `test` | Run Robot Framework tests |
39+
| `flash` | Flash Ghaf image to device |
40+
| `analyze` | Parse results, propose fixes |
41+
| `run` | Full test-fix loop |
42+
43+
## Copilot CLI Integration
44+
45+
This skill is designed to be used with GitHub Copilot CLI. It appears in `/skills list` and can be invoked to guide automated testing workflows.
46+
47+
See [SKILL.md](SKILL.md) for full documentation.
48+
49+
## Files
50+
51+
```
52+
.github/skills/ghaf-hw-test/
53+
├── SKILL.md # Skill definition (AI instructions)
54+
├── README.md # This file
55+
├── config.yaml # Device profiles
56+
├── ghaf-hw-test # CLI executable
57+
├── shell.nix # Nix environment
58+
├── lib/
59+
│ └── result_parser.py # Output parser & analyzer
60+
└── templates/
61+
├── fix_proposal.md.j2
62+
└── test_report.md.j2
63+
```

‎.github/skills/ghaf-hw-test/SKILL.md‎

Lines changed: 213 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
<!--
2+
SPDX-FileCopyrightText: 2022-2026 TII (SSRC) and the Ghaf contributors
3+
SPDX-License-Identifier: Apache-2.0
4+
-->
5+
---
6+
name: ghaf-hw-test
7+
description: Run hardware tests on Ghaf devices (darter-pro, jetson-agx) with automatic fix proposals. Use this skill to flash devices, run Robot Framework tests, analyze failures, and generate code fixes.
8+
license: Apache-2.0 (SPDX-FileCopyrightText 2022-2026 TII (SSRC) and the Ghaf contributors)
9+
---
10+
11+
# Ghaf Hardware Test Automation Skill
12+
13+
This skill automates the hardware testing workflow for Ghaf Framework devices using the ci-test-automation Robot Framework test suite. It supports running tests on locally connected devices, flashing images, analyzing test failures, and proposing code fixes.
14+
15+
## When to Use This Skill
16+
17+
Use this skill when the user wants to:
18+
- Run hardware tests on a connected Ghaf device (darter-pro, jetson-agx, lenovo-x1, etc.)
19+
- Flash a new Ghaf image to a device
20+
- Analyze test failures and identify root causes
21+
- Generate fix proposals for failing tests
22+
- Run a test-fix loop until all tests pass
23+
24+
## Supported Devices
25+
26+
| Device | Architecture | Ghaf Target | Test Device Name |
27+
|--------|-------------|-------------|------------------|
28+
| System76 Darter Pro | x86_64 | system76-darp11-b-debug | darter-pro |
29+
| NVIDIA Jetson AGX Orin | aarch64 | nvidia-jetson-orin-agx-debug | Orin-AGX |
30+
| NVIDIA Jetson Orin NX | aarch64 | nvidia-jetson-orin-nx-debug | Orin-NX |
31+
| Lenovo ThinkPad X1 | x86_64 | lenovo-x1-carbon-gen11-debug | Lenovo-X1 |
32+
| Dell Latitude 7330 | x86_64 | dell-latitude-7330-debug | dell-7330 |
33+
| Intel NUC | x86_64 | generic-x86_64-debug | NUC |
34+
35+
## Available Commands
36+
37+
Run these commands from the Ghaf repository root:
38+
39+
```bash
40+
# Check device connectivity and readiness
41+
.github/skills/ghaf-hw-test/ghaf-hw-test status --device darter-pro --ip 192.168.1.100
42+
43+
# Run tests on a device
44+
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip 192.168.1.100
45+
.github/skills/ghaf-hw-test/ghaf-hw-test test --device Orin-AGX --ip 192.168.1.101 --tag boot
46+
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip 192.168.1.100 --tag pre-merge
47+
48+
# Flash a device with a new image
49+
.github/skills/ghaf-hw-test/ghaf-hw-test flash --device darter-pro --drive /dev/sda
50+
.github/skills/ghaf-hw-test/ghaf-hw-test flash --device Orin-AGX --recovery
51+
52+
# Analyze test results and propose fixes
53+
.github/skills/ghaf-hw-test/ghaf-hw-test analyze
54+
.github/skills/ghaf-hw-test/ghaf-hw-test analyze --results /tmp/test_results
55+
.github/skills/ghaf-hw-test/ghaf-hw-test analyze --propose-fixes
56+
57+
# Full workflow: test → analyze → fix → repeat
58+
.github/skills/ghaf-hw-test/ghaf-hw-test run --device darter-pro --ip 192.168.1.100 --fix-loop
59+
```
60+
61+
## Typical Workflow
62+
63+
### 1. Check Device Connectivity
64+
65+
Before running tests, verify the device is reachable:
66+
67+
```bash
68+
.github/skills/ghaf-hw-test/ghaf-hw-test status --device darter-pro --ip 192.168.1.100
69+
```
70+
71+
This checks SSH connectivity and reports device readiness.
72+
73+
### 2. Run Tests
74+
75+
Run the pre-merge test suite (default):
76+
77+
```bash
78+
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip 192.168.1.100
79+
```
80+
81+
Or run specific test tags:
82+
83+
```bash
84+
# Boot tests only
85+
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip 192.168.1.100 --tag boot
86+
87+
# Performance tests
88+
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip 192.168.1.100 --tag performance
89+
```
90+
91+
### 3. Analyze Failures
92+
93+
If tests fail, analyze the results:
94+
95+
```bash
96+
.github/skills/ghaf-hw-test/ghaf-hw-test analyze --propose-fixes
97+
```
98+
99+
This parses `/tmp/test_results/output.xml` and:
100+
- Identifies failing tests
101+
- Maps failures to Ghaf module paths
102+
- Generates actionable fix proposals
103+
104+
### 4. Apply Fixes and Re-run
105+
106+
Review the proposed fixes, apply them, then re-run tests:
107+
108+
```bash
109+
# Edit the files suggested in the fix proposals
110+
# Then re-run tests
111+
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip 192.168.1.100
112+
```
113+
114+
### 5. Fix Loop Mode (Automated)
115+
116+
For automated test-fix iterations:
117+
118+
```bash
119+
.github/skills/ghaf-hw-test/ghaf-hw-test run --device darter-pro --ip 192.168.1.100 --fix-loop
120+
```
121+
122+
This will:
123+
1. Run tests
124+
2. If failures occur, analyze and output fix proposals
125+
3. Wait for fixes to be applied
126+
4. Re-run tests
127+
5. Repeat until all tests pass or max iterations reached
128+
129+
## Flashing Devices
130+
131+
### x86 Devices (darter-pro, lenovo-x1, dell-7330, nuc)
132+
133+
```bash
134+
# Build the image first
135+
nix build .#system76-darp11-b-debug
136+
137+
# Flash to USB/internal drive
138+
.github/skills/ghaf-hw-test/ghaf-hw-test flash --device darter-pro --drive /dev/sda
139+
```
140+
141+
### Jetson Devices (Orin-AGX, Orin-NX)
142+
143+
```bash
144+
# Build the flash script
145+
nix build .#nvidia-jetson-orin-agx-debug-flash-script
146+
147+
# Put device in recovery mode (hold recovery button while powering on)
148+
# Then run guided flash
149+
.github/skills/ghaf-hw-test/ghaf-hw-test flash --device Orin-AGX --recovery
150+
```
151+
152+
## Test Tags Reference
153+
154+
| Tag | Description |
155+
|-----|-------------|
156+
| `pre-merge` | Quick validation tests (default) |
157+
| `bat` | Basic Acceptance Tests |
158+
| `boot` | Boot and connectivity tests |
159+
| `functional` | VM, host, networking, apps |
160+
| `gui` | Desktop/GUI functionality |
161+
| `performance` | Benchmarks, boot time |
162+
| `security` | Security validation |
163+
| `suspension` | Suspend/resume cycles |
164+
| `update` | OTA update verification |
165+
166+
## Configuration
167+
168+
Device profiles and defaults are stored in `.github/skills/ghaf-hw-test/config.yaml`.
169+
170+
You can override defaults via command-line arguments or by editing the config file.
171+
172+
## Output Files
173+
174+
Test results are stored in `/tmp/test_results/`:
175+
- `output.xml` - Robot Framework results (parsed by `analyze`)
176+
- `log.html` - Detailed test log
177+
- `report.html` - Test summary report
178+
179+
## Integration with smoke-test Devshell
180+
181+
This skill wraps the existing `robot-test` command from the smoke-test devshell. You can also use the devshell directly:
182+
183+
```bash
184+
nix develop .#smoke-test
185+
robot-test -i 192.168.1.100 -d darter-pro -p ghaf
186+
```
187+
188+
## Tips for AI Agents
189+
190+
When operating in fix-loop mode:
191+
192+
1. **Read the fix proposals** generated by `analyze --propose-fixes`
193+
2. **Locate the relevant Ghaf modules** using the paths in the proposals
194+
3. **Make minimal, targeted fixes** - change only what's needed
195+
4. **Re-run the specific failing tests** first before running the full suite
196+
5. **Check the test code** in ci-test-automation if the failure is unclear
197+
198+
### Common Failure Patterns
199+
200+
| Failure Type | Likely Cause | Where to Look |
201+
|-------------|--------------|---------------|
202+
| SSH connection timeout | Network/firewall | modules/common/networking/ |
203+
| Service not running | systemd unit issue | modules/common/services/ |
204+
| GUI test failure | Desktop/graphics | modules/desktop/ |
205+
| VM communication failure | GIVC configuration | modules/givc/ |
206+
| Performance regression | Config or hardware | modules/hardware/ |
207+
208+
## Prerequisites
209+
210+
- Ghaf repository cloned
211+
- Device connected via network (SSH) or serial
212+
- For flashing: physical access to device
213+
- Nix with flakes enabled

0 commit comments

Comments
 (0)