|
| 1 | +--- |
| 2 | +name: run-spread-test |
| 3 | +description: Run spread integration tests for snapd. Tests are slow - collect as much information as possible in each run to avoid wasting cycles. |
| 4 | +metadata: |
| 5 | + project: snapd |
| 6 | + task-type: test |
| 7 | + execution-time: slow |
| 8 | +--- |
| 9 | + |
| 10 | +## Run spread tests |
| 11 | + |
| 12 | +Command: `./run-spread [spread-flags] <backend>:<system>:<test-path>` |
| 13 | + |
| 14 | +Tests take 5-15 minutes each. Maximize information per run. |
| 15 | + |
| 16 | +## Prerequisites |
| 17 | + |
| 18 | +Build the snap first (see `build-snapd-snap` skill), then use `NO_REBUILD=1`. |
| 19 | +If no local snap exists: either build one, or use `./run-spread --download` to fetch from master. |
| 20 | + |
| 21 | +## Quick reference |
| 22 | + |
| 23 | +```bash |
| 24 | +# Standard (snap already built) |
| 25 | +NO_REBUILD=1 ./run-spread -debug garden:ubuntu-24.04-64:tests/main/snap-mgmt |
| 26 | + |
| 27 | +# First run (auto-builds snap) |
| 28 | +./run-spread -debug garden:ubuntu-24.04-64:tests/main/snap-mgmt |
| 29 | + |
| 30 | +# Download prebuilt from master (test-only changes) |
| 31 | +./run-spread --download -debug garden:ubuntu-24.04-64:tests/main/snap-mgmt |
| 32 | + |
| 33 | +# Reuse VMs for faster iteration |
| 34 | +NO_REBUILD=1 ./run-spread -reuse garden:ubuntu-24.04-64:tests/main/snap-mgmt |
| 35 | +``` |
| 36 | + |
| 37 | +Always specify `<backend>:<system>:<test-path>` — bare `./run-spread` runs everything. |
| 38 | + |
| 39 | +## Flags |
| 40 | + |
| 41 | +- `-debug`: Get SSH shell on failure. Use by default. |
| 42 | +- `-reuse`: Keep VMs alive between runs (faster iteration). |
| 43 | + |
| 44 | +## Backend and system |
| 45 | + |
| 46 | +Always use `garden`. Do not use `qemu`, `google-*`, `openstack` etc. unless user explicitly says so. |
| 47 | + |
| 48 | +System selection: |
| 49 | +- AppArmor changes → recent Ubuntu LTS, optionally Debian |
| 50 | +- SELinux changes → openSUSE SELinux or Fedora |
| 51 | +- Systemd changes → recent Ubuntu + Fedora |
| 52 | +- General → most recent Ubuntu LTS |
| 53 | + |
| 54 | +Discover system names from `spread.yaml` under `backends:garden:systems:`. Don't hardcode — they change. |
| 55 | + |
| 56 | +## Suites that don't need the snap |
| 57 | + |
| 58 | +Use `spread` directly (not `run-spread`) for: |
| 59 | +- `tests/unit/` — Go unit tests |
| 60 | +- `tests/cross/` — cross-compilation |
| 61 | + |
| 62 | +```bash |
| 63 | +spread garden:ubuntu-24.04-64:tests/unit/go |
| 64 | +``` |
| 65 | + |
| 66 | +All other suites (`tests/main/`, `tests/nested/`, `tests/regression/`) require the snap — use `run-spread`. |
| 67 | + |
| 68 | +## Parallel execution |
| 69 | + |
| 70 | +Single invocation is serial on garden (1 worker). For true parallelism, delegate to subagents — max 4 concurrent (each VM uses 3-4 GB RAM). |
| 71 | + |
| 72 | +## Test path patterns |
| 73 | + |
| 74 | +```bash |
| 75 | +tests/main/snap-mgmt # Single test |
| 76 | +tests/main/interfaces-... # Glob |
| 77 | +tests/main/... # All in directory |
| 78 | +``` |
| 79 | + |
| 80 | +## Failure output structure |
| 81 | + |
| 82 | +``` |
| 83 | +Error executing garden:<system>:<test> : |
| 84 | +----- |
| 85 | +<execute section output — shows the failing commands> |
| 86 | +----- |
| 87 | +Debug output for garden:<system>:<test> : |
| 88 | +----- |
| 89 | +<debug section output — extra diagnostic info> |
| 90 | +----- |
| 91 | +Starting shell to debug... |
| 92 | +``` |
| 93 | + |
| 94 | +Capture as much of the error and debug output as possible to avoid re-runs. |
| 95 | + |
| 96 | +## Interpreting failures |
| 97 | + |
| 98 | +Phases: prepare → execute → restore → debug (on failure) |
| 99 | + |
| 100 | +- Execute failure: look for MATCH/NOMATCH assertions (`grep -qE` wrappers from `tests/lib/`) |
| 101 | +- Prepare failure: missing deps or system incompatibility |
| 102 | +- Restore failure: incomplete cleanup (fix to avoid pollution) |
| 103 | + |
| 104 | +Debug shell useful commands: |
| 105 | +```bash |
| 106 | +journalctl -u snapd |
| 107 | +snap changes |
| 108 | +snap tasks <change-id> |
| 109 | +``` |
| 110 | + |
| 111 | +## spread.yaml structure |
| 112 | + |
| 113 | +Key sections: `environment:` (global vars), `backends:` (garden/google/qemu configs), `backends:<backend>:systems:` (available systems), `suites:` (test suite configs). Individual tests can override settings in their `task.yaml`. |
| 114 | + |
| 115 | +## Rules |
| 116 | + |
| 117 | +- Always use `-debug` to avoid re-runs on failure |
| 118 | +- Always set `NO_REBUILD=1` when snap exists |
| 119 | +- Never run `./run-spread` without a test path |
| 120 | +- Never use non-garden backends unless instructed |
| 121 | +- Max 4 parallel subagent tests (RAM) |
| 122 | +- When unsure if suite needs snap, assume yes — build first |
| 123 | +- Verify `built-snap/snapd_*.snap.keep` exists before running with `NO_REBUILD=1` |
0 commit comments