Skip to content

Commit 9d3c6f8

Browse files
grandixximograndixximo
committed
test: add UI smoke tests for axis, touchy, gmoccapy, qtdragon
Adds a minimal harness under tests/ui-smoke/ that launches each GUI against its sim config under xvfb-run and verifies it reaches the 'task ready' NML state without crashing. Auto-discovered by scripts/runtests via per-GUI test.sh + checkresult + skip files. Layout: _lib/launch.sh - spawns linuxcnc -r under xvfb, runs driver, handles clean shutdown (group-SIGTERM with 60s wait, escalate to SIGKILL + shm cleanup) _lib/drive.py - polls linuxcnc.stat() until task ready, prints UI_SMOKE_OK / UI_SMOKE_FAIL _lib/checkresult.sh - grep for UI_SMOKE_OK / absence of FAIL _lib/skip-if-missing.sh - skip when xvfb-run absent (dev env) _lib/cleanup-runtime.sh - pre/post belt-and-braces daemon + shm cleanup; SHM key list mirrors scripts/runtests:157 (full 6-key set) _lib/run-gui.sh - dispatcher taking a relpath under configs/sim/, exec'd by per-GUI test.sh axis|touchy|gmoccapy|qtdragon/test.sh - one-line wrappers Force software OpenGL via LIBGL_ALWAYS_SOFTWARE + Qt RHI/QSG/QtQuick software backends; CI runners have no GPU and Qt GL paths segfault on headless display. Skip vs fail policy (BsAtHome / hdiethelm review): only xvfb-run absence skips; missing Python/typelib deps fail loudly so review catches them. Required deps are gated under !nocheck in debian/control.top.in (separate commit).
1 parent f6bfe9c commit 9d3c6f8

20 files changed

Lines changed: 318 additions & 0 deletions

File tree

‎tests/ui-smoke/.gitignore‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Runtime artifacts left by launch.sh; per Bertho's clean-tree rule,
2+
# generated build/test outputs must be gitignored or committed.
3+
linuxcnc.out
4+
linuxcnc.err
5+
linuxcnc.pid
6+
ui-smoke.out
7+
ui-smoke.err

‎tests/ui-smoke/README‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
UI smoke tests
2+
~~~~~~~~~~~~~~
3+
4+
These tests launch each GUI (axis, touchy, gmoccapy, qtdragon) under
5+
xvfb-run against an existing sim config and verify the GUI starts and
6+
the NML task is reachable. Phase 1 ("does it start"); functional
7+
checks (load G-code, jog, MDI) belong in tests/ui-functional/.
8+
9+
Each test directory contains:
10+
test.sh launches the GUI under xvfb-run, runs drive.py
11+
checkresult examines the captured output for crash markers
12+
skip skips this test only when xvfb-run is not on the host
13+
14+
Shared helpers live in _lib/:
15+
drive.py common NML driver, prints UI_SMOKE_OK on success
16+
launch.sh xvfb-run wrapper, signal escalation for shutdown
17+
cleanup-runtime.sh belt-and-suspenders: kill stray daemons, ipcrm
18+
shared memory, drop /tmp/linuxcnc.lock
19+
checkresult.sh shared pass/fail predicate
20+
skip-if-missing.sh shared skip predicate
21+
22+
Skip vs fail policy: the only condition we skip on is xvfb-run absence
23+
(rare local dev env). Python and gi typelib deps the GUIs need are
24+
declared in debian/control under !nocheck so apt-get build-dep
25+
installs them on CI; if they are missing the test should fail loudly
26+
rather than silently skip, so missing deps surface during review.

‎tests/ui-smoke/_lib/checkresult.sh‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
#!/bin/bash
2+
# Shared result check for UI smoke tests.
3+
#
4+
# Pass if the driver printed UI_SMOKE_OK and did not print UI_SMOKE_FAIL.
5+
# The driver only emits UI_SMOKE_OK after a successful NML round-trip
6+
# (linuxcnc task ready and stat.poll() still alive after settle), so
7+
# this is sufficient evidence that the GUI booted. We do not grep for
8+
# generic crash markers like "Segmentation fault" or "Traceback":
9+
# linuxcnc's own scripts/linuxcnc Cleanup may emit shutdown-side
10+
# segfaults (Qt/GTK teardown races) that are out of scope for a
11+
# startup smoke test, and the driver-signal approach catches the
12+
# startup-time failures we actually care about.
13+
set -u
14+
15+
if [ $# -lt 1 ]; then
16+
echo "FAIL: checkresult requires the result-log path as argument" >&2
17+
exit 1
18+
fi
19+
20+
LOG="$1"
21+
22+
if grep -q '^UI_SMOKE_FAIL' "$LOG"; then
23+
echo "FAIL: driver reported UI_SMOKE_FAIL" >&2
24+
exit 1
25+
fi
26+
27+
if ! grep -q '^UI_SMOKE_OK$' "$LOG"; then
28+
echo "FAIL: driver did not report UI_SMOKE_OK" >&2
29+
exit 1
30+
fi
31+
32+
exit 0

‎tests/ui-smoke/_lib/cleanup-runtime.sh‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
#!/bin/bash
2+
# Reset linuxcnc-related runtime state so the next ui-smoke test starts
3+
# from a clean environment. Used as both a pre-launch belt-and-braces
4+
# cleanup and as a post-shutdown last-resort if scripts/linuxcnc's own
5+
# SIGTERM trap could not reap everything in time.
6+
#
7+
# SHM_KEYS mirrors SHMEM_KEY in scripts/runtests:157. If a ui-smoke
8+
# crash leaks any of these, the next runtests invocation aborts in
9+
# test_shmem(); we must clean the full set.
10+
11+
set -u
12+
13+
DAEMONS=(linuxcncsvr milltask halui rtapi_app)
14+
SHM_KEYS=(0x00000064 0x48414c32 0x48484c34 0x90280a48 0x130cf406 0x434c522b)
15+
16+
for proc in "${DAEMONS[@]}"; do
17+
pkill -KILL -x "$proc" 2>/dev/null || true
18+
done
19+
20+
rm -f /tmp/linuxcnc.lock
21+
halrun -U 2>/dev/null || true
22+
23+
for key in "${SHM_KEYS[@]}"; do
24+
shmid=$(LC_ALL=C ipcs -m | awk -v k="$key" 'tolower($1)==k {print $2}')
25+
if [ -n "$shmid" ]; then
26+
ipcrm -m "$shmid" 2>/dev/null || true
27+
fi
28+
done
29+
30+
exit 0

‎tests/ui-smoke/_lib/drive.py‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
#!/usr/bin/env python3
2+
# Minimal UI smoke driver: confirm linuxcnc task came up and the GUI
3+
# did not crash. The smoke layer answers Bertho's "does it start"
4+
# question only; functional behaviour (home, run a file, verify
5+
# position) belongs in tests/ui-functional/ (Phase 2).
6+
7+
import linuxcnc
8+
import sys
9+
import time
10+
11+
CONNECT_TIMEOUT_S = 60.0
12+
SETTLE_S = 3.0
13+
14+
15+
def connect_and_wait_ready(timeout):
16+
"""Wait until linuxcnc.stat().poll() returns without error and
17+
reports a non-negative echo_serial_number. The NML status buffer
18+
can be 'invalid err=3' for the first ~30s while linuxcncsvr is
19+
still initialising; recreate the stat object on every iteration so
20+
a stale invalid buffer does not stick after linuxcncsvr is ready."""
21+
deadline = time.monotonic() + timeout
22+
last_err = None
23+
while time.monotonic() < deadline:
24+
try:
25+
stat = linuxcnc.stat()
26+
stat.poll()
27+
if stat.echo_serial_number >= 0:
28+
return linuxcnc.command(), stat
29+
except linuxcnc.error as e:
30+
last_err = e
31+
time.sleep(0.5)
32+
sys.stderr.write(
33+
f"UI_SMOKE_FAIL: task not ready within {timeout}s "
34+
f"(last NML error: {last_err})\n")
35+
return None, None
36+
37+
38+
def main():
39+
cmd, stat = connect_and_wait_ready(CONNECT_TIMEOUT_S)
40+
if cmd is None:
41+
return 1
42+
43+
# Give the GUI process enough time to finish constructing itself
44+
# (load .ui files, compile resources.py if needed, etc.) and
45+
# settle. If the GUI was going to crash on startup it has crashed
46+
# by now.
47+
time.sleep(SETTLE_S)
48+
49+
# Re-check task is still alive; a GUI crash may have torn linuxcnc
50+
# down via Cleanup.
51+
try:
52+
stat.poll()
53+
except linuxcnc.error as e:
54+
sys.stderr.write(f"UI_SMOKE_FAIL: task disappeared after GUI startup: {e}\n")
55+
return 1
56+
57+
print("UI_SMOKE_OK")
58+
return 0
59+
60+
61+
if __name__ == "__main__":
62+
sys.exit(main())

‎tests/ui-smoke/_lib/launch.sh‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
#!/bin/bash
2+
# Shared launcher for UI smoke tests.
3+
# Usage: launch.sh <sim-config-ini>
4+
#
5+
# Spawns linuxcnc -r <ini> under xvfb-run, then runs the common driver
6+
# script against it via NML. Captures stdout/stderr to per-test files.
7+
#
8+
# Skip vs fail (BsAtHome / hdiethelm review, PR #3999): xvfb-run absence
9+
# is handled by the per-test skip files (skip-if-missing.sh), which
10+
# runtests gates on before invoking test.sh. CI is expected to have all
11+
# required deps; if any python module the GUI needs is missing the test
12+
# should fail loudly rather than silently skip. Per-GUI deps are
13+
# declared in debian/control under !nocheck, so apt-get build-dep
14+
# installs them on CI.
15+
16+
set -u
17+
18+
CONFIG_INI="$1"
19+
TEST_DIR="${TEST_DIR:-$(cd "$(dirname "$0")" && pwd)}"
20+
LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
21+
22+
cd "$TEST_DIR" || exit 1
23+
rm -f ui-smoke.out ui-smoke.err linuxcnc.pid
24+
25+
# Pre-launch cleanup: a previous ui-smoke test in the same job may
26+
# have left a daemon listening on the NML TCP port or HAL shared
27+
# memory still attached. Run the shared cleanup once before we start.
28+
bash "$LIB_DIR/cleanup-runtime.sh"
29+
30+
# Launch linuxcnc inside xvfb-run. The outer timeout is a safety net
31+
# so a wedged GUI cannot hang CI.
32+
LINUXCNC_TIMEOUT=240
33+
DRIVER_TIMEOUT=90
34+
35+
# Force software OpenGL (Mesa llvmpipe). CI runners have no GPU and
36+
# Qt/GL widgets segfault under hardware GL with no display. The Qt-
37+
# specific knobs cover qtdragon's QtQuick + RHI paths.
38+
export LIBGL_ALWAYS_SOFTWARE=1
39+
export GALLIUM_DRIVER=llvmpipe
40+
export QT_QUICK_BACKEND=software
41+
export QSG_RHI_BACKEND=software
42+
export QT_OPENGL=software
43+
44+
# Export the per-invocation values so the inner bash -c receives them
45+
# as proper env vars (avoids embedding paths into the inner script
46+
# via quoting, which breaks on apostrophes / spaces).
47+
export CONFIG_INI LIB_DIR DRIVER_TIMEOUT
48+
49+
# Single quotes around the inner script are intentional: CONFIG_INI,
50+
# LIB_DIR and DRIVER_TIMEOUT are expanded by the inner bash (which sees
51+
# them via the exported env), not by the outer shell.
52+
# shellcheck disable=SC2016
53+
xvfb-run -a --server-args="-screen 0 1024x768x24" \
54+
timeout "$LINUXCNC_TIMEOUT" \
55+
bash -c '
56+
# Run linuxcnc in its own process group so we can signal the
57+
# whole group cleanly (linuxcnc forks task, motion, GUI, halrun).
58+
# setsid makes the child a session leader, so its PID equals
59+
# the PGID and we can group-signal via "kill -- -$PID".
60+
setsid linuxcnc -r "$CONFIG_INI" >linuxcnc.out 2>linuxcnc.err &
61+
LINUXCNC_PID=$!
62+
echo "$LINUXCNC_PID" >linuxcnc.pid
63+
64+
# The driver polls NML readiness itself (BsAtHome review:
65+
# avoid real-clock waits where status polling will do).
66+
timeout "$DRIVER_TIMEOUT" python3 "$LIB_DIR/drive.py" >ui-smoke.out 2>ui-smoke.err
67+
DRIVE_RC=$?
68+
69+
# Clean shutdown: GUI-specific quit first (lets linuxcnc end
70+
# its own SIGTERM trap run Cleanup which unloads halrun and
71+
# reaps shared memory). axis-remote works only for axis but is
72+
# harmless otherwise. Then group-SIGTERM so the trap runs
73+
# in-process. Wait up to 60s for Cleanup to finish before
74+
# falling back to SIGKILL + cleanup-runtime.sh.
75+
if command -v axis-remote >/dev/null 2>&1; then
76+
axis-remote --quit 2>/dev/null || true
77+
fi
78+
79+
kill -TERM -- -"$LINUXCNC_PID" 2>/dev/null || true
80+
for _ in $(seq 60); do
81+
kill -0 "$LINUXCNC_PID" 2>/dev/null || break
82+
sleep 1
83+
done
84+
if kill -0 "$LINUXCNC_PID" 2>/dev/null; then
85+
echo "WARN: linuxcnc did not exit on SIGTERM, escalating to KILL" >&2
86+
kill -KILL -- -"$LINUXCNC_PID" 2>/dev/null || true
87+
sleep 2
88+
bash "$LIB_DIR/cleanup-runtime.sh"
89+
fi
90+
91+
exit "$DRIVE_RC"
92+
'
93+
RC=$?
94+
95+
# Surface logs so checkresult and CI artifact upload can see them.
96+
echo "=== linuxcnc.out ==="
97+
[ -f linuxcnc.out ] && cat linuxcnc.out
98+
echo "=== linuxcnc.err ==="
99+
[ -f linuxcnc.err ] && cat linuxcnc.err
100+
echo "=== ui-smoke.out ==="
101+
[ -f ui-smoke.out ] && cat ui-smoke.out
102+
echo "=== ui-smoke.err ==="
103+
[ -f ui-smoke.err ] && cat ui-smoke.err
104+
105+
exit "$RC"

‎tests/ui-smoke/_lib/run-gui.sh‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/bin/bash
2+
# Dispatcher invoked from each per-GUI test.sh. Resolves an INI path
3+
# under configs/sim/ and execs launch.sh in the caller's test dir.
4+
# Usage: run-gui.sh <relpath-under-configs/sim>
5+
# e.g. run-gui.sh axis/axis.ini
6+
# run-gui.sh qtdragon/qtdragon_xyz/qtdragon_metric.ini
7+
8+
set -u
9+
10+
LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
11+
TEST_DIR="${TEST_DIR:-$(cd "$(dirname "$0")" && pwd)}"
12+
CONFIGS_DIR="$(cd "$LIB_DIR/../../../configs/sim" && pwd)"
13+
14+
export TEST_DIR
15+
exec "$LIB_DIR/launch.sh" "$CONFIGS_DIR/$1"

‎tests/ui-smoke/_lib/skip-if-missing.sh‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
#!/bin/bash
2+
# Shared "skip" predicate for ui-smoke tests.
3+
# runtests semantics: a `skip` script that returns non-zero causes the
4+
# test to be skipped. Per-test skip files invoke this.
5+
#
6+
# We only skip on xvfb-run absence (rare local dev env). Python /
7+
# typelib deps are declared in debian/control under !nocheck so CI
8+
# always has them; missing deps should fail the test loudly rather
9+
# than silently skip (BsAtHome / hdiethelm review, PR #3999).
10+
set -u
11+
12+
if ! command -v xvfb-run >/dev/null 2>&1; then
13+
echo "skip: xvfb-run not installed" >&2
14+
exit 1
15+
fi
16+
17+
exit 0

‎tests/ui-smoke/axis/checkresult‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
#!/bin/bash
2+
exec "$(dirname "$0")/../_lib/checkresult.sh" "$@"

‎tests/ui-smoke/axis/skip‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
#!/bin/bash
2+
exec "$(dirname "$0")/../_lib/skip-if-missing.sh"

0 commit comments

Comments
 (0)