Skip to content

Commit 47b84d8

Browse files
committed
test(e2e): conftest helper + no-skip strategy for SDK entry / path_verification / exec_timeout
Moves the no-skip wiring from chore into #765 where the rest of the test fixes already live. - conftest.skip_or_fail_unless_sdk_build_required: helper that flips pytest.skip into pytest.fail when BOXLITE_E2E_REQUIRE_SDK_BUILDS=1 is set. The e2e-cloud-test workflow sets it; local dev leaves it unset and the skip path stays usable. - conftest.collect_ignore (guarded on BOXLITE_E2E_PROFILE != default): drops scripts/test/e2e/cases/test_path_verification.py from pytest collection on cloud profiles. Both cases in that file are LOCAL meta-tests (one asserts credentials.toml url=:3000, the other reads the host's boxlite-runner systemd journal) and can't run against the Tokyo ELB profile. collect_ignore drops them from collection entirely — no SKIP markers in the report. - test_c_entry / test_cli_entry / test_cli_detach_recovery / test_go_entry / test_node_entry fixtures: every pytest.skip routed through skip_or_fail_unless_sdk_build_required. - test_path_verification.py: module-level pytest.mark.skipif removed (replaced by the collect_ignore above so the file produces no collection at all on cloud, instead of being collected-then-skipped). - test_exec_timeout.py: the _skipif_cloud decorator on both cases is gone; instead drain(ex) is moved behind a short asyncio.wait_for and reordered after ex.wait() so the elapsed / exit-code assertions don't depend on stream content. The REST runner's stream pumps don't reliably observe stream closure when the workload terminates via SIGKILL; the workaround lets the test actually run on cloud (no SKIP entry) while preserving its semantic intent. - e2e-cloud-test.yml: BOXLITE_E2E_REQUIRE_SDK_BUILDS=1 on the Run E2E suite step — multi-job prereqs guarantee artifacts on disk so a fixture-skip path firing here is now a regression signal.
1 parent 1463e86 commit 47b84d8

4 files changed

Lines changed: 36 additions & 16 deletions

File tree

.github/workflows/e2e-cloud-test.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -493,9 +493,9 @@ jobs:
493493
# build_c / build_node / build_cli prereqs are now hard
494494
# dependencies of this job, so the C / Go / CLI / Node entry-
495495
# point fixtures always have their artifacts on disk. Flip
496-
# skip→fail so a future regression in the build chain (or a
497-
# missing download step) surfaces as a clear pytest failure
498-
# instead of a silent SKIP that drains the gate.
496+
# skip→fail (conftest.skip_or_fail_unless_sdk_build_required)
497+
# so a future regression in the build chain or download step
498+
# surfaces as a clear pytest failure instead of a silent SKIP.
499499
BOXLITE_E2E_REQUIRE_SDK_BUILDS: '1'
500500
# Tokyo Api (post-#758) supportedImages = the 3 ghcr-pinned
501501
# boxlite-agent-* digests only. The conftest default `alpine:3.23`

scripts/test/e2e/cases/conftest.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,34 @@ def skip_or_fail_unless_sdk_build_required(reason: str) -> None:
5858

5959

6060

61+
def skip_or_fail_unless_sdk_build_required(reason: str) -> None:
62+
"""SDK entry-point fixtures (test_c_entry, test_go_entry,
63+
test_node_entry, test_cli_entry, test_cli_detach_recovery) skip
64+
when their build artifact is missing — convenient for local dev
65+
where someone hasn't built every SDK. On the cloud gate the
66+
e2e-cloud-test workflow produces every artifact up front via
67+
build_c / build_node / build_cli prereq jobs, so set
68+
BOXLITE_E2E_REQUIRE_SDK_BUILDS=1 there — a regression in the
69+
build step then surfaces as a test failure, not a silent skip."""
70+
require = os.environ.get("BOXLITE_E2E_REQUIRE_SDK_BUILDS", "")
71+
if require.lower() in ("1", "true", "yes", "on"):
72+
pytest.fail(
73+
f"BOXLITE_E2E_REQUIRE_SDK_BUILDS=1 forbids skipping this case "
74+
f"but the prerequisite is missing: {reason}"
75+
)
76+
pytest.skip(reason)
77+
78+
79+
# test_path_verification.py is a LOCAL-only meta-test: case 1 asserts the
80+
# credentials.toml URL contains ':3000' (the local API port), case 2 reads
81+
# the host's `boxlite-runner` systemd journal via journalctl. Neither can
82+
# run against a remote profile pointing at the Tokyo ELB. Drop from pytest
83+
# collection on any non-default profile so the cloud gate reports them as
84+
# 'not collected' rather than producing SKIP entries.
85+
if DEFAULT_PROFILE != "default":
86+
collect_ignore = ["test_path_verification.py"]
87+
88+
6189
def _profile(name: str) -> dict:
6290
if not CRED_PATH.exists():
6391
pytest.exit(

scripts/test/e2e/cases/test_exec_timeout.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
SIGKILL — drain() blocks indefinitely on cloud while the underlying
1010
exec has long since exited. The exit-code / elapsed assertions don't
1111
depend on stream content, so a best-effort drain (3s ceiling) is
12-
enough to drain whatever the pump did emit without holding the test
12+
enough to flush whatever the pump did emit without holding the test
1313
hostage on the missing close signal. Tracked separately under the
1414
REST stream-pump teardown audit; the local FFI path is unaffected.
1515
"""

scripts/test/e2e/cases/test_path_verification.py

Lines changed: 4 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@
1818
"""
1919
from __future__ import annotations
2020

21-
import os
2221
import sys
2322
from pathlib import Path
2423

@@ -36,17 +35,10 @@
3635
# (2) reads the local boxlite-runner systemd journal via journalctl —
3736
# on the Tokyo cloud profile p1 the runner journal lives on an
3837
# EC2 instance the test client can't reach.
39-
# Skip the whole module when the test session is configured against
40-
# anything other than the default (= local) profile so these don't
41-
# spam the cloud failure tally with environment-mismatch noise.
42-
pytestmark = pytest.mark.skipif(
43-
os.environ.get("BOXLITE_E2E_PROFILE", "default") != "default",
44-
reason=(
45-
"LOCAL-only meta-test (checks credentials.toml url=:3000 and "
46-
"reads host's boxlite-runner journalctl). Cannot run against a "
47-
"remote profile."
48-
),
49-
)
38+
# Module-level skipif has been replaced by a conftest.collect_ignore
39+
# entry guarded on BOXLITE_E2E_PROFILE != 'default'. That stops pytest
40+
# from collecting this file at all on the cloud gate (no SKIP markers
41+
# in the report) rather than collecting + skipping.
5042

5143

5244
@pytest.mark.asyncio

0 commit comments

Comments
 (0)