Skip to content

Commit df50574

Browse files
committed
replay: sweep -H / -H -m / -H -j per fixture
The harness only ran `kasld -v`, so the hardening renderers had no per-arch, real-fixture coverage. After the main run, re-run each fixture with -H (text), -H -m (markdown) and -H -j (json) — all three go through the shared build_hardening_report() model — and require each to finish without an emulation signal and emit its hardening section header.
1 parent 3068891 commit df50574

1 file changed

Lines changed: 34 additions & 5 deletions

File tree

tests/replay

Lines changed: 34 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,10 @@
55
# fixtures.
66
#
77
# For each fixture under tests/fixtures/<arch>/<host>/, reconstruct a scratch
8-
# sysroot from its captured filesystem state and run `kasld -v` over it. The
9-
# check is that kasld RUNS to completion — parses the fixture, resolves the
10-
# engine, renders a summary — without crashing. There is no in-process oracle to
8+
# sysroot from its captured filesystem state and run `kasld -v` over it, then
9+
# re-run with -H / -H -m / -H -j to sweep the hardening renderers. The check is
10+
# that kasld RUNS to completion — parses the fixture, resolves the engine,
11+
# renders the summary and the hardening assessment — without crashing. There is no in-process oracle to
1112
# diff against and no stored golden master: the committed fixture is just
1213
# sysroot/ + meta.txt + sizes.txt, and nothing depends on the volatile verbose
1314
# output. Truth-containment and per-rule soundness are covered by the pure unit
@@ -192,16 +193,44 @@ run_fixture() {
192193
"$sc/replay.txt" 2>/dev/null || true)
193194
n_results=$(grep -cE '^[VPD] ' "$sc/replay.txt" 2>/dev/null || true)
194195

196+
# Hardening sweep: re-run with -H (text), -H -m (markdown) and -H -j (json).
197+
# All three render the assessment through the shared build_hardening_report()
198+
# model, so this is the only per-arch, real-fixture coverage of those paths —
199+
# it catches an emulation crash (signal) or a relocation/codegen artifact that
200+
# the host-only renderer unit tests cannot. Each must finish without a signal
201+
# (rc < 128) and emit its hardening section header. Skipped if the main run
202+
# already crashed.
203+
hard_fail=""
204+
if [ "$rc" -lt 128 ]; then
205+
for hflags in "-H" "-H -m" "-H -j"; do
206+
QEMU_UNAME="$release" KASLD_UNAME_RELEASE="$release" KASLD_SYSROOT="$sc/r" \
207+
KASLD_COMPONENT_DIR="$comps" KASLD_EXEC_WRAPPER="$runner" \
208+
$runner "$kasld" $hflags > "$sc/hard.txt" 2>/dev/null
209+
hrc=$?
210+
if [ "$hrc" -ge 128 ]; then
211+
hard_fail="$hflags signal $((hrc - 128))"
212+
break
213+
fi
214+
if ! grep -qE 'Hardening Assessment|"hardening"' "$sc/hard.txt"; then
215+
hard_fail="$hflags no hardening section"
216+
break
217+
fi
218+
done
219+
fi
220+
195221
if [ "$rc" -ge 128 ]; then
196222
printf 'FAIL %-44s (crashed under emulation: signal %d)\n' \
197223
"$name" "$((rc - 128))"
198224
echo fail >"$st"
225+
elif [ -n "$hard_fail" ]; then
226+
printf 'FAIL %-44s (hardening sweep: %s)\n' "$name" "$hard_fail"
227+
echo fail >"$st"
199228
elif [ "$has_summary" -gt 0 ]; then
200-
printf 'PASS %-44s rc=%d, %d results, summary rendered\n' \
229+
printf 'PASS %-44s rc=%d, %d results, summary + hardening rendered\n' \
201230
"$name" "$rc" "$n_results"
202231
echo pass >"$st"
203232
else
204-
printf 'PASS %-44s rc=%d, no results\n' "$name" "$rc"
233+
printf 'PASS %-44s rc=%d, no results, hardening rendered\n' "$name" "$rc"
205234
echo pass >"$st"
206235
fi
207236

0 commit comments

Comments
 (0)