Skip to content

Commit 484c4ce

Browse files
sebastian-pf9claude
andcommitted
ci: fail generate-e2e-matrix.sh on a stale GINKGO_FOCUS mapping
Verified directly: Ginkgo v2's -ginkgo.focus reports "SUCCESS!" and exits 0 when it matches zero specs. This script hardcodes a Scenario -> GINKGO_FOCUS mapping; if a Describe() tag in a spec file is ever renamed without updating this script, the corresponding matrix case would silently run 0 specs and report green forever -- prompted by checking https://www.brokenpip3.com/posts/2024-25-02-ginkgo-github-actions-matrix/, whose own "focus values must be globally unique" gotcha is the same underlying fragility (GINKGO_FOCUS trusting a hand-maintained string against source that can drift out from under it), just the inverse failure direction. Now greps test/e2e/(packaging/)*.go for each generated focus value's literal text before emitting the matrix, refusing to generate one if anything's stale. Verified both directions: passes clean today, and fails loudly when a tag is deliberately renamed (reverted after confirming). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent c03c1c7 commit 484c4ce

1 file changed

Lines changed: 38 additions & 0 deletions

File tree

.ci/generate-e2e-matrix.sh

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,42 @@ set -Eeuo pipefail
1313
# Writes "matrix=<json>" to $GITHUB_OUTPUT if set, otherwise prints the JSON
1414
# to stdout (for local debugging via `make generate-e2e-matrix-json`).
1515

16+
# Ginkgo v2's -ginkgo.focus reports "SUCCESS!" and exits 0 when it matches
17+
# zero specs (verified directly: a typo'd focus runs 0 of N specs and still
18+
# passes) -- a Describe() tag renamed in a spec file without updating the
19+
# case statement above would silently turn a matrix case into a no-op that
20+
# reports green forever. Catch that here, once, before the expensive matrix
21+
# job starts, by checking every focus value actually appears in the source
22+
# it's supposed to select.
23+
verify_focus_values_are_real() {
24+
local json=$1
25+
local suite focus dir stale=0
26+
27+
while IFS= read -r obj; do
28+
suite=$(jq -r '.suite' <<<"${obj}")
29+
focus=$(jq -r '.ginkgo_focus' <<<"${obj}")
30+
[[ -z "${focus}" ]] && continue
31+
dir="test/e2e"
32+
[[ "${suite}" == "packaging" ]] && dir="test/e2e/packaging"
33+
34+
# @tsv would double-escape the backslashes in a \[Tag\] focus regex, so
35+
# fields are pulled directly off each object above instead. Unescape
36+
# \[ / \] back to literal [ / ] to grep for the plain Describe() text.
37+
local literal=${focus//\\[/[}
38+
literal=${literal//\\]/]}
39+
40+
if ! grep -rFq -- "${literal}" "${dir}"/*.go; then
41+
echo "generate-e2e-matrix.sh: GINKGO_FOCUS '${focus}' (literal: '${literal}') matches no Describe() under ${dir}/*.go -- stale mapping in this script" >&2
42+
stale=1
43+
fi
44+
done < <(jq -c '.[]' <<<"${json}")
45+
46+
if ((stale)); then
47+
echo "generate-e2e-matrix.sh: refusing to generate a matrix with stale focus values -- a case above would silently run 0 specs and report success" >&2
48+
exit 1
49+
fi
50+
}
51+
1652
main() {
1753
local tsv=$1
1854
local rows="[]"
@@ -72,6 +108,8 @@ main() {
72108
json=$(jq -c '.' <<<"${rows}")
73109
echo "generated $(jq 'length' <<<"${json}") matrix cases from ${tsv}" >&2
74110

111+
verify_focus_values_are_real "${json}"
112+
75113
if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
76114
echo "matrix=${json}" >>"${GITHUB_OUTPUT}"
77115
else

0 commit comments

Comments
 (0)