Skip to content

Commit c075f6e

Browse files
committed
fix(ci): write pessimistic test status before Playwright execution
When Prow kills a job (2h timeout) or Playwright hangs, the mark_test_result call at the end of testing::run_tests never executes. This leaves STATUS_TEST_FAILED.txt and STATUS_NUMBER_OF_TEST_FAILED.txt with fewer entries than STATUS_DEPLOYMENT_NAMESPACE.txt, creating misaligned arrays that break downstream reporting (Slack notifications). Fix: write a pessimistic default (failed=true, count=N/A) immediately after registering the test run. The real result overwrites it after Playwright completes. If the job is killed mid-test, the STATUS files still have entries for all registered test runs. To support this, save_status_test_failed and save_status_number_of_test_failed now regenerate the file from the in-memory array instead of appending, making them safe to call multiple times for the same deployment ID. Assisted-by: OpenCode
1 parent 09387b5 commit c075f6e

2 files changed

Lines changed: 25 additions & 4 deletions

File tree

.ci/pipelines/lib/testing.sh

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,12 @@ testing::run_tests() {
5252
test_run_tracker::register "$artifacts_subdir"
5353
test_run_tracker::mark_deploy_success
5454

55+
# Pessimistic default: assume tests failed until Playwright proves otherwise.
56+
# If the job is killed (Prow timeout) or Playwright hangs, the STATUS files
57+
# still have entries for all registered test runs — preventing misaligned
58+
# arrays that break downstream reporting (Slack notifications).
59+
test_run_tracker::mark_test_result "false" "N/A"
60+
5561
BASE_URL="${url}"
5662
export BASE_URL
5763
log::info "BASE_URL: ${BASE_URL}"

.ci/pipelines/reporting.sh

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,17 +40,32 @@ save_status_test_failed() {
4040
local status=$2
4141
log::debug "Saving STATUS_TEST_FAILED[\"${current_deployment}\"]=${status}"
4242
STATUS_TEST_FAILED["${current_deployment}"]="${status}"
43-
printf "%s\n" "${STATUS_TEST_FAILED["${current_deployment}"]}" >> "$SHARED_DIR/STATUS_TEST_FAILED.txt"
44-
cp "$SHARED_DIR/STATUS_TEST_FAILED.txt" "$ARTIFACT_DIR/reporting/STATUS_TEST_FAILED.txt"
43+
_regenerate_status_file "STATUS_TEST_FAILED"
4544
}
4645

4746
save_status_number_of_test_failed() {
4847
local current_deployment=$1
4948
local number=$2
5049
log::debug "Saving STATUS_NUMBER_OF_TEST_FAILED[\"${current_deployment}\"]=${number}"
5150
STATUS_NUMBER_OF_TEST_FAILED["${current_deployment}"]="${number}"
52-
printf "%s\n" "${STATUS_NUMBER_OF_TEST_FAILED["${current_deployment}"]}" >> "$SHARED_DIR/STATUS_NUMBER_OF_TEST_FAILED.txt"
53-
cp "$SHARED_DIR/STATUS_NUMBER_OF_TEST_FAILED.txt" "$ARTIFACT_DIR/reporting/STATUS_NUMBER_OF_TEST_FAILED.txt"
51+
_regenerate_status_file "STATUS_NUMBER_OF_TEST_FAILED"
52+
}
53+
54+
# Regenerate a STATUS file from its in-memory associative array.
55+
# Unlike the append-only pattern used by save_status_deployment_namespace and
56+
# save_status_failed_to_deploy, this writes the file from scratch so that
57+
# the same deployment ID can be updated multiple times (e.g. pessimistic
58+
# default → real result after Playwright completes).
59+
_regenerate_status_file() {
60+
local var_name=$1
61+
local -n _arr="${var_name}"
62+
local file="$SHARED_DIR/${var_name}.txt"
63+
: > "$file"
64+
local key
65+
for key in $(echo "${!_arr[@]}" | tr ' ' '\n' | sort -n); do
66+
printf "%s\n" "${_arr[$key]}" >> "$file"
67+
done
68+
cp "$file" "$ARTIFACT_DIR/reporting/${var_name}.txt"
5469
}
5570

5671
save_overall_result() {

0 commit comments

Comments
 (0)