ci(gke ap): run allowlist check on push to main by basti1302 · Pull Request #1197 · dash0hq/dash0-operator

basti1302 · 2026-07-10T13:07:20Z

No description provided.

basti1302 · 2026-07-10T13:35:35Z

+# SPDX-FileCopyrightText: Copyright 2026 Dash0 Inc.
+# SPDX-License-Identifier: Apache-2.0
+
+name: send Slack notification


extracted this into a reusable action

basti1302 · 2026-07-10T13:38:03Z

/run-allowlist-check

github-actions · 2026-07-10T13:38:13Z

🔍 GKE Autopilot WorkloadAllowlist check triggered

Branch: run-gke-workload-allowlist-on-main-after-every-push
Triggered by: @basti1302

Results will be posted here once the check completes. You can also view the workflow runs directly.

basti1302 · 2026-07-10T13:38:57Z

run-allowlist-check

^ will fail, but I want to double check whether the GH action in general still works correctly, independent of the check's success

github-actions · 2026-07-10T13:46:25Z

❌ GKE Autopilot WorkloadAllowlist check failed (image build)

Branch: run-gke-workload-allowlist-on-main-after-every-push
Run: #29096785859

basti1302 · 2026-07-10T14:41:21Z

/run-allowlist-check

github-actions · 2026-07-10T14:41:31Z

🔍 GKE Autopilot WorkloadAllowlist check triggered

Branch: run-gke-workload-allowlist-on-main-after-every-push
Triggered by: @basti1302

Results will be posted here once the check completes. You can also view the workflow runs directly.

basti1302 · 2026-07-10T14:41:56Z

❌ GKE Autopilot WorkloadAllowlist check failed (image build)

Branch: run-gke-workload-allowlist-on-main-after-every-push

Run: #29096785859

That failed in the wrong way due to an unrelated access issue for the new agent0-connector image. Re-running.

github-actions · 2026-07-10T14:52:32Z

❌ GKE Autopilot WorkloadAllowlist check has failed

Branch: run-gke-workload-allowlist-on-main-after-every-push
Run: #29100781521

basti1302 · 2026-07-10T14:54:17Z

❌ GKE Autopilot WorkloadAllowlist check has failed

Branch: run-gke-workload-allowlist-on-main-after-every-push

Run: #29100781521

This failed in the correct/expected way now (it's a new command line flag for the deployment collector). 👍

basti1302 · 2026-07-10T15:33:07Z

Added one more thing (collecting diagnostics on failure), so let's do another test run.

basti1302 · 2026-07-10T15:33:11Z

/run-allowlist-check

github-actions · 2026-07-10T15:33:21Z

🔍 GKE Autopilot WorkloadAllowlist check triggered

Branch: run-gke-workload-allowlist-on-main-after-every-push
Triggered by: @basti1302

Results will be posted here once the check completes. You can also view the workflow runs directly.

github-actions · 2026-07-10T15:45:43Z

❌ GKE Autopilot WorkloadAllowlist check has failed

Branch: run-gke-workload-allowlist-on-main-after-every-push
Run: #29104104065

basti1302 · 2026-07-10T16:08:19Z


+# Directory and archive used to collect cluster diagnostics when the check fails. These paths are relative to the
+# repository root (we cd into it above), which is also the working directory of the GitHub Actions job, so the workflow
+# can upload the archive as an artifact.
+diagnostics_dir=gke-ap-allowlist-check-diagnostics
+diagnostics_archive="${diagnostics_dir}.tar.gz"
+
+# Set to "true" once all checks have passed. As long as this is not "true" when cleanup() runs, we assume the check has
+# failed and collect cluster diagnostics before tearing everything down.
+check_succeeded=false
+
+# Collects diagnostic information from the cluster (workloads, pods, their logs, events, config maps and the relevant
+# custom resources) into $diagnostics_dir and compresses it into $diagnostics_archive. This runs at the start of
+# cleanup() when the check has failed, that is, before "helm uninstall" removes everything from the cluster.
+collect_diagnostics() {
+  set +e
+  echo "the check did not succeed, collecting cluster diagnostics into \"$diagnostics_dir\" before cleanup"
+  mkdir -p "$diagnostics_dir"
+
+  # Runs a command and stores its combined stdout/stderr in $diagnostics_dir/$1. Best effort: a failing command must
+  # never abort the diagnostics collection or the cleanup.
+  dump() {
+    local file="$diagnostics_dir/$1"
+    shift
+    mkdir -p "$(dirname "$file")"
+    {
+      echo "\$ $*"
+      echo
+      "$@" 2>&1
+    } > "$file" || echo "(command exited with a non-zero status)" >> "$file"
+  }
+
+  # Cluster-scoped GKE Autopilot allowlist resources. A mismatch or a not-yet-ready AllowlistSynchronizer is the most
+  # likely reason for this check to fail.
+  dump allowlistsynchronizers-get.txt kubectl get allowlistsynchronizers.auto.gke.io -o wide
+  dump allowlistsynchronizers-describe.txt kubectl describe allowlistsynchronizers.auto.gke.io
+  dump workloadallowlists-get.txt kubectl get workloadallowlists.auto.gke.io -o wide
+  dump workloadallowlists-describe.txt kubectl describe workloadallowlists.auto.gke.io
+
+  # Dash0 custom resources.
+  dump dash0monitorings-get.txt kubectl get dash0monitorings.operator.dash0.com --all-namespaces -o wide
+  dump dash0monitorings-describe.txt kubectl describe dash0monitorings.operator.dash0.com --all-namespaces
+  dump dash0operatorconfigurations-get.txt kubectl get dash0operatorconfigurations.operator.dash0.com -o wide
+  dump dash0operatorconfigurations-describe.txt kubectl describe dash0operatorconfigurations.operator.dash0.com
+
+  # Cluster-wide events (scheduling and allowlist rejections often show up here).
+  dump events-all-namespaces.txt kubectl get events --all-namespaces --sort-by=.lastTimestamp
+
+  local namespace
+  for namespace in "$operator_namespace" "$monitored_namespace"; do
+    dump "$namespace/workloads-get.txt" \
+      kubectl get deployments,daemonsets,statefulsets,replicasets,jobs,pods,configmaps,services \
+      --namespace "$namespace" -o wide
+    dump "$namespace/workloads-describe.txt" \
+      kubectl describe deployments,daemonsets,statefulsets,jobs --namespace "$namespace"
+    dump "$namespace/pods-describe.txt" kubectl describe pods --namespace "$namespace"
+    dump "$namespace/configmaps-describe.txt" kubectl describe configmaps --namespace "$namespace"
+    dump "$namespace/events.txt" kubectl get events --namespace "$namespace" --sort-by=.lastTimestamp
+
+    # Collect logs (all containers, current and previous) for every pod in the namespace.
+    local pod
+    while IFS= read -r pod; do
+      [[ -z "$pod" ]] && continue
+      dump "$namespace/logs-${pod}.txt" \
+        kubectl logs --namespace "$namespace" "$pod" --all-containers=true --prefix=true
+      # Previous logs only exist for containers that have restarted; ignore the file if there are none.
+      if ! kubectl logs --namespace "$namespace" "$pod" --all-containers=true --prefix=true --previous \
+        > "$diagnostics_dir/$namespace/logs-${pod}-previous.txt" 2>/dev/null; then
+        rm -f "$diagnostics_dir/$namespace/logs-${pod}-previous.txt"
+      fi
+    done < <(kubectl get pods --namespace "$namespace" \
+      -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' 2>/dev/null)
+  done
+
+  if tar -czf "$diagnostics_archive" "$diagnostics_dir"; then
+    echo "cluster diagnostics have been collected in \"$diagnostics_archive\""
+  else
+    echo "WARNING: failed to create the diagnostics archive \"$diagnostics_archive\""
+  fi
+  set -e
+}


Slightly unrelated to the original intent, but so far the allowlist check only failed, and left us guessing what went wrong. All the relevant logs etc. would have been lost once the GH action reported failure (due to cleanup). Collecting evidence before doing helm uninstall makes this much more convenient.

To test the new workflow_call trigger.

This reverts commit 8cd4c8b.

basti1302 · 2026-07-10T18:03:57Z

This is ready for review, but we should only merge it after #1124.

basti1302 added 3 commits July 10, 2026 15:03

ci(gke ap): run allowlist check on push to main

108598a

ci: refactor Slack notification into shared action

d2819d9

ci: use allowlist check as reusable workflow

51af862

basti1302 commented Jul 10, 2026

View reviewed changes

ci: collect diagnostics if allowlist check fails

6fa19b7

basti1302 commented Jul 10, 2026

View reviewed changes

basti1302 added 2 commits July 10, 2026 18:09

temporarily removing the restriction branch main

8cd4c8b

To test the new workflow_call trigger.

Revert "temporarily removing the restriction branch main"

5e78eeb

This reverts commit 8cd4c8b.

basti1302 marked this pull request as ready for review July 10, 2026 16:25

basti1302 requested a review from a team as a code owner July 10, 2026 16:25

basti1302 added the blocked label Jul 10, 2026

p-d0 approved these changes Jul 13, 2026

View reviewed changes

Uh oh!

Conversation

basti1302 commented Jul 10, 2026

Uh oh!

basti1302 Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

basti1302 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

🔍 GKE Autopilot WorkloadAllowlist check triggered

Uh oh!

basti1302 commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 10, 2026

❌ GKE Autopilot WorkloadAllowlist check failed (image build)

Uh oh!

basti1302 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

🔍 GKE Autopilot WorkloadAllowlist check triggered

Uh oh!

basti1302 commented Jul 10, 2026

❌ GKE Autopilot WorkloadAllowlist check failed (image build)

Uh oh!

github-actions Bot commented Jul 10, 2026

❌ GKE Autopilot WorkloadAllowlist check has failed

Uh oh!

basti1302 commented Jul 10, 2026

❌ GKE Autopilot WorkloadAllowlist check has failed

Uh oh!

basti1302 commented Jul 10, 2026

Uh oh!

basti1302 commented Jul 10, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

🔍 GKE Autopilot WorkloadAllowlist check triggered

Uh oh!

github-actions Bot commented Jul 10, 2026

❌ GKE Autopilot WorkloadAllowlist check has failed

Uh oh!

basti1302 Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

basti1302 commented Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

basti1302 commented Jul 10, 2026 •

edited

Loading