argoci: HCP support (vendored scripts + GCS cross-stage handoff)#13162
Draft
lwr20 wants to merge 5 commits into
Draft
argoci: HCP support (vendored scripts + GCS cross-stage handoff)#13162lwr20 wants to merge 5 commits into
lwr20 wants to merge 5 commits into
Conversation
The hand-ported .argoci/scripts diverged from .semaphore/end-to-end/scripts in ways that change what runs. Revert to match Semaphore (keeping only the behaviour-neutral platform adaptations): run_tests.sh: - drop the added '&& -n E2E_TEST_CONFIG' from the binary-acquire and make-e2e-run conditions. k8s-e2e jobs with an empty E2E_TEST_CONFIG now run the monorepo e2e binary via 'make e2e-run' (as Semaphore does) instead of falling through to 'bz tests' (the k8s-e2e image's [sig-calico] suite) -- a different test binary/suite. - restore the aws-iam-authenticator bind-mount (EKS make-e2e-run can't auth without it). global_prologue.sh: - INSTALLER default operator -> manual. - restore dropped defaults: K8S_E2E_EXTRA_FLAGS (calicoctl-opensource-image pin), K8S_VERSION, HELM_PATCH, CALICOCTL_INSTALL_TYPE. - restore GOOGLE_REGION randomisation + gcloud-derived GOOGLE_ZONE. global_epilogue.sh: - restore the 'always capture diags for ocp-cert' branch (code matched the existing comment). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Restore the flannel/canal-migration body dropped in the initial port. Faithful copy of .semaphore/end-to-end/scripts/body_flannel-migration.sh; only the PHASES path is adapted to the absolute form (matching body_standard.sh). No Semaphore cache/artifact deps. Wired to the iptables/patch-verification flannel jobs in the crons PR. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
marvin-tigera
added
release-note-required
lwr20
added
docs-not-required
Contributor
There was a problem hiding this comment.
Pull request overview
Adds ArgoCI support for running the OSS certification pipeline’s HCP (hosted control plane) stages by vendoring the ocp-hcp/* helper scripts and introducing a GCS-backed cross-stage handoff for the multi-cluster bzprofiles tree, while adjusting the ArgoCI scaffold to route HCP jobs through that flow and publish per-cluster JUnit artifacts.
Changes:
- Add an HCP stage dispatcher (
phases/hcp.sh) that switches onHCP_STAGEand pushes/pulls thebzprofilestree via a singlegs://…/bzprofiles.tgzobject. - Vendor
hcp-*.shscripts into.argoci/scripts/hcp/and wire scaffold scripts to setBZ_PROFILES_PATH/BZ_SECRETS_PATHand adjust epilogue behavior for HCP. - Update test execution plumbing to match Semaphore behavior (notably
run_tests.shselection) and add a flannel-migration body script.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| .argoci/scripts/phases/run_tests.sh | Align k8s-e2e selection logic with Semaphore; add optional aws-iam-authenticator mount for EKS kubeconfigs. |
| .argoci/scripts/phases/hcp.sh | New HCP stage dispatcher with GCS tarball handoff and per-stage exit behavior. |
| .argoci/scripts/hcp/hcp-test.sh | Vendored HCP test runner (bzprofiles multi-cluster loop + JUnit rename/copy). |
| .argoci/scripts/hcp/hcp-provision.sh | Vendored HCP provision/install loop over hosting + hosted clusters. |
| .argoci/scripts/hcp/hcp-init.sh | Vendored HCP profile initialization and .status file generation. |
| .argoci/scripts/hcp/hcp-diags.sh | Vendored per-cluster diagnostics collection into a shared directory. |
| .argoci/scripts/hcp/hcp-destroy.sh | Vendored per-cluster destroy/cleanup logic for the bzprofiles tree. |
| .argoci/scripts/global_prologue.sh | Add defaults/randomization + HCP env/path wiring for bzprofiles-based flow. |
| .argoci/scripts/global_epilogue.sh | Skip single-cluster diags/destroy for HCP; publish all REPORT_DIR/*.xml JUnit files. |
| .argoci/scripts/body_standard.sh | Route HCP-enabled jobs to phases/hcp.sh and bypass standard phase execution. |
| .argoci/scripts/body_flannel-migration.sh | New flannel-to-Calico migration flow script using legacy bz.sh tests:run. |
| .argoci/design/hcp.md | Design notes explaining the HCP stage split and GCS handoff model. |
Comment on lines
+51
to
+53
| export GOOGLE_REGIONS=("us-central1" "us-west1") | ||
| export GOOGLE_REGION=${GOOGLE_REGION:-${GOOGLE_REGIONS[RANDOM%${#GOOGLE_REGIONS[@]}]}} | ||
| export GOOGLE_ZONE=${GOOGLE_ZONE:-$(gcloud compute zones list --filter="region~'$GOOGLE_REGION'" --format="value(name)" | awk 'BEGIN {srand()} {a[NR]=$0} rand() * NR < 1 {zone=$0} END {print zone}')} |
Comment on lines
+10
to
+12
| # Sourced by body_standard.sh only when HCP_ENABLED=true. Dispatches on | ||
| # HCP_STAGE via a case with no default, so a non-HCP job never reaches it. | ||
| # Does NOT set an EXIT trap (that would clobber global_epilogue's). |
Comment on lines
+27
to
+33
| pull_tree() { | ||
| set -o pipefail | ||
| rm -rf "${BZ_PROFILES_PATH}" | ||
| gsutil -q stat "${BLOB}" || return 1 | ||
| echo "[INFO] hcp: pulling profile tree from ${BLOB}" | ||
| gsutil cp "${BLOB}" - | tar xzf - -C "$(dirname "${BZ_PROFILES_PATH}")" | ||
| } |
Comment on lines
+72
to
+80
| terminationGracePeriodSeconds: 0 | ||
| tolerations: | ||
| - effect: NoSchedule | ||
| operator: Exists | ||
| - key: CriticalAddonsOnly | ||
| operator: Exists | ||
| - effect: NoExecute | ||
| operator: Exists | ||
| terminationGracePeriodSeconds: 0 |
Comment on lines
+118
to
+121
| # Run a basic services test to check that flannel networking is working. | ||
| K8S_E2E_FLAGS='--ginkgo.focus=should.serve.a.basic.endpoint.from.pods' \ | ||
| ./bz.sh tests:run |& tee >(gzip --stdout > "${BZ_LOGS_DIR}/e2e-tests-pre.log") | ||
|
|
Comment on lines
+141
to
+142
| # Run e2e on uplevel calico. | ||
| ./bz.sh tests:run |& tee >(gzip --stdout > "${BZ_LOGS_DIR}/e2e-tests.log") |
…andoff) The certification pipeline runs HCP as the standard body_standard flow gated by HCP_STAGE (not the banzai-utils ocp-hcp orchestrator / HCP_ENABLED path, which OSS does not use). bz provisions the hosting/hosted OpenShift clusters via OPENSHIFT_CLUSTER_TYPE; only the hosting cluster's state crosses stages, which Semaphore did via cache+artifact — replaced here with one GCS object. - body_standard.sh: restore the HCP_STAGE guards from Semaphore (hosting/ destroy-hosting skip provision+install; *-hosting* stages exit before tests). - global_prologue.sh: hosted pulls the hosting kubeconfig and sets OPENSHIFT_HOSTING_KUBECONFIG; destroy-hosting restores the hosting BZ_HOME. - global_epilogue.sh: setup-hosting pushes the hosting BZ_HOME (and skips bz destroy so the hosting cluster survives); destroy-hosting destroys then gsutil-rm's the blob; hosted/non-HCP unchanged. Supersedes the earlier vendored-ocp-hcp approach. Design: .argoci/design/hcp.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nds) Confirmed onExit steps get the full globalPrologue/globalEpilogue wrapping, so destroy-hosting as an onExit handler gets the hosting-BZ_HOME restore + bz destroy and runs once after the workflow regardless of the hosted matrix outcome — no fan-in depends to express. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lwr20
added a commit
to lwr20/calico-argoci-test
that referenced
this pull request
Jul 8, 2026
The setup-hosting + hosted(matrix) HCP steps already carry HCP_STAGE/ HOSTING_CLUSTER. Add destroy-hosting as an onExit entry so it runs once after the workflow regardless of the hosted cells' outcomes (no fan-in depends). It gets the full prologue/epilogue: prologue restores the hosting BZ_HOME, epilogue runs bz destroy + removes the GCS handoff blob. Pairs with the HCP scaffold in projectcalico#13162. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nfig object Local-kind plumbing smoke surfaced that the whole-BZ_HOME handoff carries a Python venv, which is not relocatable. setup-hosting and destroy-hosting drew random BZ_HOME paths, so the restored venv's pip vanished. Pin BZ_PROFILE_NAME for *-hosting stages so both land on one path (mirrors Semaphore's restore-to-original-path). Also split the handoff to match Semaphore: whole BZ_HOME for destroy, a separate small kubeconfig object for hosted; clean up both on destroy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runs the OSS certification pipeline's HCP (hosted control plane) stages on ArgoCI.
HCP uses a multi-cluster
bzprofilestree (notBZ_HOME) and, on Semaphore, hands state between stages via thecache/artifactCLIs that ArgoCI lacks. So this vendors thebanzai-utilsocp-hcp/*scripts into.argoci/scripts/hcp/(env-driven, logic unchanged) and addsphases/hcp.sh, which dispatches onHCP_STAGE(setup-hosting/hosted/destroy-hosting) and replaces the cache/artifact handoff with a single GCS object (the whole tree, tarred).Scaffold edits: prologue sets
BZ_PROFILES_PATH/BZ_SECRETS_PATH;body_standardsourceshcp.shwhenHCP_ENABLED=true; epilogue skips the single-clusterbz diags/bz destroyfor HCP and glob-publishes junit.Design rationale:
.argoci/design/hcp.md(5-round design review; may be removed before merge).🚧 Stacked on #13153 (faithful-scaffold) — the scaffold edits build on it; the diff will settle once #13153 merges. Cron wiring (
HCP_ENABLED/destroy-hosting step ine2e-certification.yaml) rides in #13133.Release Note