Skip to content

Rebuild our Prow Build Clusters to support dualstack networking #8904

@upodroid

Description

@upodroid

Our Prow build clusters don't support IPv6, and IPv6 has been GA in Kubernetes for several years:

The GKE cluster running the prow control plane has IPv6 enabled already.

Clusters in scope:

k8s-infra-prow-build
k8s-infra-prow-build-trusted
eks-prow-build-cluster
k8s-infra-aks-admin (This cluster never worked)

Runbook:

  1. Fix all the following outstanding issues:
    1. KES: Migrate to SecretStore from ExternalSecret #3123
    2. eks-prow-build-cluster: Replace FluxCD with ArgoCD #6442
  2. Provision new Infrastructure
    1. New dual-stack VPCs for gke prow build clusters
    2. Modify the existing AWS VPC to support IPv6
    3. Deploy new GKE clusters like this: https://github.com/kubernetes/k8s.io/blob/main/infra/gcp/terraform/k8s-infra-prow/gke.tf
    4. Delete the old clusters
    5. Deploy the new clusters
  3. Post cluster provisioning
    1. Update the endpoints of the clusters in ArgoCD, https://github.com/kubernetes/k8s.io/blob/main/kubernetes/gke-utility/argocd/clusters.yaml
    2. Update the endpoints of the clusters in Prow by modifying the kubeconfig secrets in GCP Secret Manager

We expect this operation to take 3 hours and ideally be executed on the weekend before the start of v1.36 release cycle.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions