Skip to content

Commit e3a79d8

Browse files
neoaggelosneoaggelos
authored
Kubernetes v1.36 support (#214)
* Kubernetes v1.36 support * Bump to kind v1.35.0 for e2e tests * Update load balancer after Machines become Running. This addresses two critical issues: 1. Issues with OVN load balancer, which becomes temporarily unavailable when updating the backends, which translated into control plane machines failing to join the cluster because the control plane endpoint was getting updated at the same time 2. DOA machines were getting added to the load balancer without ever becoming healthy, adding more flakiness and delays to the process. --------- Signed-off-by: Angelos Kolaitis <neoaggelos@gmail.com>
1 parent 60e38bb commit e3a79d8

24 files changed

Lines changed: 144 additions & 37 deletions

‎docs/book/src/howto/developer-guide.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ chmod +x ./clusterctl
2525
sudo mv ./clusterctl /usr/local/bin/clusterctl
2626

2727
# kubectl
28-
curl -L --remote-name-all "https://dl.k8s.io/release/v1.34.0/bin/linux/amd64/kubectl" -o ./kubectl
28+
curl -L --remote-name-all "https://dl.k8s.io/release/v1.36.1/bin/linux/amd64/kubectl" -o ./kubectl
2929
chmod +x ./kubectl
3030
sudo mv ./kubectl /usr/local/bin/kubectl
3131
```
@@ -125,7 +125,7 @@ On a separate window, generate a cluster manifest and deploy:
125125
```bash
126126
export LOAD_BALANCER="lxc: {}"
127127
export LXC_SECRET_NAME="lxc-secret"
128-
export KUBERNETES_VERSION="v1.34.0"
128+
export KUBERNETES_VERSION="v1.36.1"
129129
export CONTROL_PLANE_MACHINE_COUNT=1
130130
export WORKER_MACHINE_COUNT=1
131131

‎docs/book/src/reference/default-simplestreams-server.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,8 @@ The following images are currently provided:
3838
| kubeadm/v1.33.5 | Ubuntu 24.04 | Kubeadm image for Kubernetes v1.33.5 | X | X |
3939
| kubeadm/v1.34.0 | Ubuntu 24.04 | Kubeadm image for Kubernetes v1.34.0 | X | X |
4040
| kubeadm/v1.35.0 | Ubuntu 24.04 | Kubeadm image for Kubernetes v1.35.0 | X | X |
41+
| kubeadm/v1.35.5 | Ubuntu 24.04 | Kubeadm image for Kubernetes v1.35.5 | X | X |
42+
| kubeadm/v1.36.1 | Ubuntu 24.04 | Kubeadm image for Kubernetes v1.36.1 | X | X |
4143

4244
Note that the table above might be out of date. See [streams/v1/index.json] and [streams/v1/images.json] for the list of versions currently available.
4345

‎docs/book/src/tutorial/quick-start.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ chmod +x ./clusterctl
3232
sudo mv ./clusterctl /usr/local/bin/clusterctl
3333

3434
# kubectl
35-
curl -L --remote-name-all "https://dl.k8s.io/release/v1.34.0/bin/linux/amd64/kubectl" -o ./kubectl
35+
curl -L --remote-name-all "https://dl.k8s.io/release/v1.36.1/bin/linux/amd64/kubectl" -o ./kubectl
3636
chmod +x ./kubectl
3737
sudo mv ./kubectl /usr/local/bin/kubectl
3838
```
@@ -250,7 +250,7 @@ Then generate a cluster manifest for a cluster with 1 control plane and 1 worker
250250
```bash
251251
# generate manifest in 'cluster.yaml'
252252
clusterctl generate cluster c1 -i incus \
253-
--kubernetes-version v1.34.0 \
253+
--kubernetes-version v1.36.1 \
254254
--control-plane-machine-count 1 \
255255
--worker-machine-count 1 \
256256
> cluster.yaml
@@ -299,14 +299,14 @@ Cluster/c1 False Info Bootstrapping @
299299

300300
# kubectl get cluster,lxccluster,machine,lxcmachine
301301
NAME CLUSTERCLASS PHASE AGE VERSION
302-
cluster.cluster.x-k8s.io/c1 capn-v1beta2 Provisioned 22s v1.34.0
302+
cluster.cluster.x-k8s.io/c1 capn-v1beta2 Provisioned 22s v1.36.1
303303

304304
NAME CLUSTER LOAD BALANCER READY AGE
305305
lxccluster.infrastructure.cluster.x-k8s.io/c1-vtf7d c1 10.130.1.162 true 22s
306306

307307
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
308-
machine.cluster.x-k8s.io/c1-6n84z-lxj6v c1 Provisioning 17s v1.34.0
309-
machine.cluster.x-k8s.io/c1-md-0-v42br-vh2wd-7sn5p c1 Pending 6s v1.34.0
308+
machine.cluster.x-k8s.io/c1-6n84z-lxj6v c1 Provisioning 17s v1.36.1
309+
machine.cluster.x-k8s.io/c1-md-0-v42br-vh2wd-7sn5p c1 Pending 6s v1.36.1
310310

311311
NAME CLUSTER MACHINE PROVIDERID READY AGE
312312
lxcmachine.infrastructure.cluster.x-k8s.io/c1-6n84z-lxj6v c1 c1-6n84z-lxj6v 17s
@@ -328,14 +328,14 @@ Cluster/c1 True 23s
328328

329329
# kubectl get cluster,lxccluster,machine,lxcmachine
330330
NAME CLUSTERCLASS PHASE AGE VERSION
331-
cluster.cluster.x-k8s.io/c1 capn-v1beta2 Provisioned 59s v1.34.0
331+
cluster.cluster.x-k8s.io/c1 capn-v1beta2 Provisioned 59s v1.36.1
332332

333333
NAME CLUSTER LOAD BALANCER READY AGE
334334
lxccluster.infrastructure.cluster.x-k8s.io/c1-vtf7d c1 10.130.1.162 true 59s
335335

336336
NAME CLUSTER NODENAME PROVIDERID PHASE AGE VERSION
337-
machine.cluster.x-k8s.io/c1-6n84z-lxj6v c1 c1-6n84z-lxj6v lxc:///c1-6n84z-lxj6v Running 54s v1.34.0
338-
machine.cluster.x-k8s.io/c1-md-0-v42br-vh2wd-7sn5p c1 c1-md-0-v42br-vh2wd-7sn5p lxc:///c1-md-0-v42br-vh2wd-7sn5p Running 43s v1.34.0
337+
machine.cluster.x-k8s.io/c1-6n84z-lxj6v c1 c1-6n84z-lxj6v lxc:///c1-6n84z-lxj6v Running 54s v1.36.1
338+
machine.cluster.x-k8s.io/c1-md-0-v42br-vh2wd-7sn5p c1 c1-md-0-v42br-vh2wd-7sn5p lxc:///c1-md-0-v42br-vh2wd-7sn5p Running 43s v1.36.1
339339

340340
NAME CLUSTER MACHINE PROVIDERID READY AGE
341341
lxcmachine.infrastructure.cluster.x-k8s.io/c1-6n84z-lxj6v c1 c1-6n84z-lxj6v lxc:///c1-6n84z-lxj6v true 54s
@@ -413,8 +413,8 @@ kube-system pod/kube-proxy-zkwcc 1/1 Running 0
413413
kube-system pod/kube-scheduler-c1-6n84z-lxj6v 1/1 Running 0 2m16s 10.130.1.97 c1-6n84z-lxj6v <none> <none>
414414

415415
NAMESPACE NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
416-
node/c1-6n84z-lxj6v Ready control-plane 2m18s v1.34.0 10.130.1.97 <none> Ubuntu 24.04.3 LTS 6.8.0-83-generic containerd://2.1.4
417-
node/c1-md-0-v42br-vh2wd-7sn5p Ready <none> 112s v1.34.0 10.130.1.195 <none> Ubuntu 24.04.3 LTS 6.8.0-83-generic containerd://2.1.4
416+
node/c1-6n84z-lxj6v Ready control-plane 2m18s v1.36.1 10.130.1.97 <none> Ubuntu 24.04.3 LTS 6.8.0-83-generic containerd://2.3.0
417+
node/c1-md-0-v42br-vh2wd-7sn5p Ready <none> 112s v1.36.1 10.130.1.195 <none> Ubuntu 24.04.3 LTS 6.8.0-83-generic containerd://2.3.0
418418
```
419419

420420
## Delete cluster

‎hack/infra/images.yaml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,3 +72,33 @@ images:
7272
type: virtual-machine
7373
alias: [kubeadm/v1.35.0, kubeadm/v1.35.0/ubuntu]
7474
source: https://github.com/lxc/cluster-api-provider-incus/releases/download/kubeadm-images/kubeadm-v1.35.0-virtual-machine-ubuntu-amd64.tar.gz
75+
76+
kubeadm-v1.35.5-container-amd64:
77+
type: container
78+
alias: [kubeadm/v1.35.5, kubeadm/v1.35.5/ubuntu]
79+
source: https://github.com/lxc/cluster-api-provider-incus/releases/download/kubeadm-images/kubeadm-v1.35.5-container-ubuntu-amd64.tar.gz
80+
checksum: sha256:0a6c0d07c22864d3ccd01689aca9a31fd6e8da001672165997f19f7f552dcbb3
81+
kubeadm-v1.35.5-container-arm64:
82+
type: container
83+
alias: [kubeadm/v1.35.5, kubeadm/v1.35.5/ubuntu]
84+
source: https://github.com/lxc/cluster-api-provider-incus/releases/download/kubeadm-images/kubeadm-v1.35.5-container-ubuntu-arm64.tar.gz
85+
checksum: sha256:32cd7f73ee14780febc533539fa3dd0b6eef9d15fcd7d3e069c2977e910f9bfe
86+
kubeadm-v1.35.5-virtual-machine-amd64:
87+
type: virtual-machine
88+
alias: [kubeadm/v1.35.5, kubeadm/v1.35.5/ubuntu]
89+
source: https://github.com/lxc/cluster-api-provider-incus/releases/download/kubeadm-images/kubeadm-v1.35.5-virtual-machine-ubuntu-amd64.tar.gz
90+
91+
kubeadm-v1.36.1-container-amd64:
92+
type: container
93+
alias: [kubeadm/v1.36.1, kubeadm/v1.36.1/ubuntu]
94+
source: https://github.com/lxc/cluster-api-provider-incus/releases/download/kubeadm-images/kubeadm-v1.36.1-container-ubuntu-amd64.tar.gz
95+
checksum: sha256:241075592fa5d280d6d952a0eb8aa434069d27746675ac22a587ac469a57078b
96+
kubeadm-v1.36.1-container-arm64:
97+
type: container
98+
alias: [kubeadm/v1.36.1, kubeadm/v1.36.1/ubuntu]
99+
source: https://github.com/lxc/cluster-api-provider-incus/releases/download/kubeadm-images/kubeadm-v1.36.1-container-ubuntu-arm64.tar.gz
100+
checksum: sha256:a658ebfd4775e94cb6c3391e6aa2d52089b6ba4df562eb4f39e1ecc7ee80c28c
101+
kubeadm-v1.36.1-virtual-machine-amd64:
102+
type: virtual-machine
103+
alias: [kubeadm/v1.36.1, kubeadm/v1.36.1/ubuntu]
104+
source: https://github.com/lxc/cluster-api-provider-incus/releases/download/kubeadm-images/kubeadm-v1.36.1-virtual-machine-ubuntu-amd64.tar.gz

‎internal/controller/lxcmachine/controller_normal.go‎

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,21 @@ func (r *LXCMachineReconciler) reconcileNormal(ctx context.Context, cluster *clu
5050
conditions.Set(lxcMachine, metav1.Condition{Type: infrav1.InstanceProvisionedCondition, Status: metav1.ConditionTrue, Reason: infrav1.InstanceProvisionedReason})
5151
r.setLXCMachineAddresses(lxcMachine, lxc.ParseHostAddresses(state))
5252

53+
// Handle control plane load balancer configuration
54+
if util.IsControlPlaneMachine(machine) && !lxcMachine.Status.LoadBalancerConfigured {
55+
machinePhase := machine.Status.GetTypedPhase()
56+
if cluster.Spec.ControlPlaneRef.IsDefined() && ptr.Deref(cluster.Status.Initialization.ControlPlaneInitialized, false) && machinePhase != clusterv1.MachinePhaseRunning {
57+
log.FromContext(ctx).Info("ControlPlane is initialized, waiting for Machine to become Running before updating control plane load balancer", "phase", machinePhase)
58+
return ctrl.Result{RequeueAfter: 10 * time.Second}, nil
59+
}
60+
61+
log.FromContext(ctx).Info("Updating control plane load balancer")
62+
if err := loadbalancer.ManagerForCluster(cluster, lxcCluster, lxcClient).Reconfigure(ctx); err != nil {
63+
return ctrl.Result{}, fmt.Errorf("failed to update loadbalancer configuration: %w", err)
64+
}
65+
lxcMachine.Status.LoadBalancerConfigured = true
66+
}
67+
5368
// Handle cloud provider node patch
5469
if lxcCluster.Spec.CloudProviderNodePatch && !lxcMachine.Status.CloudProviderNodePatchConfigured {
5570
// If the Cluster is using a control plane and the control plane is not yet initialized, there is no API server
@@ -131,16 +146,6 @@ func (r *LXCMachineReconciler) reconcileNormal(ctx context.Context, cluster *clu
131146
r.setLXCMachineAddresses(lxcMachine, addresses)
132147
conditions.Set(lxcMachine, metav1.Condition{Type: infrav1.InstanceProvisionedCondition, Status: metav1.ConditionTrue, Reason: infrav1.InstanceProvisionedReason})
133148

134-
// update load balancer
135-
if util.IsControlPlaneMachine(machine) && !lxcMachine.Status.LoadBalancerConfigured {
136-
log.FromContext(ctx).Info("Updating control plane load balancer")
137-
138-
if err := loadbalancer.ManagerForCluster(cluster, lxcCluster, lxcClient).Reconfigure(ctx); err != nil {
139-
return ctrl.Result{}, fmt.Errorf("failed to update loadbalancer configuration: %w", err)
140-
}
141-
lxcMachine.Status.LoadBalancerConfigured = true
142-
}
143-
144149
lxcMachine.Spec.ProviderID = lxcMachine.GetExpectedProviderID()
145150
lxcMachine.Status.Initialization.Provisioned = new(true)
146151

‎internal/lxc/consts.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ const (
99
// DefaultStagingSimplestreamsServer is the default staging simplestreams server for fetching images.
1010
DefaultStagingSimplestreamsServer = "https://images-stg.capn.open-cloud.xyz/capn/staging/"
1111

12+
// DefaultDevelopmentSimplestreamsServer is the default development simplestreams server for fetching images.
13+
DefaultDevelopmentSimplestreamsServer = "https://images-stg.capn.open-cloud.xyz/capn/testing/"
14+
1215
// DefaultIncusSimplestreamsServer is the default simplestreams server for Incus.
1316
DefaultIncusSimplestreamsServer = "https://images.linuxcontainers.org"
1417

‎internal/lxc/images_parse.go‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ var (
1414
"images": DefaultImage,
1515
"capi": func(in string) ImageFamily { return CapnImage(in) },
1616
"capi-stg": func(in string) ImageFamily { return CapnStagingImage(in) },
17+
"capi-dev": func(in string) ImageFamily { return CapnDevelopmentImage(in) },
1718
"kind": func(in string) ImageFamily { return KindestNodeImage(in) },
1819
}
1920
)

‎internal/lxc/images_parse_test.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ func TestParseImage(t *testing.T) {
4444
{server: "incus", image: "images:almalinux/9/cloud", expectParse: true, expectImageSource: simplestreamsImage("https://images.linuxcontainers.org", "almalinux/9/cloud")},
4545
{server: "incus", image: "capi:kubeadm/v1.33.0", expectParse: true, expectImageSource: simplestreamsImage("https://images.linuxcontainers.org/capn/", "kubeadm/v1.33.0")},
4646
{server: "incus", image: "capi-stg:kubeadm/v1.33.0", expectParse: true, expectImageSource: simplestreamsImage("https://images-stg.capn.open-cloud.xyz/capn/staging/", "kubeadm/v1.33.0")},
47+
{server: "incus", image: "capi-dev:kubeadm/v1.33.0", expectParse: true, expectImageSource: simplestreamsImage("https://images-stg.capn.open-cloud.xyz/capn/testing/", "kubeadm/v1.33.0")},
4748
{server: "incus", image: "kind:v1.33.0", expectParse: true, expectImageSource: ociImage("https://docker.io", "kindest/node:v1.33.0")},
4849
// verify lxd prefixes
4950
{server: "lxd", image: "image-name"},
@@ -53,6 +54,7 @@ func TestParseImage(t *testing.T) {
5354
{server: "lxd", image: "images:almalinux/9/cloud", expectParse: true, expectImageSource: simplestreamsImage("https://images.lxd.canonical.com", "almalinux/9/cloud")},
5455
{server: "lxd", image: "capi:kubeadm/v1.33.0", expectParse: true, expectImageSource: simplestreamsImage("https://images.linuxcontainers.org/capn/", "kubeadm/v1.33.0")},
5556
{server: "lxd", image: "capi-stg:kubeadm/v1.33.0", expectParse: true, expectImageSource: simplestreamsImage("https://images-stg.capn.open-cloud.xyz/capn/staging/", "kubeadm/v1.33.0")},
57+
{server: "lxd", image: "capi-dev:kubeadm/v1.33.0", expectParse: true, expectImageSource: simplestreamsImage("https://images-stg.capn.open-cloud.xyz/capn/testing/", "kubeadm/v1.33.0")},
5658
{server: "lxd", image: "kind:v1.33.0", expectParse: true, expectImageSource: ociImage("https://docker.io", "kindest/node:v1.33.0")},
5759
// verify prefixes for unknown
5860
{server: "unknown", image: "image-name"},

‎internal/lxc/images_sources.go‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,14 @@ func CapnStagingImage(image string) Image {
7474
}
7575
}
7676

77+
func CapnDevelopmentImage(image string) Image {
78+
return Image{ // "capi-dev:IMAGE" -> "IMAGE" from "https://images-stg.capn.open-cloud.xyz/capn/testing/"
79+
Protocol: Simplestreams,
80+
Server: DefaultDevelopmentSimplestreamsServer,
81+
Alias: image,
82+
}
83+
}
84+
7785
func KindestNodeImage(version string) Image {
7886
return Image{ // "kind:VERSION" -> "kindest/node:VERSION" from "https://docker.io"
7987
Protocol: OCI,

‎internal/static/embed/cloud-init-launch.service‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ ConditionPathExists=!/hack/cloud-init.success
55

66
[Service]
77
Type=oneshot
8-
ExecStart=/bin/bash -xe -c 'python3 /hack/cloud-init.py && touch /hack/cloud-init.success'
8+
ExecStart=/bin/bash -xe -c 'crictl ps -a && python3 /hack/cloud-init.py && touch /hack/cloud-init.success'
99
Restart=on-failure
1010
RestartSec=5s
1111
StartLimitInterval=60

0 commit comments

Comments
 (0)