Skip to content

Commit 5cadcd3

Browse files
gunjan5gunjan5
authored
Merge pull request #1025 from gunjan5/upgrade
Calico v2.5 TPR->CRD migration and upgrade k8s job and docs
2 parents d202f9f + 8075143 commit 5cadcd3

File tree

8 files changed

+455
-0
lines changed

8 files changed

+455
-0
lines changed

‎upgrade/v2.5/Dockerfile

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
FROM busybox
2+
3+
MAINTAINER Gunjan Patel <[email protected]>
4+
5+
ADD dist/calicoctl-v1.4 /sbin/calicoctl-v1.4
6+
ADD dist/calicoctl-v1.5 /sbin/calicoctl-v1.5
7+
ADD dist/kubectl /sbin/kubectl
8+
9+
ADD script/upgrade.sh /upgrade.sh
10+
ADD manifests/crds.yaml /crds.yaml
11+
ADD manifests/tprs.yaml /tprs.yaml
12+
ADD manifests/globalbgpconfig.yaml /globalbgpconfig.yaml
13+
14+
WORKDIR /
15+
16+
CMD ["/upgrade.sh"]

‎upgrade/v2.5/Makefile

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
dist/clis:
2+
mkdir -p dist
3+
curl -sSf -L https://github.com/projectcalico/calicoctl/releases/download/v1.4.1/calicoctl -o dist/calicoctl-v1.4
4+
curl -sSf -L https://github.com/projectcalico/calicoctl/releases/download/v1.5.0/calicoctl -o dist/calicoctl-v1.5
5+
curl -sSf -L https://storage.googleapis.com/kubernetes-release/release/v1.7.4/bin/linux/amd64/kubectl -o dist/kubectl
6+
chmod +x dist/calicoctl-v1.4
7+
chmod +x dist/calicoctl-v1.5
8+
chmod +x dist/kubectl
9+
10+
build-container: dist/clis
11+
docker build -t calico/v2.5-upgrade:v0.0.1 .
12+
13+
clean:
14+
rm -rf dist

‎upgrade/v2.5/README.md

+94
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Calico v2.5.0 Upgrade Procedure
2+
3+
## Why this is required
4+
5+
Kubernetes v1.7 introduces a new API data type called CustomResourceDefinition (CRD) which will replace the alpha ThirdPartyResource (TPR).
6+
Calico relies on TPR to store it's config data when it uses kubernetes datastore backend. Moving forward from kubernetes v1.8,
7+
Kubernetes will remove the support for TPR in favor of CRD. In order to preserve the config data backed by TPR, and continue to
8+
work with the future Kubernetes versions, it is required to go through this migration process.
9+
See [this blog post](https://coreos.com/blog/custom-resource-kubernetes-v17) for more information on this new data type.
10+
11+
## Steps required
12+
13+
At a high level we need to do the following steps to make sure we have a successful upgrade.
14+
15+
1. Check to make sure if you need to do the migration
16+
2. Backup your TPR data (Data is in TPR right now)
17+
3. Run the migration job (Data is copied from TPR to CRD)
18+
4. Verify the migration (Data is still in both TPR and CRD, Calico is still using TPR)
19+
5. Upgrade Calico version to `v2.5.0` (Data is still in TPR and CRD, but Calico now uses CRD backed data)
20+
6. Verify Calico policy and networking works as expected
21+
7. Delete the TPRs
22+
23+
## 1. Before you begin
24+
25+
### 1.1 Do I need to go through this migration process?
26+
27+
This is only required if you meet ALL of the following criteria:
28+
29+
- Running Calico with Kubernetes
30+
- Current Kubernetes version is `v1.7.x`
31+
- Current Calico version is `v2.4.x` or lower
32+
- Upgrading Calico to `v2.5.x`
33+
- Calico is running with Kubernetes datastore backend (By running `calicoctl version` and making sure `Cluster Type` is `KDD`)
34+
35+
### 1.2 Backup your config data
36+
37+
We highly recommend backing up your config data before proceeding with the migration process.
38+
We only need to backup the config backed by TPR resources; in the event the migration needs to be halted, you'll be able to restore the original data.
39+
40+
> Note: the migration job does not delete your old data, so your config data backed by TPR will still be there until
41+
deleted manually (explained in the last step of this doc).
42+
43+
Run the following commands to backup your config data:
44+
45+
> Note: use `calicoctl` version [v1.4.1](https://github.com/projectcalico/calicoctl/releases/tag/v1.4.1)
46+
and `kubectl` version [v1.7.4](https://kubernetes.io/docs/tasks/tools/install-kubectl/) to backup the data.
47+
48+
Since we will need `calicoctl` versions [v1.4.1](https://github.com/projectcalico/calicoctl/releases/tag/v1.4.1) and [v1.5.0](https://github.com/projectcalico/calicoctl/releases/tag/v1.5.0) for this upgrade,
49+
we recommend downloading them both and suffixing the binaries with their respective versions (you can check the version by running `calicoctl version`)
50+
51+
1.2.1. `calicoctl_v1.4 get ippools -o yaml > ippool.yaml`
52+
53+
1.2.2. `calicoctl_v1.4 get bgppeers -o yaml > bgppeer.yaml`
54+
55+
1.2.3. `kubectl get globalconfig --all-namespaces -o yaml > tpr-felixconfig.yaml`
56+
57+
1.2.4. `kubectl get globalbgpconfig --all-namespaces -o yaml > tpr-bgpconfig.yaml`
58+
59+
> Note: you may not have some of these resources if you're using Calico in policy-only mode.
60+
61+
## 2. Migration process
62+
63+
2.1. Create the migration kubernetes job for your cluster: `kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/master/upgrade/v2.5/manifests/upgrade-job.yaml`
64+
65+
2.2. Check the job status `kubectl describe job/calico-upgrade-v2.5`
66+
67+
2.3. Check the upgrade job logs `kubectl logs <upgrade-pod-name>` to make sure there aren't any errors (You can get the upgrade job's pod name from the previous command output 'Message' field)
68+
69+
## 3. After the migration
70+
71+
### 3.1 Verify that the data has been copied correctly
72+
73+
Make sure you have all the `IPPools`, `BGPPeers`, `GlobalFelixConfig` and `GlobalBGPConfig` resource configs you had before the upgrade:
74+
75+
3.1.1. `calicoctl_v1.5 get ippools -o wide`
76+
77+
3.1.2. `calicoctl_v1.5 get bgppeers -o wide` (This is only if you're running Calico BGP networking)
78+
79+
3.1.3. `kubectl get globalfelixconfigs.crd.projectcalico.org -o wide`
80+
81+
3.1.4. `kubectl get globalbgpconfigs.crd.projectcalico.org -o wide`
82+
83+
## 4. Upgrade calico
84+
85+
4.1. (If you have RBAC enabled) Apply the updated RBAC manifest `kubectl apply -f https://docs.projectcalico.org/v2.5/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml` (this will revoke access to TPRs from calico-node)
86+
87+
4.2. Now you can upgrade Calico version to `v2.5.0` in your kubernetes Calico DaemonSet. (Make sure you reboot your calico-node pods one at a time if calico-node `updateStrategy` is not set to `RollingUpdate`)
88+
89+
4.3. Verify that everything is working as expected.
90+
91+
## 5. Delete the old TPRs
92+
93+
You can now delete your TPRs by running the following command:
94+
`kubectl delete -f https://raw.githubusercontent.com/projectcalico/calico/master/upgrade/v2.5/manifests/tprs.yaml`

‎upgrade/v2.5/manifests/crds.yaml

+61
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
apiVersion: apiextensions.k8s.io/v1beta1
2+
description: Calico Global Felix Configuration
3+
kind: CustomResourceDefinition
4+
metadata:
5+
name: globalfelixconfigs.crd.projectcalico.org
6+
spec:
7+
scope: Cluster
8+
group: crd.projectcalico.org
9+
version: v1
10+
names:
11+
kind: GlobalFelixConfig
12+
plural: globalfelixconfigs
13+
singular: globalfelixconfig
14+
15+
---
16+
17+
apiVersion: apiextensions.k8s.io/v1beta1
18+
description: Calico BGP Peers
19+
kind: CustomResourceDefinition
20+
metadata:
21+
name: bgppeers.crd.projectcalico.org
22+
spec:
23+
scope: Cluster
24+
group: crd.projectcalico.org
25+
version: v1
26+
names:
27+
kind: BGPPeer
28+
plural: bgppeers
29+
singular: bgppeer
30+
31+
---
32+
33+
apiVersion: apiextensions.k8s.io/v1beta1
34+
description: Calico IP Pools
35+
kind: CustomResourceDefinition
36+
metadata:
37+
name: ippools.crd.projectcalico.org
38+
spec:
39+
scope: Cluster
40+
group: crd.projectcalico.org
41+
version: v1
42+
names:
43+
kind: IPPool
44+
plural: ippools
45+
singular: ippool
46+
47+
---
48+
49+
apiVersion: apiextensions.k8s.io/v1beta1
50+
description: Calico Global Network Policies
51+
kind: CustomResourceDefinition
52+
metadata:
53+
name: globalnetworkpolicies.crd.projectcalico.org
54+
spec:
55+
scope: Cluster
56+
group: crd.projectcalico.org
57+
version: v1
58+
names:
59+
kind: GlobalNetworkPolicy
60+
plural: globalnetworkpolicies
61+
singular: globalnetworkpolicy

‎upgrade/v2.5/manifests/globalbgpconfig.yaml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: apiextensions.k8s.io/v1beta1
2+
description: Calico Global BGP Configuration
3+
kind: CustomResourceDefinition
4+
metadata:
5+
name: globalbgpconfigs.crd.projectcalico.org
6+
spec:
7+
scope: Cluster
8+
group: crd.projectcalico.org
9+
version: v1
10+
names:
11+
kind: GlobalBGPConfig
12+
plural: globalbgpconfigs
13+
singular: globalbgpconfig

‎upgrade/v2.5/manifests/tprs.yaml

+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
2+
apiVersion: extensions/v1beta1
3+
description: Calico Global Configuration
4+
kind: ThirdPartyResource
5+
metadata:
6+
name: global-config.projectcalico.org
7+
versions:
8+
- name: v1
9+
10+
---
11+
12+
apiVersion: extensions/v1beta1
13+
description: Calico IP Pools
14+
kind: ThirdPartyResource
15+
metadata:
16+
name: ip-pool.projectcalico.org
17+
versions:
18+
- name: v1
19+
20+
---
21+
22+
apiVersion: extensions/v1beta1
23+
description: Calico Global BGP Configuration
24+
kind: ThirdPartyResource
25+
metadata:
26+
name: global-bgp-config.projectcalico.org
27+
versions:
28+
- name: v1
29+
30+
---
31+
32+
apiVersion: extensions/v1beta1
33+
description: Calico Global BGP Peers
34+
kind: ThirdPartyResource
35+
metadata:
36+
name: global-bgp-peer.projectcalico.org
37+
versions:
38+
- name: v1

‎upgrade/v2.5/manifests/upgrade-job.yaml

+106
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
kind: ClusterRole
2+
apiVersion: rbac.authorization.k8s.io/v1beta1
3+
metadata:
4+
name: calico-25-migration
5+
rules:
6+
- apiGroups: [""]
7+
resources:
8+
- nodes
9+
verbs:
10+
- create
11+
- get
12+
- list
13+
- post
14+
- patch
15+
- update
16+
- watch
17+
- apiGroups: ["extensions"]
18+
resources:
19+
- thirdpartyresources
20+
verbs:
21+
- create
22+
- get
23+
- list
24+
- post
25+
- patch
26+
- update
27+
- watch
28+
- apiGroups: ["apiextensions.k8s.io"]
29+
resources:
30+
- customresourcedefinitions
31+
verbs:
32+
- create
33+
- get
34+
- list
35+
- post
36+
- patch
37+
- update
38+
- watch
39+
- apiGroups: ["projectcalico.org"]
40+
resources:
41+
- globalbgppeers
42+
- globalconfigs
43+
- globalbgpconfigs
44+
- ippools
45+
verbs:
46+
- create
47+
- get
48+
- list
49+
- post
50+
- patch
51+
- update
52+
- watch
53+
- apiGroups: ["crd.projectcalico.org"]
54+
resources:
55+
- globalfelixconfigs
56+
- bgppeers
57+
- globalbgpconfigs
58+
- ippools
59+
- globalnetworkpolicies
60+
verbs:
61+
- create
62+
- get
63+
- list
64+
- post
65+
- patch
66+
- update
67+
- watch
68+
69+
---
70+
71+
apiVersion: rbac.authorization.k8s.io/v1beta1
72+
kind: ClusterRoleBinding
73+
metadata:
74+
name: calico-25-migration
75+
roleRef:
76+
apiGroup: rbac.authorization.k8s.io
77+
kind: ClusterRole
78+
name: calico-25-migration
79+
subjects:
80+
- kind: ServiceAccount
81+
name: calico-25-migration
82+
namespace: default
83+
84+
---
85+
86+
apiVersion: v1
87+
kind: ServiceAccount
88+
metadata:
89+
name: calico-25-migration
90+
91+
---
92+
93+
apiVersion: batch/v1
94+
kind: Job
95+
metadata:
96+
name: calico-upgrade-v2.5
97+
spec:
98+
template:
99+
metadata:
100+
name: calico-upgrade-v2.5
101+
spec:
102+
serviceAccountName: calico-25-migration
103+
containers:
104+
- name: calico-upgrade
105+
image: gunjan5/upgrade:v2.5.16
106+
restartPolicy: Never

0 commit comments

Comments
 (0)