New Repository https://forgejo.anyops.de/AnyOps/antrea2cilium
I have no tanzu to mess around, so I use kind
It's meant for a demo, so I run it in a tmux session and some panes will be opened to watch what is happening:
- in the cluster
- possible nginx errors
- and cilium state
This is based on Tutorial: How to Migrate to Cilium by Nico Vibert and Migrating from Calico Lab
I have added some tmux magic in the scripts, so you can see what's happening without arranging terminals. Have a big monitor :)
https://asciinema.org/a/723314
My scripts expect a few installed commands
- tmux (used for automatically splitting terminal to be used for output)
- docker and compose plugin
- kind
- kubectl
- k9s
- helm
- cilium
- jq
I added some 'infrastructure' for convenience which is running in docker
To speed things up, as images are pulled multiple time alone for a single run and to avoid waiting and hitting rate limits I added mirrors to be used (props to Duffie Cooley)
- docker
- github
- redhat
- vmware
They are deployed with network kind
To check availibility of workload without different ports, while control-planes are upgraded This needs the running kind setup
- 3 control-planes
- 4 worker-nodes
You need at least 2 for redundancy... but this should be obvious :)
kind will be setup with config for the mentioned mirrors and without CNI
just applying a manifest
Cilium will be installed without being activated and with special parameters to work with workload using a different CNI. Aditionally a CiliumNodeConfig will be created, to define when a node is switched to cilium.
Migration is triggered node by node by
- cordon and then drain the nodes
- setting the label of nodes to be migrated io.cilium.migration/cilium-default to "true"
- reboot and uncordon the node(s)
The cilium migration will be finalized by removing parameters needed for co-working with existing antrea
kubectl delete installation manifest
Start tmux and then execute scripts in oder:
# docker registry mirrors in kind network
$ ./00_registry_mirrors.sh
# kind cluster without cni, k9s to watch all pods will be started in a tmux pane
$ ./01_create_cluster.sh
# deploy antrea
$ ./02_install_antrea.sh
# start a loadbalancer in docker within kind network
$ ./03_loadbalancer.sh
# run curl in a loop and output any non-200 state in a tmux pane
$ ./04_check_nginx.sh
# create nginx deployment
$ ./05_nginx.sh
# check antreas subnets with configured nets for cilium
$ ./06_check_subnets.sh
# should be no conflict, so deploy cilium with migration setting
# and add CiliumNodeConfig for switching node by node
$ ./07_deploy_cilium.sh
# call next script for each node.
$ ./08_switch_node.sh antrea2cilium-worker
# remove migration settings
$ ./09_finalize_migration.sh
# remove antrea
$ ./11_delete_antrea.sh
# finished, no errors destroy everything including tmux session.
$ ./12_stop.sh To watch cilium state, cilium.sh will start another tmux pane calling cilimu status every few seconds
# can be called any time
$ ./cilium.sh