Configure Networking

[nikhiljha]

Cilium supports a whole bunch of stuff that isn't (properly) configured yet.

• LoadBalancer
  • set aside some IPs for the Kubernetes cluster to use (.81-.90)
  • configure cilium to take those IPs
• Hubble (see Observability and Monitoring #5)
• Enable XDP to go at very hihg speed
• IPv6 Dual Stack

Bonus Points: Enable multi-cluster mesh so we can run untrusted pods on one w/ Kata Containers and run OCF things on the other.

[nikhiljha]

LoadBalancer was done through MetalLB (7aade48, de3169b).

[nikhiljha]

XDP is not happening (our QEMU is too old)

[emmatyping]

We probably could use XDP on Bullseye

[nikhiljha]

@ethanhs It's more that we weren't running on bare metal before. This should be no problem with dedicated hosts.

[emmatyping]

It seems dual stack is enabled by default in Kubernetes 1.21, and I think cilium mostly supports it? It seems we mostly just need to configure the service/node CIDRs with v4 and v6 https://kubernetes.io/docs/concepts/services-networking/dual-stack/#configure-ipv4-ipv6-dual-stack

Also Hubble is enabled, it just needs keycloak set up to be in front of it. (ref #3)