Description
/kind bug
What happened:
The default behavior of OCCM is not to create health monitors for load balancers, unless enabled globally via config file or for individual LoadBalancers using the loadbalancer.openstack.org/enable-health-monitor annotation.
With the default OCCM configuration, if a LoadBalancer service is created with externalTrafficPolicy: Local, the OpenStack LB will be configured incorrectly and with no warning. Importantly, the failure mode here can be extremely confusing and difficult to pin down: not every workload behind a LoadBalancer configured this way would fail necessarily, depending on replica count and distribution of pods, and some workloads might fail in different ways than others or otherwise exhibit unexpected behavior.
What you expected to happen:
OCCM should either:
- Error when attempting to reconcile a LoadBalancer service with
externalTrafficPolicy: Localif health monitors are disabled, or - Always create health monitors when reconciling a LoadBalancer service with
externalTrafficPolicy: Local, regardless of the presence of the annotation or the config option.
How to reproduce it:
- Create a cluster with more than one worker node and OCCM configured with only global settings and everything else default
- Deploy a workload with 1 replica (or fewer replicas than worker nodes) behind a LoadBalancer service with
externalTrafficPolicy: Local - Repeatedly attempt to make a request to the service
- Observe that only one out of every [# worker nodes] requests goes through
Anything else we need to know?:
Would be happy to contribute a fix for this
Environment:
- openstack-cloud-controller-manager(or other related binary) version: 1.31.1
- OpenStack version: yoga
- Others: