Description
Description:
What issue is being seen? Describe what should be happening instead of
the bug, for example: The expected value isn't returned, etc.
After Gateway is deployed we get External/Internal IP-address from GKE-loadbalancer.
During daily operations the gateway randomly looses connection to the underlying service, and get status PROGRAMMED: False, but the service still have the IP-address assigned"
We initially suspected that this could have something to do with the HPA and autoscaling, but we have ensured that we are not setting any static replicas and that we are only setting minReplicas and maxReplicas with the HPA.
we have also done some load testing to ensure that the gateways are not set to "PROGRAMMED: False" during scaling operations.
The Message from Gateway is when it set to False is this..
"No addresses have been assigned to the Gateway"
and we are hitting this function.
https://github.com/envoyproxy/gateway/blob/main/internal/gatewayapi/status/gateway.go#L143
We have test both version 1.30 and version 1.32 and still hit the issue..
$ k get gateway -n gw
NAME CLASS ADDRESS PROGRAMMED AGE
gw-public eg-public 35.XX.XXX.XX False 8d
Sometimes GATEWAY is set to FALSE.
$ k get gateway -n gw
NAME CLASS ADDRESS PROGRAMMED AGE
gw-public eg-public False 8d
but the service for gateway gw-public/internal still has IP Assigned
$ k get svc -n gw
gw-public-cc293c26 LoadBalancer 10.145.6.0 35.XX.XXX.XX 80:31781/TCP,443:30118/TCP 8d
After around 15 - 60 minutes Gateway are coming back with PROGRAMMED True. and we can reach our site/url.