gRPC xDS client stops working after a period of inactivity

[rdesgroppes]

We've been using your xDS Server for gRPC on Kubernetes, which suits our needs and is easy to use.
However, we are experiencing problems when the channels switch to idle mode. Disabling idleness helps, but we'd now like to restore the default behavior.

I'm far from being an expert on the matter, that's why I'm kindly requesting your point of view on the issue that was originally filed against the gRPC-Java client: grpc/grpc-java#9632.

According to grpc/grpc-java#9632 (comment):

[...] This seems like a go-control-plane bug to me. Once grpc-java sends a request with the removed resources, go-control-plane needs to honor that; it should not need any subsequent ACK/NACK. [...]

Is the xDS Server for gRPC on Kubernetes behaving like the aforementioned go-control-plane?

[whs]

Internally this software use go control plane, so any bugs by them will have to be fixed upstream.

Not sure if related but we do find idle bugs in grpc go, and currently our workaround is to pin grpc at a specific version that do not have revamped idle behavior

[rdesgroppes]

Not sure if related but we do find idle bugs in grpc go, and currently our workaround is to pin grpc at a specific version that do not have revamped idle behavior

Right, idleness is enabled by default since gRPC-Go 1.59.0. We had to create channels with an additional option (grpc.WithIdleTimeout(0)) to get around it.

[hanikesn]

We're facing a very similar issue using grpc v1.65.0, where channels who resume from idle mode won't resolve endpoints via xDS correctly.

Here's the error so this issue is easier to find:

rpc error: code = Unavailable desc = name resolver error: produced zero addresses

I'm wondering whether that's something worth reporting upstream.

[hanikesn]

Looks like this is the upstream issue to track:

• Resource not re-sent if clients unsubscribe and then resubscribe envoyproxy/go-control-plane#431

[hanikesn]

Upstream fix is on the way:

• cache/simple: handle unsubscription for a resource followed by resubscription envoyproxy/go-control-plane#1293
• [Sotw][Linear cache] Ensure watches are properly considering subscription changes to not miss new resources envoyproxy/go-control-plane#1301