Rethink idle and keep alive timeouts

[KirillLykov]

Problem

Users report me in DM that they see a lot of timeout errors from leaders they are connecting to.
They believe that it is due to we set recently in quic the keepalive interval to 45 s, and timeout to 60.
@stablebits also pointed out that typically NAT are configured with timeout 30s in mind.
Firedancer is using idle timeout 2s see and keepalive 1s see.

Folks tried to use the values from FD and the errors have disappeared from their logs.

Proposed Solution

Investigate.

[t-nelson]

do we know why we selected such large values?

[stablebits]

20–30s timeouts for UDP flows on NAT devices are quite common. e.g., Google Cloud’s default is 30s (https://cloud.google.com/nat/docs/tuning-inbound-nat-logging-and-timeouts).

In that case, a keepalive interval of 10–15 seconds should be fine.

ICE default (and min) recommendation is 15 seconds -- https://datatracker.ietf.org/doc/html/rfc5245#section-10.

[alexpyattaev]

do we know why we selected such large values?

See relevant discussion in anza-xyz/solana-sdk#29 (comment) .

[KirillLykov]

So how do we resolve this problem? I don't get why FD and agave are using different constants. If not this, I would assume that we need to collect the data of occurrence of these errors and find the optimal parameters (minimizing both errors and ping traffic). Btw not sure if this PING frames traffic is an issue

[alexpyattaev]

Let us stick with 15s.

[stablebits]

btw., normally the client side should send keepalives if necessary. Is this part of our SDK (example) client implementation?

[KirillLykov]

btw., normally the client side should send keepalives if necessary. Is this part of our SDK (example) client implementation?

it is implemented in quinn, we just set up keep alive and other TransportConfig parameters:

agave/tpu-client-next/src/quic_networking.rs

Line 37 in 5f0b43d

res.keep_alive_interval(Some(QUIC_KEEP_ALIVE));