s2n-quic MTU probing is sending too many packets when probing a low MTU

[maddeleine]

Security issue notifications

If you discover a potential security issue in s2n-quic we ask that you notify
AWS Security via our vulnerability reporting page. Please do not create a public github issue.

Problem:

We've noticed that if s2n-quic is configured with an initial MTU of 8940 and the network MTU is 1450, there will be a lot of packets sent, ~100, to fully probe this value. This seems like a mistake, a basic s2n-quic handshake only sends ~10 packets, so seeing 100 packets with the handshake + probing seems wrong. Since the MTU algorithm is a binary search, in order to probe from 8940 to 1450 MTU, there should be ~9 rounds of probing. With 3 probe retries we expect ~30 packets sent for fully probing this MTU. But we're sending way more than that. Minimally, we should investigate what's going on here. This matters because there are limits on the amount of packets that can be received by a given host, and with this packet count it puts us in danger of going above that limit.

Need By Date:

N/A

[WesleyRosenblum]

With the current MTU probing design, this is working as designed, though it is not optimal for the s2n-quic-dc usecase.

Each candidate MTU probe size is tried 3 times. Since the network MTU is lower than the probe size, the packets are getting lost. But since no other data is being sent over the connection and triggering ACKs from the peer, we need a way to confirm that the probes are lost. The QUIC protocol's mechanism for this is PTO (probe timeout) packets. For each of the MTU probes that are suspected lost, s2n-quic will send 2 PTO packets to trigger the peer to reply with an ACK. When the ACK is received (which presumably does not acknowledge the oversized MTU probe), we are able to declare the MTU probe lost.

So by design, s2n-quic is sending 2X the number of MTU probe packets as PTO packets. So if the binary search math is saying 9 different probe values, that could result in 93 MTU probes + 93*2 PTO packets, totaling up to 81 packets.