Skip to content

QUIC reliable-ping exempt-cap cannot enforce its awareness-max floor at construction #178

Description

@al8n

Summary

QuicOptions::max_reliable_ping_exempt_pops_per_pass (default 8) bounds how many front-parked ReliablePing dials the coordinator's service_peer_bucket may attempt past an exhausted per-pass dial budget. Its correctness floor is "an honest reliable-ping fallback is never deferred to its probe deadline" — i.e. the cap must be ≥ the maximum number of concurrent same-peer reliable-ping fallbacks, which is bounded by the membership awareness maximum (awareness_max_multiplier, default 8). At the default (cap 8 = awareness default 8) it is correct.

The gap: QuicOptions::validate() can only enforce cap >= 1, not cap >= awareness_max, because the QUIC coordinator cannot read the awareness bound at construction time — it lives on the inner membership Endpoint, whose config is private and whose module is frozen (no accessor). So a config that pairs a lowered ping-cap with a default/raised awareness max (or a raised awareness max with the default cap) passes validation but can defer an honest reliable-ping fallback behind the exhausted budget until its probe cumulative deadline — retiring it into a false Suspect of a live peer.

This is a soundness gap in a configuration guard, not a default-config defect: the shipped defaults are safe, and it only bites a specific mis-pairing of two independently-set knobs. It was surfaced during the #176 deadline-phased-dial-wake work (PR #176), where the ping exempt-pop mechanism was introduced; the field currently carries a doc warning ("must stay >= the awareness max or honest pings defer to a false Suspect") as the interim mitigation.

Where

  • memberlist-proto/src/quic/crypto/mod.rsmax_reliable_ping_exempt_pops_per_pass field, with_*/accessor, MaxReliablePingExemptPopsPerPassZero validation (the >= 1 floor is all that can be enforced today).
  • memberlist-proto/src/quic/mod.rsservice_peer_bucket's front-ReliablePing exempt-pop reads the cap; the effective bound should be max(configured_cap, awareness_max).
  • memberlist-proto/src/endpoint/mod.rs — the frozen membership endpoint that owns the awareness config; currently exposes no accessor for the awareness max.

Options

  1. Additive endpoint accessor + enforce the floor at use (preferred). Add a small read-only accessor on Endpoint (e.g. awareness_max() / the relevant probe-fanout bound) — additive, no FSM/wire change, so within the frozen-machine additive-extension rule. The coordinator then takes the effective exempt-cap as max(self.cfg.max_reliable_ping_exempt_pops_per_pass(), self.ep.awareness_max()) at the exempt-pop site, making the floor correct by construction regardless of how the two knobs are set. No cross-struct validation needed; the doc warning can be dropped.
  2. Derive the cap entirely from the awareness max (drop the independent knob). Simpler surface, but removes the operator's ability to tune it above the awareness bound (a legitimate want if a driver-facing ping path can stack more fallbacks).
  3. Cross-struct validation at composition time. Validate cap >= awareness_max where both configs are assembled (the driver/coordinator constructor), rather than inside QuicOptions::validate(). Keeps the knobs independent but enforces the pairing where both are visible.

Recommendation: option 1 — the additive Endpoint accessor plus a point-of-use max(...) floor. It closes the gap by construction, preserves the tunable knob, needs no wire/FSM change, and lets the interim doc warning be removed.

Scope

Foundations phase; fits alongside #165 (core Endpoint) since the fix is a small additive accessor on the frozen endpoint. Not a blocker for #176 (defaults are safe; the doc warning is the interim guard).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions