perf: reconnect stampede after rolling deploy drains WebSocket connections

[TheSentinel454]

Problem

The relay's graceful shutdown sends a 1012 (Service Restart) close frame to every connected WebSocket. With drainJitterMs: 0 (the current default in values.yaml), all sockets on a pod close simultaneously. Clients reconnect immediately, hitting the surviving pods.

Cascade effect

1. Pod A drains: N clients disconnect simultaneously
2. N clients reconnect to Pod B and Pod C within 1-2 seconds
3. Each reconnect triggers: WebSocket upgrade, NIP-42 AUTH, db.lookup_community_by_host, db.is_member, subscription REQ, db.query_events for each subscribed channel
4. The burst of N simultaneous query_events calls hits the DB pool
5. If N exceeds the pool's headroom, requests start queuing on acquire_timeout
6. The draining pod's readiness goes 503 and K8s shifts ALL traffic to survivors, amplifying the burst

What exists today

The chart has drainJitterMs support (configurable up to 20s), but it defaults to 0. The connection manager (state.rs) implements the jittered drain correctly.

What's missing

• No reconnect-rate metric: there is no gauge or counter for WebSocket connections established per second. The existing buzz_ws_connections_active tracks the total, not the rate.
• No per-request DB pool wait metric: when the pool is saturated, the time spent waiting to acquire a connection is hidden inside the datastore span. A separate buzz_db_pool_acquire_wait_seconds histogram would distinguish "query was slow" from "waited for a connection."
• Default jitter is off: the safest default for production should be non-zero. A 5-10s jitter spreads the reconnect herd across the grace period.

Proposed changes

1. Add buzz_ws_connections_per_second or a connection-rate counter.
2. Add buzz_db_pool_acquire_wait_seconds histogram to surface pool saturation directly.
3. Consider a non-zero default for drainJitterMs (5000-10000ms).
4. Implement client-side exponential backoff guidance via the 1012 close frame reason payload.

Priority

Medium — jitter is already implemented but defaulted off; the metrics are the missing diagnostic piece.

🤖 AI review update (2026-08-23)

Absorb #30 here. Remove the missing-rate-metric claim because buzz_ws_connections_total already exists and rate() provides the reconnect rate. Define success as a bounded reconnect distribution during a staged rolling deploy, use a positive multi-replica/production jitter without penalizing single-replica defaults, and verify exponential backoff with full jitter in every first-party client rather than introducing an ad-hoc JSON close-reason protocol.