websocket: subscribeNewBlock broadcast re-serializes/compresses payload per connection (O(N) → O(distinct ids))

[cranycrane]

Summary

The subscribeNewBlock broadcast re-serializes (and re-compresses) an identical payload once per connection. In outputLoop each connection independently calls c.conn.WriteJSON(m) (server/websocket.go:730), so a single new-block notification to N subscribers performs N JSON marshals and N flate compressions (the upgrader sets EnableCompression: true, server/websocket.go:198) of data that differs only by the client-chosen WsRes.ID.

Toward the AGENTS.md target of ≥20 000 websocket connections, and because Blockbook runs one instance per coin, every new block fans the same payload out to (potentially) all connected clients — 20 000 marshal+compress operations per block.

Why this is a good fit for subscribeNewBlock specifically

• Every Trezor Suite instance subscribes to new blocks; a per-coin Blockbook therefore broadcasts one identical payload to (nearly) all connections on each block.
• gorilla/websocket (already a dependency, v1.5.0) provides NewPreparedMessage / WritePreparedMessage, which serialize+compress once and reuse the cached wire frame across connections.
• The only per-connection variance is WsRes.ID. In Suite the id is not a shared constant — it is a per-connection counter that resets to 0 on each connect (blockchain-link → @trezor/utils/createDeferredManager, initialId = 0, counter++) and is shared across all requests on that connection. The id assigned to subscribeNewBlock is therefore "how many requests preceded the subscribe on that connection", which varies per user (handshake, account-discovery calls, reconnects). The result is a small set of low integers (distinct ids ≪ N) — a handful to low hundreds across the fleet, not one value and not N. Grouping prepared frames by distinct id thus reduces serialize+compress from O(N) to O(distinct ids).

Proposed change

Build one websocket.NewPreparedMessage(TextMessage, marshal(WsRes{ID, Data})) per distinct id in the broadcast path, then fan it out via WritePreparedMessage. ~30 lines; the per-connection socket write is unchanged.

Measured speed-up (subscribeNewBlock, N = 10 000 connections)

Benchmark uses gorilla's exact compression path (flate level 1, no-context-takeover) so the cost matches the real WriteJSON write path. serOps = marshal+compress operations per broadcast.

distinct ids	ns/op (before → after)	allocs/op (before → after)	bytes/op (before → after)	serOps (before → after)	speed-up
1 — best case (all share id, or id hoisted out of body)	106 ms → 0.11 ms	120 020 → 12	7.1 MB → 6.7 KB	10 000 → 1	~965×
100 — Suite-like (small distinct set)	127 ms → 1.5 ms	127 127 → 1 280	9.4 MB → 111 KB	10 000 → 100	~84×
10 000 — worst case (every client a unique id)	153 ms → 125 ms	~120 k → ~120 k	~7.3 MB → ~8.9 MB	10 000 → 10 000	~1.2× (none)

Apple M4. The win is a function of id cardinality; Suite sits between the top two rows (small distinct set of low integers).

Implementation notes — preparing all the frames at once

Route A — group by id (no protocol change). Per broadcast:

1. Take a short critical section under newBlockSubscriptionsLock and build a map[string]*websocket.PreparedMessage keyed by distinct id.
2. For each distinct id: json.Marshal(WsRes{ID: id, Data: data}) once, then websocket.NewPreparedMessage(websocket.TextMessage, b).
3. Fan out: enqueue a WsRes{prepared: frameForThisID} to each subscriber's out channel; outputLoop calls WritePreparedMessage instead of WriteJSON.

Frames prepared per broadcast = number of distinct ids (a handful to low hundreds). Note gorilla computes the flate compression lazily: NewPreparedMessage only eagerly builds the uncompressed frame; the compressed frame is built on the first WritePreparedMessage from a compression-enabled connection and cached (thread-safe sync.Once) for every other connection sharing that frame. So you pay marshal-once + compress-once per distinct id no matter how many thousands of connections share it.

Route B — a single frame for everyone (needs an id change). To prepare exactly one frame for all N connections, the id must not live inside the compressed body. It currently does ({"id":…,"data":…}), and Suite routes each notification by matching id to its subscription (baseWebsocket.ts), so it cannot simply be dropped. This requires a protocol change on both Blockbook and Suite — notifications keyed by a stable channel/subscription identifier rather than the per-connection request id — after which it is one marshal + one compress per block fanned to all N. This is the cardinality-independent version (guarantees the top row regardless of client id behavior).

Getting it right when preparing at once:

• Lock hold time — building the distinct-id→frame map is cheap (marshal + NewPreparedMessage, no compression yet); keep the fan-out/enqueue (which can back-pressure on a slow connection's out channel) out of the lock where possible.
• Concurrency — a *PreparedMessage is immutable once built and safe to WritePreparedMessage concurrently from every per-connection outputLoop goroutine.
• Memory — you hold O(distinct ids) frames alive for the broadcast (each caches its (un)compressed bytes); hundreds × a few hundred bytes is negligible, and it replaces the current O(N) transient marshal/compress buffers.
• Lazy-compression placement — the first connection in each id-group pays the compression inline on its outputLoop; if that matters, prime each frame once right after building so the compression happens off the hot path.

Honest scope / caveats

• The ~1000× applies to the marshal+compress component only. Each connection still needs its own socket write, so end-to-end broadcast CPU drops ~5–6× (that component is nonetheless the largest single cost of the broadcast).
• No steady-state RAM reduction — the allocations removed are transient garbage. Peak memory during a broadcast to a slow/back-pressured fleet does improve (one shared frame per id vs. up to N in-flight buffers).
• Absolute CPU saved is modest on slow-block chains (~1–2% of a core amortized on Ethereum @ ~12 s) and becomes meaningful on fast-block chains (~10–20% of a core at ~1–2 s blocks) and as connection counts approach 20 000. It also removes a per-block CPU burst / notification tail-latency.
• Same technique generalizes to any one-payload-to-many-connections path (sendOnNewTx), but subscribeNewTransaction is gated off by default and not used by Suite, and sendOnNewTxAddr has per-address fan-out ≈1 so it does not benefit.