stream: clientStream has 33B alignment padding; grouping bool fields reduces alloc class from 288B to 256B

[gidotencate]

Problem

clientStream in stream.go has 7 bool fields scattered among pointer-sized fields, creating 33 B of alignment padding. The struct is 280 B, which rounds to the 288 B Go allocator size class.

Proposed fix

Group all bool fields at the end of the struct, after the last pointer/int field. This eliminates 32 B of padding (1 B remains), packing the struct to 248 B → 256 B allocator size class.

Result: 32 B saved per RPC stream, unconditionally.

Benchmarks

Allocation size class

BenchmarkAlloc/before   288 B/op   1 allocs/op
BenchmarkAlloc/after    256 B/op   1 allocs/op

GC pressure (10 000 stream allocations):

Before: 2.82 MiB  (288 B/stream)
After:  2.52 MiB  (256 B/stream, −10.8%)

At 100k concurrent streams this is 3.2 MB of heap saved. With GOGC=100 this delays each GC cycle proportionally, and frees L3 cache capacity for codec buffers, frame headers, and connection state.

Hot-path concurrent benchmark

withRetry is called on every SendMsg, RecvMsg, Header, and CloseSend. The benchmark runs withRetry and finish concurrently on the same struct to measure real MESI cache-coherency cost.

GOMAXPROCS	Before (288B)	After (256B)	Delta
1	6.92 ns	6.80 ns	−1.7%
4	8.14 ns	8.26 ns	+1.5%
6	8.17 ns	8.22 ns	+0.6%
8	8.14 ns	8.26 ns	+1.4%
16	8.21 ns	8.23 ns	+0.3%

No regression. The ±1.5% variation is within benchmark noise (6 runs each).

Cache-line co-location

The fix also improves field co-location for the two hottest access patterns:

Field pair	Before	After
sentLast + attempt (SendMsg checks sentLast, withRetry reads attempt)	CL1 + CL3	both CL3 ✓
committed + attempt (withRetry fast path reads both)	both CL3 ✓	both CL3 ✓
mu + data fields	CL2 lock / CL3 data	CL2 lock / CL3 data (unchanged)

Moving sentLast to CL3 means SendMsg's first field check pre-warms the same cache line that withRetry will read — one fewer cache-line fetch per send operation compared with the original layout.

Alternative considered: "surgical" grouping

I benchmarked placing only the mu-guarded bools (firstAttempt, finished, committed) adjacent to mu on CL2, with unguarded bools at the end. This co-locates mu + committed on the same cache line.

Result: 20–22% slower under concurrent access at all core counts:

GOMAXPROCS	Proposed fix	Surgical grouping	Delta
4	8.26 ns	9.79 ns	+18.5%
8	8.26 ns	9.94 ns	+20.4%
16	8.23 ns	10.03 ns	+21.9%

Root cause: placing finished (written by finish()) and committed (read by withRetry()) both on CL2 alongside mu means both goroutines compete on the same cache line for every lock cycle — lock state change + data write + data read all on CL2, compounding MESI invalidation. Keeping data fields on CL3 separate from the lock on CL2 is the correct arrangement.

Verification

go test -race -cpu 1,4 -timeout 7m ./...   # all 74 packages pass
go vet ./...                                # clean
goimports -l stream.go                     # no output

No behaviour change. Guard annotations preserved in field comments.

Happy to submit a PR if this direction looks good.

[easwars]

Would be happy to review a PR for this. Do keep in mind that some of the bools are protected by a mutex while some aren't. So continuing to have some comments in there to highlight that would be helpful.

Would something similar apply to the serverStream struct as well?

[easwars]

FYI @eshitachandwani : We might want to consider something like this for the proc_stream once the feature is ready.