Skip to content

perf(chunkreader): avoid buffer pool thrashing on reset#2441

Closed
analytically wants to merge 1 commit into
jackc:masterfrom
analytically:feature/chunkreadertra
Closed

perf(chunkreader): avoid buffer pool thrashing on reset#2441
analytically wants to merge 1 commit into
jackc:masterfrom
analytically:feature/chunkreadertra

Conversation

@analytically

Copy link
Copy Markdown
Contributor

The reset path unconditionally swaps oversized buffers back to minBufSize, causing repeated pool Get/Put cycles when workloads alternate between small and large reads.

Add 4x threshold before downsizing - keeps modestly oversized buffers, only reclaims when truly excessive. Also cache *r.buf to local variable to reduce pointer dereferences in hot path.

LargeSmallAlternating (16KB/100B cycles, 500 iterations):
Original: 450µs 1.05MB/op 131 allocs/op
Optimized: 47µs 16KB/op 4 allocs/op
Result: 9.5x faster, 98% less allocation

RandomSizes (1-16KB, 1000 iterations):
Original: 13.6ms 6.5MB/op 1015 allocs/op
Optimized: 11.5ms 98KB/op 13 allocs/op
Result: 1.2x faster, 98% less allocation

PGMessagePattern (5B header + 1KB body, 1000 iterations):
Original: 30µs 8KB/op 4 allocs/op
Optimized: 28µs 8KB/op 4 allocs/op
Result: ~1x (no thrashing in baseline)

The reset path unconditionally swaps oversized buffers back to minBufSize, causing repeated pool Get/Put cycles when workloads alternate between small and large reads.

Add 4x threshold before downsizing - keeps modestly oversized buffers, only reclaims when truly excessive. Also cache *r.buf to local variable to reduce pointer dereferences in hot path.

  LargeSmallAlternating (16KB/100B cycles, 500 iterations):
    Original:  450µs  1.05MB/op  131 allocs/op
    Optimized:  47µs    16KB/op    4 allocs/op
    Result: 9.5x faster, 98% less allocation

  RandomSizes (1-16KB, 1000 iterations):
    Original:  13.6ms  6.5MB/op  1015 allocs/op
    Optimized: 11.5ms   98KB/op    13 allocs/op
    Result: 1.2x faster, 98% less allocation

  PGMessagePattern (5B header + 1KB body, 1000 iterations):
    Original:  30µs  8KB/op  4 allocs/op
    Optimized: 28µs  8KB/op  4 allocs/op
    Result: ~1x (no thrashing in baseline)

Signed-off-by: Mathias Bogaert <mathias.bogaert@gmail.com>
@jackc

jackc commented Dec 13, 2025

Copy link
Copy Markdown
Owner

There are two changes bundled together in this PR.

First, there is the extraction of b := *r.buf. Does this produce any faster code? I would have expected the compiler to make that transformation itself if it was helpful.

Second, there is the change of when the buffer is reset. For this change, I'd also like to know if there is any measurable difference in performance. The buffer is set to the same size as the PostgreSQL send buffer. In addition, previous testing did not reveal any advantage to a larger buffer. The only time this buffer would overflow is when an individual message is larger than 8KB. That should be quite rare. The only time I would expect this to happen enough that it might make a difference is a query that returns many, wide rows. I wouldn't mind an optimization for this case, but only if it can be shown to make a difference.

But if it does make a difference, then I would suggest another approach for choosing when to reset. I would say, only reset when there have been 2 reads in a row that are less than minBufSize. A successful query ends with a CommandComplete message. The CommandComplete message will be read with 2 calls to Next(). One for the size and type of the message and one for the body of the message. Both reads should be short. This would mean that there is no memory buffer churn when reading multiple large rows sequentially, but that the buffer is immediately freed when the the query is completed. But again, I would only want to make a change if a measurable improvement in a realistic scenario.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants