storeapi: enable batching of series per SeriesResponse

[adezxc]

This PR adds a batch field to the SeriesResponse message and allows for sending/receiving
multiple Series per request.

The size of the batch is set using query.series-response-batch-size flag and is included in SeriesRequest.
If the StoreAPI is running an older version it will just send series one-by-one, this way keeping backwards-compatibility.

Original suggestion: #7929

• I added CHANGELOG entry for this change.

Changes

• Add SeriesBatch batch field to SeriesResponse
• Create batchableServer struct that accumulates forwarded responses into an array.
• Update RespSet and *seriesServer.Send functions to handle batches correctly.

Verification

Some tests, benchmarks are included. We are running this change in production for a few weeks and the network savings are big with query.series-response-batch-size=50, queriers also seem to use a bit less CPU.

[adezxc]

Benchmark results with fake, repeating labels on my laptop:

$ benchstat -row ".name /series /sample_count_per_series" -col "/batch_size" ~/playground/benchmark_results.txt
goos: linux
goarch: amd64
pkg: github.com/thanos-io/thanos/pkg/query
cpu: AMD Ryzen 7 PRO 8840U w/ Radeon 780M Graphics
                              │      1       │                  10                  │                 100                  │                 1000                 │                10000                │
                              │    sec/op    │    sec/op     vs base                │    sec/op     vs base                │    sec/op     vs base                │   sec/op     vs base                │
GRPCServerBatching 100 100      771.0µ ±  1%   448.5µ ±  3%  -41.83% (p=0.000 n=10)   535.0µ ± 10%  -30.61% (p=0.000 n=10)   483.8µ ± 24%  -37.24% (p=0.000 n=10)   530.3µ ± 6%  -31.22% (p=0.000 n=10)
GRPCServerBatching 100 240      920.7µ ±  5%   646.9µ ±  3%  -29.73% (p=0.000 n=10)   769.5µ ±  3%  -16.42% (p=0.000 n=10)   739.0µ ±  3%  -19.73% (p=0.000 n=10)   731.2µ ± 3%  -20.58% (p=0.000 n=10)
GRPCServerBatching 1000 100     6.797m ± 14%   2.657m ±  5%  -60.90% (p=0.000 n=10)   3.134m ±  2%  -53.88% (p=0.000 n=10)   3.622m ±  3%  -46.71% (p=0.000 n=10)   3.557m ± 5%  -47.67% (p=0.000 n=10)
GRPCServerBatching 1000 240     7.950m ±  4%   4.720m ±  3%  -40.63% (p=0.000 n=10)   4.261m ±  5%  -46.40% (p=0.000 n=10)   6.270m ±  2%  -21.14% (p=0.000 n=10)   6.420m ± 3%  -19.25% (p=0.000 n=10)
GRPCServerBatching 10000 100    65.57m ±  3%   23.86m ±  3%  -63.61% (p=0.000 n=10)   27.01m ±  3%  -58.80% (p=0.000 n=10)   24.74m ±  5%  -62.26% (p=0.000 n=10)   35.06m ± 7%  -46.54% (p=0.000 n=10)
GRPCServerBatching 10000 240    79.53m ±  6%   41.26m ± 10%  -48.12% (p=0.000 n=10)   36.85m ± 12%  -53.66% (p=0.000 n=10)   45.24m ±  1%  -43.11% (p=0.000 n=10)   65.08m ± 4%  -18.16% (p=0.000 n=10)
GRPCServerBatching 100000 100   635.9m ±  6%   227.7m ±  4%  -64.19% (p=0.000 n=10)   269.1m ±  5%  -57.68% (p=0.000 n=10)   282.1m ± 13%  -55.64% (p=0.000 n=10)   280.1m ± 7%  -55.95% (p=0.000 n=10)
GRPCServerBatching 100000 240   750.3m ±  6%   451.1m ± 25%  -39.88% (p=0.000 n=10)   407.8m ± 12%  -45.65% (p=0.000 n=10)   455.3m ± 29%  -39.32% (p=0.000 n=10)   555.5m ± 3%  -25.96% (p=0.000 n=10)
geomean                         23.58m         11.77m        -50.08%                  12.51m        -46.93%                  13.60m        -42.33%                  15.39m       -34.73%

                              │       1       │                  10                   │                 100                  │                 1000                 │                 10000                 │
                              │     B/op      │     B/op       vs base                │     B/op      vs base                │     B/op      vs base                │     B/op       vs base                │
GRPCServerBatching 100 100      1052.4Ki ± 1%    585.2Ki ± 1%  -44.39% (p=0.000 n=10)   602.0Ki ± 2%  -42.80% (p=0.000 n=10)   604.6Ki ± 3%  -42.55% (p=0.000 n=10)   600.9Ki ±  3%  -42.91% (p=0.000 n=10)
GRPCServerBatching 100 240      1189.4Ki ± 1%   1114.4Ki ± 1%   -6.30% (p=0.000 n=10)   932.1Ki ± 2%  -21.63% (p=0.000 n=10)   919.5Ki ± 3%  -22.69% (p=0.000 n=10)   936.0Ki ±  5%  -21.30% (p=0.000 n=10)
GRPCServerBatching 1000 100      8.338Mi ± 1%    4.404Mi ± 1%  -47.19% (p=0.000 n=10)   5.550Mi ± 2%  -33.44% (p=0.000 n=10)   4.949Mi ± 3%  -40.64% (p=0.000 n=10)   4.966Mi ±  3%  -40.45% (p=0.000 n=10)
GRPCServerBatching 1000 240      9.373Mi ± 0%    7.845Mi ± 1%  -16.30% (p=0.000 n=10)   8.428Mi ± 3%  -10.08% (p=0.000 n=10)   8.498Mi ± 2%   -9.34% (p=0.000 n=10)   8.426Mi ±  4%  -10.11% (p=0.000 n=10)
GRPCServerBatching 10000 100     68.02Mi ± 0%    36.25Mi ± 1%  -46.71% (p=0.000 n=10)   40.88Mi ± 2%  -39.89% (p=0.000 n=10)   38.19Mi ± 2%  -43.86% (p=0.000 n=10)   45.98Mi ± 11%  -32.40% (p=0.000 n=10)
GRPCServerBatching 10000 240     78.89Mi ± 0%    62.62Mi ± 0%  -20.63% (p=0.000 n=10)   65.20Mi ± 1%  -17.36% (p=0.000 n=10)   67.44Mi ± 2%  -14.51% (p=0.000 n=10)   88.11Mi ±  3%  +11.69% (p=0.002 n=10)
GRPCServerBatching 100000 100    611.8Mi ± 0%    342.6Mi ± 0%  -44.00% (p=0.000 n=10)   338.4Mi ± 1%  -44.68% (p=0.000 n=10)   336.4Mi ± 1%  -45.02% (p=0.000 n=10)   405.5Mi ±  3%  -33.73% (p=0.000 n=10)
GRPCServerBatching 100000 240    762.0Mi ± 0%    589.5Mi ± 0%  -22.64% (p=0.000 n=10)   573.6Mi ± 0%  -24.72% (p=0.000 n=10)   682.8Mi ± 6%  -10.39% (p=0.000 n=10)   888.6Mi ±  5%  +16.61% (p=0.000 n=10)
geomean                          26.36Mi         17.75Mi       -32.68%                  18.37Mi       -30.33%                  18.41Mi       -30.17%                  20.63Mi        -21.76%

                              │      1      │                 10                  │                 100                 │                1000                 │                10000                │
                              │  allocs/op  │  allocs/op   vs base                │  allocs/op   vs base                │  allocs/op   vs base                │  allocs/op   vs base                │
GRPCServerBatching 100 100      6.219k ± 0%   2.060k ± 0%  -66.88% (p=0.000 n=10)   1.577k ± 0%  -74.64% (p=0.000 n=10)   1.577k ± 0%  -74.64% (p=0.000 n=10)   1.577k ± 0%  -74.64% (p=0.000 n=10)
GRPCServerBatching 100 240      6.895k ± 0%   2.730k ± 0%  -60.41% (p=0.000 n=10)   2.214k ± 0%  -67.89% (p=0.000 n=10)   2.213k ± 0%  -67.90% (p=0.000 n=10)   2.214k ± 0%  -67.89% (p=0.000 n=10)
GRPCServerBatching 1000 100     59.18k ± 0%   17.97k ± 0%  -69.64% (p=0.000 n=10)   13.22k ± 0%  -77.66% (p=0.000 n=10)   12.52k ± 0%  -78.85% (p=0.000 n=10)   12.52k ± 0%  -78.85% (p=0.000 n=10)
GRPCServerBatching 1000 240     66.62k ± 0%   24.61k ± 0%  -63.06% (p=0.000 n=10)   19.60k ± 0%  -70.58% (p=0.000 n=10)   18.83k ± 0%  -71.74% (p=0.000 n=10)   18.83k ± 0%  -71.74% (p=0.000 n=10)
GRPCServerBatching 10000 100    584.8k ± 0%   176.4k ± 0%  -69.84% (p=0.000 n=10)   129.2k ± 0%  -77.90% (p=0.000 n=10)   122.2k ± 0%  -79.11% (p=0.000 n=10)   121.2k ± 0%  -79.27% (p=0.000 n=10)
GRPCServerBatching 10000 240    662.3k ± 0%   242.1k ± 0%  -63.45% (p=0.000 n=10)   192.8k ± 0%  -70.89% (p=0.000 n=10)   185.3k ± 0%  -72.03% (p=0.000 n=10)   184.6k ± 0%  -72.13% (p=0.000 n=10)
GRPCServerBatching 100000 100   5.750M ± 0%   1.757M ± 0%  -69.45% (p=0.000 n=10)   1.288M ± 0%  -77.59% (p=0.000 n=10)   1.218M ± 0%  -78.83% (p=0.000 n=10)   1.210M ± 0%  -78.95% (p=0.000 n=10)
GRPCServerBatching 100000 240   6.617M ± 0%   2.414M ± 0%  -63.52% (p=0.000 n=10)   1.924M ± 0%  -70.92% (p=0.000 n=10)   1.849M ± 0%  -72.05% (p=0.000 n=10)   1.845M ± 0%  -72.12% (p=0.000 n=10)
geomean                         199.3k        67.87k       -65.95%                  52.29k       -73.77%                  50.44k       -74.69%                  50.32k       -74.75%