Skip to content

Conversation

@tigrannajaryan
Copy link
Collaborator

@tigrannajaryan tigrannajaryan commented Sep 10, 2025

This adds zstd sizes to Profile and JSONL examples:

File --> Size in bytes         | JSON   | Protobuf       | STEF           | JSONZ  | ProtoZ | stefz
currencies_historical          |   4197 |   4393 (1.05x) |   1082 (0.26x) |    756 |   1105 |    782
macosx_releases                |   2449 |   2604 (1.06x) |    935 (0.38x) |    523 |    766 |    596
programming_languages_keywords |  10681 |  11912 (1.12x) |   8225 (0.77x) |   3746 |   5074 |   4053

File                 | pprof  | stef   | pprofz | stefz
deser_stef.prof      |  83343 |  83768 |  33326 |  29424 (1.005x/0.883x)
otelcol_otlp.prof    |  31864 |  38859 |  13054 |   9215 (1.220x/0.706x)
wsstreamasync.prof   |  92996 | 128295 |  29093 |  28150 (1.380x/0.968x)

Also makes zstd CPU times visible as a stack bar chart in benchmark.html, for example:
image

This adds zstd sizes to Profile and JSONL examples and makes zstd
CPU times visible as a stack bar chart in benchmark.html.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances benchmark outputs to make zstd compression size/time metrics more prominent across the codebase. It adds compressed size reporting to profile and JSONL examples and introduces visual improvements to the benchmark HTML output.

Key changes:

  • Added zstd compression metrics to test output displays
  • Enhanced benchmark HTML charts to show compression timing as stacked bar charts
  • Modified chart generation to support stacked bars for better visualization of compression overhead

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
examples/profile/pprof2stef_test.go Added zstd compression comparison between pprof and STEF formats
examples/jsonl/json2stef_test.go Added compressed size columns to JSON/Protobuf/STEF comparison output
docs/benchmarks.html Updated chart IDs and configuration to use stacked bar visualization
benchmarks/size_test.go Modified chart recording to use new stacked bar format
benchmarks/charts.go Enhanced chart system to support multi-series stacked bars
benchmarks/benchmarks_test.go Restructured benchmarks to separate compression timing and show as stacked bars

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@tigrannajaryan tigrannajaryan marked this pull request as ready for review September 10, 2025 22:20
@github-actions
Copy link

Benchmark Result

Benchmark diff with base branch
goos: linux
goarch: amd64
pkg: github.com/splunk/stef/benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                                                 │ bench-main.txt │             bench-new.txt             │
                                                 │     sec/op     │    sec/op     vs base                 │
SerializeNative/STEF/none-4                           9.572m ± 4%
SerializeNative/STEFU/none-4                          34.58m ± 4%
DeserializeNative/STEF/none-4                         2.716m ± 1%
DeserializeNative/STEFU/none-4                        8.598m ± 1%
SerializeFromPdata/STEF/none-4                        218.4m ± 3%
SerializeFromPdata/STEFU/none-4                       34.75m ± 2%
DeserializeToPdata/STEF/none-4                        42.29m ± 2%
DeserializeToPdata/STEFU/none-4                       61.68m ± 2%
STEFReaderRead-4                                      2.716m ± 2%   2.648m ±  1%   -2.50% (p=0.002 n=6)
STEFSerializeMultipart/astronomy-otelmetrics-4         3.862 ± 9%    3.788 ±  7%        ~ (p=0.240 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4      74.68m ± 1%   76.86m ± 13%   +2.93% (p=0.002 n=6)
ReadSTEF-4                                            2.740m ± 1%   2.544m ±  1%   -7.15% (p=0.002 n=6)
ReadSTEFZ-4                                           4.376m ± 2%   3.498m ±  2%  -20.05% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                  8.601m ± 1%   8.212m ±  1%   -4.52% (p=0.002 n=6)
SerializeNative/STEF/serialize-4                                    13.68m ±  3%
SerializeNative/STEF/zstd-4                                         1.628m ±  4%
SerializeNative/STEFU/serialize-4                                   35.48m ±  0%
SerializeNative/STEFU/zstd-4                                        6.188m ±  2%
DeserializeNative/STEF/deser-4                                      2.631m ±  1%
DeserializeNative/STEF/zstd-4                                       251.2µ ±  1%
DeserializeNative/STEFU/deser-4                                     8.340m ±  1%
DeserializeNative/STEFU/zstd-4                                      898.3µ ±  0%
SerializeFromPdata/STEF/serialize-4                                 205.1m ±  2%
SerializeFromPdata/STEF/zstd-4                                      1.592m ±  4%
SerializeFromPdata/STEFU/serialize-4                                36.12m ±  1%
SerializeFromPdata/STEFU/zstd-4                                     5.926m ±  4%
DeserializeToPdata/STEF/deserialize-4                               37.05m ±  2%
DeserializeToPdata/STEF/zstd-4                                      250.8µ ±  2%
DeserializeToPdata/STEFU/deserialize-4                              55.51m ±  4%
DeserializeToPdata/STEFU/zstd-4                                     898.1µ ±  1%
geomean                                               23.10m        7.815m         -5.82%               ¹
¹ benchmark set differs from baseline; geomeans may not be comparable

                                                 │ bench-main.txt │             bench-new.txt             │
                                                 │   sec/point    │  sec/point    vs base                 │
SerializeNative/STEF/none-4                           143.2n ± 4%
SerializeNative/STEFU/none-4                          517.2n ± 4%
DeserializeNative/STEF/none-4                         40.62n ± 1%
DeserializeNative/STEFU/none-4                        128.6n ± 1%
SerializeFromPdata/STEF/none-4                        3.266µ ± 3%
SerializeFromPdata/STEFU/none-4                       519.8n ± 2%
DeserializeToPdata/STEF/none-4                        632.5n ± 2%
DeserializeToPdata/STEFU/none-4                       922.7n ± 2%
STEFReaderRead-4                                      40.62n ± 2%   39.61n ±  1%   -2.49% (p=0.002 n=6)
STEFSerializeMultipart/astronomy-otelmetrics-4        4.909µ ± 9%   4.815µ ±  7%        ~ (p=0.240 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4      94.91n ± 1%   97.69n ± 13%   +2.92% (p=0.002 n=6)
ReadSTEF-4                                            41.01n ± 1%   38.08n ±  1%   -7.16% (p=0.002 n=6)
ReadSTEFZ-4                                           65.49n ± 2%   52.35n ±  2%  -20.06% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                  128.8n ± 1%   122.9n ±  1%   -4.54% (p=0.002 n=6)
SerializeNative/STEF/serialize-4                                    204.6n ±  3%
SerializeNative/STEF/zstd-4                                         24.36n ±  4%
SerializeNative/STEFU/serialize-4                                   530.6n ±  0%
SerializeNative/STEFU/zstd-4                                        92.61n ±  2%
DeserializeNative/STEF/deser-4                                      39.34n ±  1%
DeserializeNative/STEF/zstd-4                                       3.759n ±  1%
DeserializeNative/STEFU/deser-4                                     124.8n ±  1%
DeserializeNative/STEFU/zstd-4                                      13.44n ±  0%
SerializeFromPdata/STEF/serialize-4                                 3.067µ ±  2%
SerializeFromPdata/STEF/zstd-4                                      23.83n ±  4%
SerializeFromPdata/STEFU/serialize-4                                540.2n ±  1%
SerializeFromPdata/STEFU/zstd-4                                     88.69n ±  4%
DeserializeToPdata/STEF/deserialize-4                               554.1n ±  2%
DeserializeToPdata/STEF/zstd-4                                      3.753n ±  2%
DeserializeToPdata/STEFU/deserialize-4                              830.1n ±  4%
DeserializeToPdata/STEFU/zstd-4                                     13.44n ±  1%
geomean                                               243.0n        93.45n         -5.83%               ¹
¹ benchmark set differs from baseline; geomeans may not be comparable

                                                 │ bench-main.txt │            bench-new.txt             │
                                                 │      B/op      │     B/op      vs base                │
SerializeNative/STEF/none-4                          3.561Mi ± 0%
SerializeNative/STEFU/none-4                         7.128Mi ± 0%
DeserializeNative/STEF/none-4                        911.5Ki ± 0%
DeserializeNative/STEFU/none-4                       1.564Mi ± 0%
SerializeFromPdata/STEF/none-4                       166.0Mi ± 0%
SerializeFromPdata/STEFU/none-4                      7.128Mi ± 0%
DeserializeToPdata/STEF/none-4                       29.88Mi ± 0%
DeserializeToPdata/STEFU/none-4                      36.63Mi ± 0%
STEFReaderRead-4                                     911.5Ki ± 0%   911.5Ki ± 0%       ~ (p=1.000 n=6) ¹
STEFSerializeMultipart/astronomy-otelmetrics-4       3.958Gi ± 0%   3.960Gi ± 0%       ~ (p=0.589 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4     20.73Mi ± 0%   20.73Mi ± 0%       ~ (p=0.422 n=6)
ReadSTEF-4                                           911.0Ki ± 0%   911.0Ki ± 0%       ~ (p=0.455 n=6)
ReadSTEFZ-4                                          10.19Mi ± 0%   10.19Mi ± 0%  -0.01% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                 13.61Mi ± 0%   13.61Mi ± 0%  -0.01% (p=0.002 n=6)
SerializeNative/STEF/serialize-4                                    3.569Mi ± 0%
SerializeNative/STEF/zstd-4                                         424.0Ki ± 0%
SerializeNative/STEFU/serialize-4                                   7.129Mi ± 0%
SerializeNative/STEFU/zstd-4                                        920.0Ki ± 0%
DeserializeNative/STEF/deser-4                                      911.5Ki ± 0%
DeserializeNative/STEF/zstd-4                                       424.0Ki ± 0%
DeserializeNative/STEFU/deser-4                                     1.564Mi ± 0%
DeserializeNative/STEFU/zstd-4                                      920.1Ki ± 0%
SerializeFromPdata/STEF/serialize-4                                 166.0Mi ± 0%
SerializeFromPdata/STEF/zstd-4                                      424.0Ki ± 0%
SerializeFromPdata/STEFU/serialize-4                                7.129Mi ± 0%
SerializeFromPdata/STEFU/zstd-4                                     920.0Ki ± 0%
DeserializeToPdata/STEF/deserialize-4                               29.88Mi ± 0%
DeserializeToPdata/STEF/zstd-4                                      424.0Ki ± 0%
DeserializeToPdata/STEFU/deserialize-4                              36.63Mi ± 0%
DeserializeToPdata/STEFU/zstd-4                                     920.1Ki ± 0%
geomean                                              11.08Mi        3.862Mi       +0.01%               ²
¹ all samples are equal
² benchmark set differs from baseline; geomeans may not be comparable

                                                 │ bench-main.txt │            bench-new.txt            │
                                                 │   allocs/op    │  allocs/op   vs base                │
SerializeNative/STEF/none-4                           2.901k ± 0%
SerializeNative/STEFU/none-4                          1.079k ± 0%
DeserializeNative/STEF/none-4                          669.0 ± 0%
DeserializeNative/STEFU/none-4                         729.0 ± 0%
SerializeFromPdata/STEF/none-4                        256.4k ± 0%
SerializeFromPdata/STEFU/none-4                       1.081k ± 0%
DeserializeToPdata/STEF/none-4                        622.8k ± 0%
DeserializeToPdata/STEFU/none-4                       811.5k ± 0%
STEFReaderRead-4                                       669.0 ± 0%    669.0 ± 0%       ~ (p=1.000 n=6) ¹
STEFSerializeMultipart/astronomy-otelmetrics-4        14.44M ± 0%   14.44M ± 0%       ~ (p=0.818 n=6)
STEFDeserializeMultipart/astronomy-otelmetrics-4      3.920k ± 0%   3.920k ± 0%       ~ (p=1.000 n=6) ¹
ReadSTEF-4                                             669.0 ± 0%    669.0 ± 0%       ~ (p=1.000 n=6) ¹
ReadSTEFZ-4                                            702.0 ± 0%    698.0 ± 0%  -0.57% (p=0.002 n=6)
ReadSTEFZWriteSTEF-4                                  1.681k ± 0%   1.675k ± 0%  -0.36% (p=0.002 n=6)
SerializeNative/STEF/serialize-4                                    2.916k ± 0%
SerializeNative/STEF/zstd-4                                          1.000 ± 0%
SerializeNative/STEFU/serialize-4                                   1.080k ± 0%
SerializeNative/STEFU/zstd-4                                         1.000 ± 0%
DeserializeNative/STEF/deser-4                                       669.0 ± 0%
DeserializeNative/STEF/zstd-4                                        1.000 ± 0%
DeserializeNative/STEFU/deser-4                                      729.0 ± 0%
DeserializeNative/STEFU/zstd-4                                       1.000 ± 0%
SerializeFromPdata/STEF/serialize-4                                 256.4k ± 0%
SerializeFromPdata/STEF/zstd-4                                       1.000 ± 0%
SerializeFromPdata/STEFU/serialize-4                                1.081k ± 0%
SerializeFromPdata/STEFU/zstd-4                                      1.000 ± 0%
DeserializeToPdata/STEF/deserialize-4                               622.8k ± 0%
DeserializeToPdata/STEF/zstd-4                                       1.000 ± 0%
DeserializeToPdata/STEFU/deserialize-4                              811.5k ± 0%
DeserializeToPdata/STEFU/zstd-4                                      1.000 ± 0%
geomean                                               8.217k         309.8       -0.15%               ²
¹ all samples are equal
² benchmark set differs from baseline; geomeans may not be comparable
Benchmark result
benchstat bench-new.txt
goos: linux
goarch: amd64
pkg: github.com/splunk/stef/benchmarks
cpu: AMD EPYC 7763 64-Core Processor                
                                                 │ bench-new.txt │
                                                 │    sec/op     │
SerializeNative/STEF/serialize-4                    13.68m ±  3%
SerializeNative/STEF/zstd-4                         1.628m ±  4%
SerializeNative/STEFU/serialize-4                   35.48m ±  0%
SerializeNative/STEFU/zstd-4                        6.188m ±  2%
DeserializeNative/STEF/deser-4                      2.631m ±  1%
DeserializeNative/STEF/zstd-4                       251.2µ ±  1%
DeserializeNative/STEFU/deser-4                     8.340m ±  1%
DeserializeNative/STEFU/zstd-4                      898.3µ ±  0%
SerializeFromPdata/STEF/serialize-4                 205.1m ±  2%
SerializeFromPdata/STEF/zstd-4                      1.592m ±  4%
SerializeFromPdata/STEFU/serialize-4                36.12m ±  1%
SerializeFromPdata/STEFU/zstd-4                     5.926m ±  4%
DeserializeToPdata/STEF/deserialize-4               37.05m ±  2%
DeserializeToPdata/STEF/zstd-4                      250.8µ ±  2%
DeserializeToPdata/STEFU/deserialize-4              55.51m ±  4%
DeserializeToPdata/STEFU/zstd-4                     898.1µ ±  1%
STEFReaderRead-4                                    2.648m ±  1%
STEFSerializeMultipart/astronomy-otelmetrics-4       3.788 ±  7%
STEFDeserializeMultipart/astronomy-otelmetrics-4    76.86m ± 13%
ReadSTEF-4                                          2.544m ±  1%
ReadSTEFZ-4                                         3.498m ±  2%
ReadSTEFZWriteSTEF-4                                8.212m ±  1%
geomean                                             7.815m

                                                 │ bench-new.txt │
                                                 │   sec/point   │
SerializeNative/STEF/serialize-4                    204.6n ±  3%
SerializeNative/STEF/zstd-4                         24.36n ±  4%
SerializeNative/STEFU/serialize-4                   530.6n ±  0%
SerializeNative/STEFU/zstd-4                        92.61n ±  2%
DeserializeNative/STEF/deser-4                      39.34n ±  1%
DeserializeNative/STEF/zstd-4                       3.759n ±  1%
DeserializeNative/STEFU/deser-4                     124.8n ±  1%
DeserializeNative/STEFU/zstd-4                      13.44n ±  0%
SerializeFromPdata/STEF/serialize-4                 3.067µ ±  2%
SerializeFromPdata/STEF/zstd-4                      23.83n ±  4%
SerializeFromPdata/STEFU/serialize-4                540.2n ±  1%
SerializeFromPdata/STEFU/zstd-4                     88.69n ±  4%
DeserializeToPdata/STEF/deserialize-4               554.1n ±  2%
DeserializeToPdata/STEF/zstd-4                      3.753n ±  2%
DeserializeToPdata/STEFU/deserialize-4              830.1n ±  4%
DeserializeToPdata/STEFU/zstd-4                     13.44n ±  1%
STEFReaderRead-4                                    39.61n ±  1%
STEFSerializeMultipart/astronomy-otelmetrics-4      4.815µ ±  7%
STEFDeserializeMultipart/astronomy-otelmetrics-4    97.69n ± 13%
ReadSTEF-4                                          38.08n ±  1%
ReadSTEFZ-4                                         52.35n ±  2%
ReadSTEFZWriteSTEF-4                                122.9n ±  1%
geomean                                             93.45n

                                                 │ bench-new.txt │
                                                 │     B/op      │
SerializeNative/STEF/serialize-4                    3.569Mi ± 0%
SerializeNative/STEF/zstd-4                         424.0Ki ± 0%
SerializeNative/STEFU/serialize-4                   7.129Mi ± 0%
SerializeNative/STEFU/zstd-4                        920.0Ki ± 0%
DeserializeNative/STEF/deser-4                      911.5Ki ± 0%
DeserializeNative/STEF/zstd-4                       424.0Ki ± 0%
DeserializeNative/STEFU/deser-4                     1.564Mi ± 0%
DeserializeNative/STEFU/zstd-4                      920.1Ki ± 0%
SerializeFromPdata/STEF/serialize-4                 166.0Mi ± 0%
SerializeFromPdata/STEF/zstd-4                      424.0Ki ± 0%
SerializeFromPdata/STEFU/serialize-4                7.129Mi ± 0%
SerializeFromPdata/STEFU/zstd-4                     920.0Ki ± 0%
DeserializeToPdata/STEF/deserialize-4               29.88Mi ± 0%
DeserializeToPdata/STEF/zstd-4                      424.0Ki ± 0%
DeserializeToPdata/STEFU/deserialize-4              36.63Mi ± 0%
DeserializeToPdata/STEFU/zstd-4                     920.1Ki ± 0%
STEFReaderRead-4                                    911.5Ki ± 0%
STEFSerializeMultipart/astronomy-otelmetrics-4      3.960Gi ± 0%
STEFDeserializeMultipart/astronomy-otelmetrics-4    20.73Mi ± 0%
ReadSTEF-4                                          911.0Ki ± 0%
ReadSTEFZ-4                                         10.19Mi ± 0%
ReadSTEFZWriteSTEF-4                                13.61Mi ± 0%
geomean                                             3.862Mi

                                                 │ bench-new.txt │
                                                 │   allocs/op   │
SerializeNative/STEF/serialize-4                     2.916k ± 0%
SerializeNative/STEF/zstd-4                           1.000 ± 0%
SerializeNative/STEFU/serialize-4                    1.080k ± 0%
SerializeNative/STEFU/zstd-4                          1.000 ± 0%
DeserializeNative/STEF/deser-4                        669.0 ± 0%
DeserializeNative/STEF/zstd-4                         1.000 ± 0%
DeserializeNative/STEFU/deser-4                       729.0 ± 0%
DeserializeNative/STEFU/zstd-4                        1.000 ± 0%
SerializeFromPdata/STEF/serialize-4                  256.4k ± 0%
SerializeFromPdata/STEF/zstd-4                        1.000 ± 0%
SerializeFromPdata/STEFU/serialize-4                 1.081k ± 0%
SerializeFromPdata/STEFU/zstd-4                       1.000 ± 0%
DeserializeToPdata/STEF/deserialize-4                622.8k ± 0%
DeserializeToPdata/STEF/zstd-4                        1.000 ± 0%
DeserializeToPdata/STEFU/deserialize-4               811.5k ± 0%
DeserializeToPdata/STEFU/zstd-4                       1.000 ± 0%
STEFReaderRead-4                                      669.0 ± 0%
STEFSerializeMultipart/astronomy-otelmetrics-4       14.44M ± 0%
STEFDeserializeMultipart/astronomy-otelmetrics-4     3.920k ± 0%
ReadSTEF-4                                            669.0 ± 0%
ReadSTEFZ-4                                           698.0 ± 0%
ReadSTEFZWriteSTEF-4                                 1.675k ± 0%
geomean                                               309.8

@tigrannajaryan tigrannajaryan merged commit b71f287 into main Sep 16, 2025
10 checks passed
@tigrannajaryan tigrannajaryan deleted the tigran/add-zstd-sizes-to-tests branch September 16, 2025 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants