tools/benchmark: add --output-format flag for JSON output by Somesh42 · Pull Request #21920 · etcd-io/etcd

Somesh42 · 2026-06-06T11:05:16Z

What this does

Adds a global --output-format flag (values: text | json) to the
benchmark tool. When json is selected, results are serialised as indented
JSON to stdout instead of the human-readable text summary.

Why

The benchmark tool currently outputs only human-readable text, which is hard
to consume programmatically or track across runs. JSON output enables
scripting, trend tracking, and cross-version comparisons without fragile
text parsing.

Changes

pkg/report/report.go — adds FormatJSON() plus JSONReport, JSONBucket,
and JSONError types. Purely additive; existing text formatting untouched.
tools/benchmark/cmd/output.go (new) — registers the --output-format flag
and a shared printReport() helper.
tools/benchmark/cmd/put.go — routes output through printReport().
pkg/report/json_test.go (new) — 8 unit tests covering JSON validity,
scalar fields, percentile keys, error inclusion/omission, determinism,
and empty stats.

Example

```
$ benchmark --output-format=json put --total=10000
{
"total_secs": 1.23,
"slowest_secs": 0.0228,
"fastest_secs": 0.00036,
"average_secs": 0.00298,
"requests_per_sec": 8103.4,
"percentiles": { "p50": 0.00088, "p99": 0.018, ... },
"histogram": [ ... ],
"errors": []
}
```

Backward compatibility

Default behaviour (--output-format=text) is byte-for-byte identical to before.

Notes

This is an initial step; the put subcommand is wired to printReport() as
the reference. Happy to extend the same one-line change to the remaining
subcommands (range, watch, txn-put, lease-keepalive, stm) in this PR or a
follow-up — whichever maintainers prefer.

AI assistance disclosure

I used an AI assistant (Bob) to help draft and review this change. I
reviewed all code, ran the tests and build locally, and take responsibility
for the contribution.

The benchmark tool currently outputs only human-readable text, making it difficult to consume results programmatically or track trends over time. Add a global --output-format flag (values: text, json). When json is selected, the report package's new FormatJSON function serialises all stats - throughput, latency histogram, percentiles, and error distribution - as indented JSON to stdout. Default behaviour is unchanged. Signed-off-by: Somesh kumar <someshkumargauda@gmail.com>

k8s-ci-robot · 2026-06-06T11:05:23Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Somesh42
Once this PR has been reviewed and has the lgtm label, please assign fuweid for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2026-06-06T11:05:26Z

Hi @Somesh42. Thanks for your PR.

I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

serathius · 2026-06-06T12:04:25Z

cc @kishen-v

kishen-v · 2026-06-08T05:25:42Z

@@ -0,0 +1,130 @@
+// Copyright 2015 The etcd Authors


Nit: across all the newly added files - tools/benchmark/cmd/output.go and pkg/report/json_test.go

Suggested change

// Copyright 2015 The etcd Authors

// Copyright 2026 The etcd Authors

Fixed in both tools/benchmark/cmd/output.go and pkg/report/json_test.go, updated to 2026

- Fix copyright year to 2026 in newly added files - Fix deadlock caused by r.Run()/r.Stats() being called after close(r.Results()); printReport now starts draining before wg.Wait() - Extend --output-format flag to all benchmark subcommands (lease, range, stm, txn_put, watch, watch_get) - Add optional prefix support to printReport for watch commands Tested locally with --total=100, both text and json output formats complete successfully with no deadlock. Signed-off-by: Somesh kumar <someshkumargauda@gmail.com>

kishen-v · 2026-07-22T14:43:52Z

+	bc := 10
+	buckets := make([]float64, bc+1)
+	counts := make([]int, bc+1)
+	bs := (s.Slowest - s.Fastest) / float64(bc)
+	for i := 0; i < bc; i++ {
+		buckets[i] = s.Fastest + bs*float64(i)
+	}
+	buckets[bc] = s.Slowest
+	var bi int
+	for i := 0; i < len(s.Lats); {
+		if s.Lats[i] <= buckets[bi] {
+			i++
+			counts[bi]++
+		} else if bi < len(buckets)-1 {
+			bi++
+		}
+	}


This section is mostly same as histogram(), apart from tracking max for the bar-chart normalisation. Can we have a helper function in that case?

@kishen-v extracted the shared bucketing logic into a new bucketLatencies() helper that both histogram() and FormatJSON() now call.
Verified with a clean test run:
`etcd % go clean -testcache && go test ./pkg/report/... -v
=== RUN TestFormatJSON_ValidJSON
--- PASS: TestFormatJSON_ValidJSON (0.00s)
=== RUN TestFormatJSON_ScalarFields
--- PASS: TestFormatJSON_ScalarFields (0.00s)
=== RUN TestFormatJSON_PercentileKeys
--- PASS: TestFormatJSON_PercentileKeys (0.00s)
=== RUN TestFormatJSON_ErrorsIncluded
--- PASS: TestFormatJSON_ErrorsIncluded (0.00s)
=== RUN TestFormatJSON_ErrorsOmittedWhenEmpty
--- PASS: TestFormatJSON_ErrorsOmittedWhenEmpty (0.00s)
=== RUN TestFormatJSON_Deterministic
--- PASS: TestFormatJSON_Deterministic (0.00s)
=== RUN TestFormatJSON_EmptyStats
--- PASS: TestFormatJSON_EmptyStats (0.00s)
=== RUN TestPercentiles
--- PASS: TestPercentiles (0.00s)
=== RUN TestReport
--- PASS: TestReport (0.00s)
=== RUN TestWeightedReport
--- PASS: TestWeightedReport (0.00s)
=== RUN TestGetTimeseries
--- PASS: TestGetTimeseries (0.00s)
PASS
ok go.etcd.io/etcd/pkg/v3/report 0.658s

kishen-v · 2026-07-22T15:47:34Z

+// printReport writes benchmark statistics to stdout in the format chosen by
+// --output-format. All sub-commands should call this function instead of
+// printing report.Run() output directly so that JSON support is automatic.
+func printReport(r report.Report, prefixes ...string) func() {


Looks like prefixes are not actually passed anywhere. Can you please take a look?

thanks. Confirmed all call sites use printReport(r) with no extra args, so prefixes was always "" in practice and removed the parameter entirely rather than wiring it up, since there's no current caller that needs it.

kubernetes-prow · 2026-07-22T18:00:03Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Somesh42
Once this PR has been reviewed and has the lgtm label, please assign fuweid for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…aram Signed-off-by: Somesh kumar <someshkumargauda@gmail.com>

kishen-v · 2026-07-23T06:25:36Z

 	}

-	rc := r.Run()
+


Nit: May need to check new line additions across tools/benchmark/cmd/watch_get.go, tools/benchmark/cmd/watch.go, ‎tools/benchmark/cmd/txn_put.go, ‎tools/benchmark/cmd/stm.go and tools/benchmark/cmd/range.go,tools/benchmark/cmd/lease.go and

go fmt against tools/benchmark/cmd/output.go

Suggested change

okay... so ran gofmt -w across all the flagged files. Fixed some stray blank-line and import-ordering issues in output.go, plus trailing blank lines in the six caller files. Pushed in c9225bf.

kishen-v · 2026-07-23T06:27:43Z

+// printReport writes benchmark statistics to stdout in the format chosen by
+// --output-format. All sub-commands should call this function instead of
+// printing report.Run() output directly so that JSON support is automatic.
+func printReport(r report.Report) func() {


Just a thought, if we'd want to print the report, we might as well redirect the contents to a file. The current implementation prints the logs+report together, unless we redirect the output to a file.

Good catch, and you're right i confirmed via grep that several subcommands (range.go, txn_mixed.go, watch_latency.go, etc.) print status/progress messages via fmt.Println/Printf to stdout, same stream as the report output. That means --output-format json > file would currently pick up interleaved logs.

This predates this PR (those prints exist independent of the --output-format work), so I'd suggest scoping the stdout/stderr cleanup as a separate follow-up PR rather than folding it into this one happy to file that as a tracked issue if that works. Let me know if you'd rather I address it here instead.

Signed-off-by: Somesh kumar <someshkumargauda@gmail.com>

k8s-ci-robot added area/performance area/tooling labels Jun 6, 2026

k8s-ci-robot added the needs-ok-to-test label Jun 6, 2026

k8s-ci-robot added the size/L label Jun 6, 2026

kishen-v reviewed Jun 8, 2026

View reviewed changes

Comment thread tools/benchmark/cmd/put.go

Somesh42 requested a review from kishen-v June 8, 2026 13:39

kishen-v reviewed Jul 22, 2026

View reviewed changes

review: extract shared bucketLatencies helper, drop unused prefixes p…

0b488ef

…aram Signed-off-by: Somesh kumar <someshkumargauda@gmail.com>

Somesh42 force-pushed the tools/benchmark-json-output branch from 99ca7ea to 0b488ef Compare July 22, 2026 18:15

kishen-v reviewed Jul 23, 2026

View reviewed changes

gofmt: fix formatting nits flagged in review

c9225bf

Signed-off-by: Somesh kumar <someshkumargauda@gmail.com>

	// Copyright 2015 The etcd Authors
	// Copyright 2026 The etcd Authors

Uh oh!

Conversation

Somesh42 commented Jun 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this does

Why

Changes

Example

Backward compatibility

Notes

AI assistance disclosure

Uh oh!

k8s-ci-robot commented Jun 6, 2026

Uh oh!

k8s-ci-robot commented Jun 6, 2026

Uh oh!

serathius commented Jun 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kubernetes-prow Bot commented Jul 22, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Somesh42 commented Jun 6, 2026 •

edited

Loading