You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Encoder always routes through Append(b, v, flags, clrs, indentr) at json.go:400-402:
// Note: unlike the original segmentio encoder, indentation is// performed via the Append function.buf.data, err=Append(buf.data[:0], v, enc.flags, enc.clrs, enc.indentr)
Unlike upstream segmentio/encoding (which renders flat then calls json.Indent as a post-pass), jsoncolor interleaves the indenter into the value walk. Per-token paths also branch on clrs == nil for color emission. Both checks run even when both clrs and indentr are nil — the common "encode to bytes, no styling" case.
Measured impact
Benchmarked against the Sakila payment table (~16k mixed-type records) on an M1 Max with -benchtime=3s -benchmem. jcolorenc is the now-deleted in-tree fork that sq used to ship — useful as a reference because it was closer to upstream segmentio:
bench
ns/op
B/op
allocs/op
Stdj (encoding/json)
~24 ms
736 KB
80k
Segmentj (segmentio/encoding)
~9 ms
0.9 MB
96k
jcolorenc flat
11.8 ms
1.16 MB
112k
jcolorenc indent
14.0 ms
2.31 MB
144k
jsoncolor flat
8.0 ms
5.65 MB
144k
jsoncolor indent
8.6 ms
5.65 MB
144k
jsoncolor is the fastest contender by latency — but its flat encode allocates the same bytes as its indented encode (5.65 MB vs 5.65 MB), and ~5× what the in-tree fork did at the same flat workload. Allocation count is the same as indent (144k), which says the cost isn't extra allocations — it's bigger allocations triggered by the always-on indenter/color machinery.
Proposal
Add a fast path on Encoder.Encode (and ideally a parallel one on package-level Append) that, when enc.clrs == nil && enc.indentr == nil, dispatches to a stripped encode walk without the per-token color/indent checks. Effectively: when nothing is configured, behave like upstream segmentio.
Two implementation shapes worth considering:
Stripped fast-path encoder. A separate code path that drops the *Colors and *Indenter arguments entirely, used when both are nil. Cheap to add, highest expected gain on the dominant non-styled case.
Restore segmentio's two-stage flow for the indent case. Render flat, then call json.Indent as a post-pass. Trades a buffer copy for a much leaner inner loop; usually a net win for non-trivial payloads. Independent of Initial code #1.
#1 alone should close most of the gap to segmentio on the no-styling path without changing any public API.
Notes
This was surfaced while replacing sq's in-tree jcolorenc fork with jsoncolor in neilotoole/sq#725. For sq's call sites (metadata/inspect/config output) the latency win matters more than allocations, so the swap landed as-is. The flat-path fast path would benefit any caller using jsoncolor outside the colored-terminal use case.
The buffer pool (encoderBufferPool at json.go:454-456) is intact and correctly Get/Put-balanced, so pool churn isn't the cause.
Benchmarks are reproducible from sq #725's cli/output/jsonw/internal/benchmark_test.go (the BenchmarkSegmentj / BenchmarkJSONColor pair).
Observation
The
Encoderalways routes throughAppend(b, v, flags, clrs, indentr)atjson.go:400-402:Unlike upstream
segmentio/encoding(which renders flat then callsjson.Indentas a post-pass),jsoncolorinterleaves the indenter into the value walk. Per-token paths also branch onclrs == nilfor color emission. Both checks run even when bothclrsandindentrare nil — the common "encode to bytes, no styling" case.Measured impact
Benchmarked against the Sakila
paymenttable (~16k mixed-type records) on an M1 Max with-benchtime=3s -benchmem.jcolorencis the now-deleted in-tree fork that sq used to ship — useful as a reference because it was closer to upstream segmentio:Stdj(encoding/json)Segmentj(segmentio/encoding)jcolorencflatjcolorencindentjsoncolorflatjsoncolorindentjsoncoloris the fastest contender by latency — but its flat encode allocates the same bytes as its indented encode (5.65 MB vs 5.65 MB), and ~5× what the in-tree fork did at the same flat workload. Allocation count is the same as indent (144k), which says the cost isn't extra allocations — it's bigger allocations triggered by the always-on indenter/color machinery.Proposal
Add a fast path on
Encoder.Encode(and ideally a parallel one on package-levelAppend) that, whenenc.clrs == nil && enc.indentr == nil, dispatches to a stripped encode walk without the per-token color/indent checks. Effectively: when nothing is configured, behave like upstream segmentio.Two implementation shapes worth considering:
*Colorsand*Indenterarguments entirely, used when both are nil. Cheap to add, highest expected gain on the dominant non-styled case.json.Indentas a post-pass. Trades a buffer copy for a much leaner inner loop; usually a net win for non-trivial payloads. Independent of Initial code #1.#1 alone should close most of the gap to segmentio on the no-styling path without changing any public API.
Notes
jcolorencfork withjsoncolorin neilotoole/sq#725. For sq's call sites (metadata/inspect/config output) the latency win matters more than allocations, so the swap landed as-is. The flat-path fast path would benefit any caller usingjsoncoloroutside the colored-terminal use case.encoderBufferPoolatjson.go:454-456) is intact and correctlyGet/Put-balanced, so pool churn isn't the cause.cli/output/jsonw/internal/benchmark_test.go(theBenchmarkSegmentj/BenchmarkJSONColorpair).