perf(debug-trace-server): serve cache hits as raw JSON bytes - #167
Conversation
Responses were serialized three times on the fill path (once for the response-size metric, once into the cache, once by jsonrpsee) and a cache hit paid a full tree re-parse plus re-serialization of multi-MB JSON. Handlers now return RawJson (Arc<RawValue>): the trace output is serialized exactly once via to_raw_value, the response-size metric reads the byte length, the cache shares the same bytes by Arc, and jsonrpsee splices them verbatim into the reply — a hit is an Arc clone with zero parsing, zero copies. CachedResponse is deleted (the cache stores RawJson directly); insert_cache loses its serialization timing (nothing left to time). trace_transaction / trace_parity_transaction also drop their double serialization. Pinned by raw_json_serializes_verbatim, shares_bytes_with assertions on the hit paths, and an end-to-end raw_json_passes_through_jsonrpsee_verbatim through a real RpcModule. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude review status
🛠️ Review did not finish Attempted head This round did not publish: MODEL_ACTION_FAILED in phase review_retry. Anything listed below is from the last round that did. Re-run the workflow or push a new commit to try again. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 084702ee16
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…e explicitly Address the Codex review note on the Serialize impl: spell out the deref to the RawValue instead of leaning on method-resolution auto-deref, which also keeps the impl independent of whether serde's optional `rc` feature gets unified into the graph. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…module /simplify pass over the PR: - move RawJson to its own leaf module — the uncached tx handlers no longer import their reply type from the response-cache module - store Arc<Box<RawValue>>: adopting the serializer's allocation removes the full-body memcpy that constructing an Arc<RawValue> (unsized pointee) paid on every fill, making the "nothing is copied" doc claims actually true - RawJson::try_new replaces the to_raw_value+into incantation at all three fill sites and the test-only from_value twin; From<Box<RawValue>> deleted - null() clones a static built from RawValue::NULL instead of allocating and validation-parsing "null" per call - serialize_reply unifies the two tx handlers' byte-identical serialize + response-size-metric epilogues Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 participant
Summary
Handlers now return
RawJson(Arc<RawValue>,response_cache.rs:299) instead ofserde_json::Value: the trace output is serialized exactly once (to_raw_value,rpc_service.rs:395), the response-size metric reads the byte length, the response cache shares the same bytes byArc, and jsonrpsee splices them verbatim into the reply. A cache hit becomes anArcclone — zero parsing, zero copies, zero re-serialization.Root cause
The cache stored pre-serialized strings but its interface to jsonrpsee was
serde_json::Value, so both sides of the cache paid full tree work: a hit re-parsed the stored string into aValueand jsonrpsee re-serialized it (measured ≈7 ms CPU per MB of response — a ~20 MB block-level callTracer hit burns ~140 ms server CPU, see the baseline below; 93.7% of production traffic is repeat-after-success, i.e. exactly this path); the fill path serialized the same response three times (value.to_string().len()for the size metric,CachedResponse::new, jsonrpsee).Measured baseline (v2.0.15 without this PR, 2026-08-01)
Bench on the tko staging deploy (response cache enabled), 51 load-test-era blocks 6907000–6907050 — 36.7k–44.1k tx each (median 41,779), callTracer responses avg 20.0 MB, 1020 MB per round:
x-execution-time-ns)The warm-round ~140 ms CPU per hit is pure
Valuere-parse + re-serialization of the stored 20 MB string — the work this PR deletes; withRawJsonthe hit path should drop to transfer cost only. Raw data:validator-data/debug_trace_server/bench/new/monster51_files_tko2.tar.gz, analysis inbench/old-vs-new-report-2026-08-01.md§7.Fix
RawJsoninresponse_cache.rs:Serializesplices the stored bytes verbatim (:338); the cache stores it directly as its value type (get/insertat:483/:512), deletingCachedResponse.compute_block_traceproducesRawJsonviaserde_json::value::to_raw_value— one serialization, no validation re-scan of our own output; the size metric readsbyte_len()(rpc_service.rs:395).debug_traceTransaction,trace_transaction) drop theirto_value+to_stringdouble serialization; parity not-found returnsRawJson::null().insert_cacheloses its slow-insert timing — the insert is now anArcclone with nothing left to time.debug_getCacheStatuskeepsserde_json::Value(tiny payload, not cached).Testing
raw_json_serializes_verbatim(response_cache.rs:985) pins the splice property the design rests on.raw_json_passes_through_jsonrpsee_verbatim(rpc_service.rs:914) proves it end-to-end through a realRpcModulereply.shares_bytes_withassertions pin that hits serve the inserted bytes without copying (rpc_service.rs:905, cache insert/get test).Notes
ESTIMATED_ITEMS=0). Verify post-deploy viadebug_trace_cpu_time/ hit p50 against the baseline table above.serde_jsongains theraw_valuefeature in this bin only.🤖 Generated with Claude Code