perf(mcp-proxy): reduce large tool response allocations - #2856
Conversation
Why this change was needed: The raw payload refactor needs behavior protection and measurable baseline data around the tools/call entry point before changing response materialization. What changed: - Added handler-level characterization tests for JSON envelope mode - Added raw response mode coverage - Added non-JSON response coverage - Added a large-response benchmark for envelope and raw-response modes Problem solved: Future refactor steps can prove they preserve the MCP response contract while reducing allocations. Co-authored-by: Cursor <cursoragent@cursor.com>
Why this change was needed: The tools/call response path needs a single owner for upstream response metadata and raw body bytes before replacing generic JSON decoding. What changed: - Added toolResponsePayload for response metadata, raw body, JSON detection, and log preview - Added envelope and raw-response serialization helpers - Covered JSON, non-JSON, and invalid JSON behavior Problem solved: Response handling can move away from map[string]any without changing the public MCP response shape. Co-authored-by: Cursor <cursoragent@cursor.com>
Why this change was needed: Large tools/call responses were decoded into generic Go objects before being serialized back to JSON, multiplying memory usage. What changed: - Read upstream response bodies as raw bytes - Created tool results from serialized JSON bytes - Preserved envelope and raw-response behavior with characterization tests - Removed the now-obsolete production envelope map helper Problem solved: The main success path no longer needs to materialize upstream JSON as map[string]any. Co-authored-by: Cursor <cursoragent@cursor.com>
Why this change was needed: Audit logging was rebuilding and reparsing large tool responses only to record a preview and upstream request id. What changed: - Reused toolResponsePayload preview for audit response logs - Reused raw body length for audit response size - Reused upstream response header for upstream_request_id Problem solved: The audit path no longer performs full response marshal/unmarshal after a successful tools/call. Co-authored-by: Cursor <cursoragent@cursor.com>
Why this change was needed: Access logging and metrics were serializing the full MCP tool result to compute previews and sizes. What changed: - Threaded toolResponsePayload into tool-call logging and metrics - Used payload preview for response logs - Used raw body length for response size metrics - Kept the old marshal path as a fallback when payload metadata is unavailable Problem solved: Large successful tool responses avoid another round of full result serialization in observability code. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Newest benchmark comparison (
|
Why this change was needed: The raw payload refactor made the old response-envelope constants and test helper production-dead, leaving confusing code that looked like part of the runtime path. What changed: - Removed obsolete response field constants from proxy.go - Inlined the fallback request_id key in upstream request ID extraction - Replaced test-only envelope helper usage with explicit fixtures - Dropped tests that only validated the removed test helper Problem solved: The proxy package no longer keeps dead response-envelope scaffolding after migrating response construction to toolResponsePayload. Co-authored-by: Cursor <cursoragent@cursor.com>
This comment was marked as outdated.
This comment was marked as outdated.
Why this change was needed: The preceding perf refactor (toolResponsePayload) eagerly materialized a single truncated preview using only AuditLogMaxResponseSize, which had two side effects: APILogResponseSize/APILogErrorResponseSize knobs were silently bypassed for tools/call, and the audit log response field for envelope mode no longer contained the envelope wrapper. The error branch was worse — for non-2xx upstream responses it overwrote auditResponse with a bare error string, dropping all upstream body context. For non-JSON error bodies (e.g. HTML 500 pages) this meant troubleshooting evidence was effectively lost. What changed: - config: added AuditLogMaxErrorResponseSize knob (default 16KB) so failed calls get a larger preview budget than successful ones; existing audit and API log size knobs keep their independent meaning. - toolResponsePayload: dropped the eagerly-cached truncatedPreview field. Added IsSuccess, PickLimit, and an on-demand EnvelopePreview that always emits valid JSON, embedding the body either raw (when it is JSON and fits the limit) or as a JSON-encoded string (for non-JSON content, truncated JSON, malformed JSON, or even binary with invalid UTF-8 substituted as U+FFFD). This guarantees non-JSON error bodies are preserved verbatim in audit logs instead of being lost as null. - proxy.go: audit and API log call sites now build the envelope preview on demand, picking the per-status limit. Submit-error branch reuses the populated payload to surface the upstream envelope on non-2xx instead of overwriting with the error message; transport errors still fall back to the error message because no upstream response exists. - Removed dead fallback paths that the previous refactor left behind: the unreachable else branch after Submit success, buildToolResult plus its four envelope-shape tests, extractUpstreamRequestID, getMapKeys, and the associated noisy debug logs. The remaining payload-nil branch in serializeToolCallResponse is genuinely only reachable on transport errors or panic-before-reader and now emits a warning if it ever fires. - tests: added EnvelopePreview suite covering empty/raw-JSON/truncated/ non-JSON HTML/plain-text/malformed-JSON bodies; rewrote serializeToolCallResponse tests for the new signature with 2xx vs 5xx budget selection, non-JSON envelope preservation, and panic-path fallback; added two non-2xx genToolHandler integration specs covering JSON and HTML error envelopes; added BenchmarkEnvelopePreview to confirm preview cost stays O(min(body, limit)) at 3-6 allocs/op. Problem solved: Operators get back the per-call-site log truncation knobs they configured, non-2xx audit logs retain full upstream envelope detail for troubleshooting (including HTML error pages and other non-JSON bodies), and the dead-code removal goal of the preceding refactor commit is now complete. Co-authored-by: Cursor <cursoragent@cursor.com>
Why this change was needed: The raw payload refactor accidentally treated non-empty invalid bodies declared as application/json as successful string responses, which hid upstream protocol errors from MCP callers. What changed: - Track whether the upstream response explicitly declared a JSON content type separately from whether the body is valid JSON - Return an error when envelope or raw-response serialization sees invalid non-empty declared JSON - Add regression coverage at both payload-helper and tool-handler levels while preserving explicit non-JSON string wrapping Problem solved: MCP tool calls again fail when an upstream declares JSON but returns malformed JSON, matching the pre-refactor contract instead of silently converting the body to a string. Co-authored-by: Cursor <cursoragent@cursor.com>
major
位置:proxy.go:1038-1083,response_payload.go:43-49,response_payload.go:69-80,response_payload.go:100-107
位置:response_payload.go:101-108(marshalRawResponse)
位置:response_payload.go:60 Minor
位置:proxy.go:1127-1130
位置:response_payload.go(新增函数)vs util/mask.go
位置:response_payload.go:71-79(responseBodyRawMessage) |
Why this change was needed: Raw response mode should preserve upstream bodies for MCP clients even when an upstream service returns text or another non-JSON payload. What changed: - Treat declared JSON as raw JSON only after validation at marshal time - Encode non-JSON raw response bodies as JSON strings for MCP tool results - Cover raw text/plain success and error responses in proxy tests - Keep the tool response reader type contract explicit without violating lint Problem solved: MCP raw response mode now returns valid JSON content for non-JSON upstream bodies while still rejecting invalid bodies declared as JSON. Co-authored-by: Cursor <cursoragent@cursor.com>
Why this change was needed: A raw-payload refactor changed empty text/plain tool responses in the envelope from an empty string to null, which could break clients that distinguish empty text bodies from absent JSON bodies. What changed: - Preserve explicitly read empty non-JSON response bodies as JSON empty strings in client envelopes and log previews - Add a regression test for text/plain empty body envelope output Problem solved: MCP tool envelopes now retain the pre-refactor empty text body semantics while keeping absent or JSON empty bodies as null. Co-authored-by: Cursor <cursoragent@cursor.com>
|
review again |
Keep audit status aligned with the tool-call outcome when the go-openapi submit result violates the expected json.RawMessage reader contract. Also remove the unused status-based payload limit helpers and document the larger audit error response budget in the config template. Co-authored-by: Cursor <cursoragent@cursor.com>
PR #2856 Code Review Report
变更概述本 PR 解决 mcp-proxy 处理大体积
问题列表🔴 Blocking无。 🟠 Major1. 空 JSON body 行为变更:现返回
2.
🟡 Minor
💬 Nit
优点
总结建议整体是一次质量较高的性能重构。建议在解决 Major 问题 #1 和 #2 后合并:
[from openclaw-internal] |
This comment was marked as outdated.
This comment was marked as outdated.
|
@robot-wklken 你也 review 下这个 PR,然后把报告提交上来 |
This comment was marked as outdated.
This comment was marked as outdated.
PR ReviewReview Scope
变更概览
历史评论核对
发现的问题Suggestion
Suggestion
优点
测试与文档建议
结论
[from-codex local repo] |
|
review again |
…y_resolve_oom # Conflicts: # src/mcp-proxy/pkg/infra/proxy/proxy_test.go
PR #2856 Code Review 汇总报告
变更概述本 PR 的核心目标是解决 mcp-proxy 在处理大 tool response 时的 OOM 问题。主要手段是引入 主要变更:
ReportCritical Issues无 High IssuesH1: 在 responseBytes, ok := submit.(json.RawMessage)
if !ok {
return handleUnexpectedSubmitResult(...), nil
}
不过,这个场景本身极难触发( H2: if p.isDeclaredJSON {
return nil, p.invalidDeclaredJSONBodyError(err)
}当 Medium IssuesM1: diff 中 M2:
结论:当前可以接受,但如果后续有更多调用点选择日志限制,再提取不迟。不属于必须修改的问题。 M3: 新增配置项 Low IssuesL1: 对于二进制响应体(如 protobuf、gzip), L2: benchmark 测试中
L3: 该函数在 Dismissed FindingsDF1: 初看会担心删除 DF2: 移除
Agent Agreement Summary
合并建议merge after fixes 本 PR 设计思路清晰,核心目标(避免大 JSON 响应的双倍内存分配)明确,实现路径合理。 主要需要修复的问题:
L1/L2/L3 可在后续 PR 中优化,不阻塞当前合并。 新增测试覆盖评价
由 OpenClaw K 直接撰写 | [from openclaw-internal] @wklken 您的 PR 已完成 review,请查看以上意见。 |
|
对原 review report 中 review points 的逐项复核如下,复核对象为当前 PR head
验证命令:
结论:阻塞项 H1/H2/M1 不成立;本 PR 不需要因为这些点继续改代码。 |
Why this change was needed: Review of the MCP proxy raw-response refactor found several audit and JSON edge cases where the new payload path could diverge from the previous behavior or log misleading success details. What changed: - Treat structured +json media types as JSON while avoiding substring matches - Reject empty declared JSON bodies on content responses and validate raw/envelope paths consistently - Keep audit failure state for early decode errors and rethrow panics after audit logging - Re-render unexpected submit audit previews with the error response budget - Document byte-oriented log preview limits in AGENTS.md Problem solved: MCP tool calls now keep client-visible JSON handling and audit logs aligned across success, upstream error, and defensive failure paths.
Why this change was needed: The raw-response and envelope response paths had duplicated body-shaping logic for nil payloads, empty bodies, declared JSON, and non-JSON responses. What changed: - Extracted bodyAsJSONValue as the shared response-body conversion helper - Updated marshalEnvelope to embed the shared JSON value as response_body - Updated marshalRawResponse to return the same JSON value as the full MCP response Problem solved: Future changes to upstream response body semantics now happen in one place while preserving the existing raw_response_enabled and envelope output contracts.
…ing#2856) * test(mcp-proxy): lock large response tool handler behavior Why this change was needed: The raw payload refactor needs behavior protection and measurable baseline data around the tools/call entry point before changing response materialization. What changed: - Added handler-level characterization tests for JSON envelope mode - Added raw response mode coverage - Added non-JSON response coverage - Added a large-response benchmark for envelope and raw-response modes Problem solved: Future refactor steps can prove they preserve the MCP response contract while reducing allocations. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(mcp-proxy): add raw tool response payload helper Why this change was needed: The tools/call response path needs a single owner for upstream response metadata and raw body bytes before replacing generic JSON decoding. What changed: - Added toolResponsePayload for response metadata, raw body, JSON detection, and log preview - Added envelope and raw-response serialization helpers - Covered JSON, non-JSON, and invalid JSON behavior Problem solved: Response handling can move away from map[string]any without changing the public MCP response shape. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(mcp-proxy): route tool responses through raw payload bytes Why this change was needed: Large tools/call responses were decoded into generic Go objects before being serialized back to JSON, multiplying memory usage. What changed: - Read upstream response bodies as raw bytes - Created tool results from serialized JSON bytes - Preserved envelope and raw-response behavior with characterization tests - Removed the now-obsolete production envelope map helper Problem solved: The main success path no longer needs to materialize upstream JSON as map[string]any. Co-authored-by: Cursor <cursoragent@cursor.com> * perf(mcp-proxy): reuse raw payload metadata for audit logs Why this change was needed: Audit logging was rebuilding and reparsing large tool responses only to record a preview and upstream request id. What changed: - Reused toolResponsePayload preview for audit response logs - Reused raw body length for audit response size - Reused upstream response header for upstream_request_id Problem solved: The audit path no longer performs full response marshal/unmarshal after a successful tools/call. Co-authored-by: Cursor <cursoragent@cursor.com> * perf(mcp-proxy): reuse raw payload metadata for logs and metrics Why this change was needed: Access logging and metrics were serializing the full MCP tool result to compute previews and sizes. What changed: - Threaded toolResponsePayload into tool-call logging and metrics - Used payload preview for response logs - Used raw body length for response size metrics - Kept the old marshal path as a fallback when payload metadata is unavailable Problem solved: Large successful tool responses avoid another round of full result serialization in observability code. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(mcp-proxy): remove dead response envelope helpers Why this change was needed: The raw payload refactor made the old response-envelope constants and test helper production-dead, leaving confusing code that looked like part of the runtime path. What changed: - Removed obsolete response field constants from proxy.go - Inlined the fallback request_id key in upstream request ID extraction - Replaced test-only envelope helper usage with explicit fixtures - Dropped tests that only validated the removed test helper Problem solved: The proxy package no longer keeps dead response-envelope scaffolding after migrating response construction to toolResponsePayload. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp-proxy): preserve upstream envelope in tool call audit/API logs Why this change was needed: The preceding perf refactor (toolResponsePayload) eagerly materialized a single truncated preview using only AuditLogMaxResponseSize, which had two side effects: APILogResponseSize/APILogErrorResponseSize knobs were silently bypassed for tools/call, and the audit log response field for envelope mode no longer contained the envelope wrapper. The error branch was worse — for non-2xx upstream responses it overwrote auditResponse with a bare error string, dropping all upstream body context. For non-JSON error bodies (e.g. HTML 500 pages) this meant troubleshooting evidence was effectively lost. What changed: - config: added AuditLogMaxErrorResponseSize knob (default 16KB) so failed calls get a larger preview budget than successful ones; existing audit and API log size knobs keep their independent meaning. - toolResponsePayload: dropped the eagerly-cached truncatedPreview field. Added IsSuccess, PickLimit, and an on-demand EnvelopePreview that always emits valid JSON, embedding the body either raw (when it is JSON and fits the limit) or as a JSON-encoded string (for non-JSON content, truncated JSON, malformed JSON, or even binary with invalid UTF-8 substituted as U+FFFD). This guarantees non-JSON error bodies are preserved verbatim in audit logs instead of being lost as null. - proxy.go: audit and API log call sites now build the envelope preview on demand, picking the per-status limit. Submit-error branch reuses the populated payload to surface the upstream envelope on non-2xx instead of overwriting with the error message; transport errors still fall back to the error message because no upstream response exists. - Removed dead fallback paths that the previous refactor left behind: the unreachable else branch after Submit success, buildToolResult plus its four envelope-shape tests, extractUpstreamRequestID, getMapKeys, and the associated noisy debug logs. The remaining payload-nil branch in serializeToolCallResponse is genuinely only reachable on transport errors or panic-before-reader and now emits a warning if it ever fires. - tests: added EnvelopePreview suite covering empty/raw-JSON/truncated/ non-JSON HTML/plain-text/malformed-JSON bodies; rewrote serializeToolCallResponse tests for the new signature with 2xx vs 5xx budget selection, non-JSON envelope preservation, and panic-path fallback; added two non-2xx genToolHandler integration specs covering JSON and HTML error envelopes; added BenchmarkEnvelopePreview to confirm preview cost stays O(min(body, limit)) at 3-6 allocs/op. Problem solved: Operators get back the per-call-site log truncation knobs they configured, non-2xx audit logs retain full upstream envelope detail for troubleshooting (including HTML error pages and other non-JSON bodies), and the dead-code removal goal of the preceding refactor commit is now complete. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp-proxy): reject invalid declared JSON responses Why this change was needed: The raw payload refactor accidentally treated non-empty invalid bodies declared as application/json as successful string responses, which hid upstream protocol errors from MCP callers. What changed: - Track whether the upstream response explicitly declared a JSON content type separately from whether the body is valid JSON - Return an error when envelope or raw-response serialization sees invalid non-empty declared JSON - Add regression coverage at both payload-helper and tool-handler levels while preserving explicit non-JSON string wrapping Problem solved: MCP tool calls again fail when an upstream declares JSON but returns malformed JSON, matching the pre-refactor contract instead of silently converting the body to a string. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp-proxy): encode raw non-json tool responses Why this change was needed: Raw response mode should preserve upstream bodies for MCP clients even when an upstream service returns text or another non-JSON payload. What changed: - Treat declared JSON as raw JSON only after validation at marshal time - Encode non-JSON raw response bodies as JSON strings for MCP tool results - Cover raw text/plain success and error responses in proxy tests - Keep the tool response reader type contract explicit without violating lint Problem solved: MCP raw response mode now returns valid JSON content for non-JSON upstream bodies while still rejecting invalid bodies declared as JSON. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp-proxy): preserve empty text response envelopes Why this change was needed: A raw-payload refactor changed empty text/plain tool responses in the envelope from an empty string to null, which could break clients that distinguish empty text bodies from absent JSON bodies. What changed: - Preserve explicitly read empty non-JSON response bodies as JSON empty strings in client envelopes and log previews - Add a regression test for text/plain empty body envelope output Problem solved: MCP tool envelopes now retain the pre-refactor empty text body semantics while keeping absent or JSON empty bodies as null. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(review/comments): fix cr comments * fix(mcp-proxy): handle unexpected submit result as tool error Keep audit status aligned with the tool-call outcome when the go-openapi submit result violates the expected json.RawMessage reader contract. Also remove the unused status-based payload limit helpers and document the larger audit error response budget in the config template. Co-authored-by: Cursor <cursoragent@cursor.com> * test(mcp-proxy/refactor): add tests * fix(mcp-proxy): preserve audit semantics for raw responses Why this change was needed: Review of the MCP proxy raw-response refactor found several audit and JSON edge cases where the new payload path could diverge from the previous behavior or log misleading success details. What changed: - Treat structured +json media types as JSON while avoiding substring matches - Reject empty declared JSON bodies on content responses and validate raw/envelope paths consistently - Keep audit failure state for early decode errors and rethrow panics after audit logging - Re-render unexpected submit audit previews with the error response budget - Document byte-oriented log preview limits in AGENTS.md Problem solved: MCP tool calls now keep client-visible JSON handling and audit logs aligned across success, upstream error, and defensive failure paths. * refactor(mcp-proxy): share response payload JSON shaping Why this change was needed: The raw-response and envelope response paths had duplicated body-shaping logic for nil payloads, empty bodies, declared JSON, and non-JSON responses. What changed: - Extracted bodyAsJSONValue as the shared response-body conversion helper - Updated marshalEnvelope to embed the shared JSON value as response_body - Updated marshalRawResponse to return the same JSON value as the full MCP response Problem solved: Future changes to upstream response body semantics now happen in one place while preserving the existing raw_response_enabled and envelope output contracts. --------- Co-authored-by: Cursor <cursoragent@cursor.com>


Summary
tools/callupstream responses through raw JSON bytes instead of decoding large JSON intomap[string]any.Test plan
cd src/mcp-proxy && go test -mod=mod ./pkg/infra/proxy -run TestProxycd src/mcp-proxy && go test -mod=mod ./pkg/infra/proxy -run '^$' -bench BenchmarkGenToolHandlerLargeJSONResponse -benchmem -benchtime=3x -count=5cd src/mcp-proxy && make depcd src/mcp-proxy && ./bin/ginkgo -r -mod=vendor ./pkg/infra/proxy/...cd src/mcp-proxy && make lintcd src/mcp-proxy && make testMade with Cursor