refactor(ws): reuse marshaled payload JSON for write and prewarm#1642
Open
penwyp wants to merge 3 commits intoWei-Shaw:mainfrom
Open
refactor(ws): reuse marshaled payload JSON for write and prewarm#1642penwyp wants to merge 3 commits intoWei-Shaw:mainfrom
penwyp wants to merge 3 commits intoWei-Shaw:mainfrom
Conversation
The branch originally mixed JSON reuse on the WS write path with payload summary micro-optimizations, benchmark churn, and a repo-local ignore rule. This follow-up narrows the PR to one reviewable theme: reuse the marshaled payload JSON for WS write and prewarm paths while keeping the focused regression coverage that proves wire equivalence and fail-closed behavior. Constraint: The upstream repo favors small, focused PRs with explicit validation Rejected: Keep payload summary micro-optimizations | broadened the PR without proving value for this submission Rejected: Keep .omx ignore change | unrelated to gateway behavior Confidence: high Scope-risk: narrow Reversibility: clean Directive: Keep future WS perf PRs scoped to one hot path unless the same benchmark justifies bundled changes Tested: /Users/penwyp/.local/share/mise/installs/go/1.26.2/bin/go test ./internal/service/... Tested: /Users/penwyp/.local/share/mise/installs/go/1.26.2/bin/go test ./... Not-tested: Benchmark delta on production-like websocket traffic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
forwardOpenAIWSV2can end up marshaling the same request payload twice: once to resolvepayload_bytesfor WS schema logging and again when the payload is written to the websocket.performOpenAIWSGeneratePrewarmhad the same pattern: it built JSON bytes for logging but still passed the map toWriteJSONWithContextTimeout, forcing another marshal on write.This PR narrows the WS optimization work to one focused change: marshal the payload once, reuse the same JSON bytes for the actual WS write/prewarm write, and fail closed if marshaling fails.
Changes
backend/internal/service/openai_ws_forwarder.goforwardOpenAIWSV2and reuse it for websocket writes viajson.RawMessageperformOpenAIWSGeneratePrewarm, log the byte size, and write the same JSON upstreambackend/internal/service/openai_ws_forwarder_success_test.gojson.Marshal(payload)json.Marshal(prewarmPayload)Test plan
cd backend && go test ./internal/service/...cd backend && go test ./...