Add opt-in request body passthrough to avoid buffering large uploads

[emmanuelchaplais2]

Is your feature request related to a problem? Please describe.

Request bodies are currently buffered at three sites in the Gateway's request path: the muxer (pre-routing) and the reverse proxy (after routing). This buffering multiplies memory consumption for large uploads.

In production, a 1 GiB chunked POST (e.g., file upload with chunked transfer encoding) causes a peak RSS delta of 2.28 GiB across the three buffering sites, despite the body flowing unchanged to the upstream. For streaming-capable upstreams (S3, file servers, media ingest), the buffering provides no value—it only delays upload completion and taxes memory on the Gateway.

Prior work acknowledged this issue (#2357, #7754, TT-5186 / #3976, TT-14016 / #8150): TT-5186 proposed a streaming path but was closed without implementation; #8150 documented the memory multiplication but did not ship a configurable opt-in.

Describe the solution you'd like

Add a default-off, two-layer opt-in for request body passthrough:

1. Gateway-wide gate: new field http_server_options.enable_request_body_passthrough (boolean, defaults to false). Controls buffering at the muxer (pre-routing sites).
2. Per-API flag: new field server.requestBodyPassthrough.enabled in OAS (or enable_request_body_passthrough in classic API definitions). Controls redundant copying at the reverse proxy (post-routing site).

When both are true for a request, the body streams directly to the upstream with no intermediate buffering. Requests where only one is true degrade gracefully to today's buffered behavior (safe, no regression).

Approach & Design Rationale:

• Muxer buffering is unavoidable when the per-API flag is unknown (pre-routing has no APISpec). The gateway-wide gate gates both muxer sites unconditionally.
• Reverse proxy buffering (deepCopyBody) runs after routing, so it can see the per-API flag independently. It skips only when that flag is true, improving memory even if the gateway-wide gate is off.
• Both gates must be true for end-to-end streaming, preventing accidental exposure on non-opted APIs while allowing each layer to improve independently.
• The implementation is config-gated (no mutable hook variables), non-panicking (plain bool reads), and compatible with existing middleware and size limits.

Describe alternatives you've considered

1. Per-API-only gate (no gateway-wide toggle): Muxer buffering happens regardless, so memory only improves at the reverse proxy. Limits the win to ~33% instead of 100%. Rejected because the muxer is the primary multiplier (greedy full read in nopCloseRequestBodyErr).

2. Labs config entry: Defers maturity feedback to after merge. Rejected because the design is straightforward and test coverage is comprehensive; a labs entry adds no safety beyond the two-layer default-off structure.

3. Hook variable (SeqOne's internal approach): Allows runtime override without restart. Rejected because it requires a mutable global state and panic-safety wrapper; config-driven opt-in aligns with Tyk's established patterns (Add per-API option to disable upstream chunked transfer encoding by buffering in Tyk (with safe defaults) #7754, labs map) and avoids the complexity.

Additional context

• Memory proof: tested on 1 GiB file uploads (Content-Length-framed and chunked). Peak RSS delta on master: 2.28 GiB (3× payload). With passthrough enabled: <0.1 GiB (near-identity). Mirrors the technique in test: add test to prove memory multiplier issue during large file uploads (TT-14016) #8150.

• Buffering sites on master (version TBD, master HEAD):

  • Site A: gateway/proxy_muxer.go:82 (nopCloseRequestBodyErr on maxRequestBodySize > 0 path) — greedy read, all frames
  • Site B: gateway/proxy_muxer.go:93 (copyRequest on maxRequestBodySize == 0 path) — lazy read, exempts chunked (ContentLength -1)
  • Site C: gateway/reverse_proxy.go:deepCopyBody — always runs unless skipped (IsStreamingRequest or new per-API flag)

• Compatibility: Streaming skips buffering, not size-limit enforcement. max_request_body_size still enforced via non-buffering http.MaxBytesReader. Body-consuming middlewares (validation, JQ, plugins) already re-read and re-wrap the body, so streaming is safe (inert if a middleware consumes the body).

• Analytics safeguard: DetailedActivityLogs and GraphQL analytics assume a re-readable body. A short-circuit early-return on streaming requests (mirroring the existing websocket/gRPC pattern) prevents empty/corrupt logs.

• Related Tyk issues: TT-5186 (streaming design, closed), [TT-5186] Request body handling can cause high memory usage #3976 (archived, same as TT-5186), TT-14016 (internal issue acknowledging the memory cost), test: add test to prove memory multiplier issue during large file uploads (TT-14016) #8150 (memory-proof PR, no config shipped), Add per-API option to disable upstream chunked transfer encoding by buffering in Tyk (with safe defaults) #7754 / TT-17896: add ability to turn off transfer encoding to specific API targets #7755 (per-API opt-in precedent for handling mechanisms).

[probelabs]

Hi @emmanuelchaplais2,

Thank you for this incredibly detailed and well-researched feature proposal. Your analysis of the request body buffering at multiple sites within the gateway and the resulting memory multiplication for large uploads is spot on. This is a known area for improvement, and we appreciate you bringing it forward with such a clear and actionable plan.

The proposed two-layer opt-in solution (http_server_options.enable_request_body_passthrough at the gateway level and a per-API flag) seems like a robust and safe approach. It provides the necessary control to enable streaming passthrough while maintaining backward compatibility and preventing accidental changes in behavior for existing APIs.

We particularly appreciate the detailed breakdown of the buffering sites, the consideration of alternative designs, and the analysis of compatibility with features like size limits, middleware, and analytics. The references to previous issues and PRs are also very helpful.

This proposal provides a strong foundation for addressing the memory overhead of large uploads. The next step will be for our engineering team to review the design in detail. Are you interested in contributing a pull request for this feature? Your deep understanding of the problem space would be a great asset.

We've labeled this as a feature and will track it accordingly. Thanks again for your excellent contribution to Tyk.

References

• None

---

Powered by Visor from Probelabs

💡 TIP: You can chat with Visor using /visor ask <your question>