HTTPRouteRequestMirror: Should request body be mirrored?

[BorysekOndrej]

What would you like to be added:

• Clarify whether HTTPRouteRequestMirror should mirror the complete HTTP request (headers + body) or only headers
• Update conformance tests to explicitly verify request body mirroring behavior (either as part of HTTPRouteRequestMirror or as part of a new feature)
• Update HTTPRouteRequestMirror documentation to clarify the expected behavior

Why this is needed:

Current conformance tests only verify HTTP header mirroring, creating ambiguity about request body handling. This leads to inconsistent implementations - for example, AgentGateway complies with current tests while only mirroring headers, not the body.

This is problematic for blue-green deployments (or prod&stage) that test POST/PUT endpoints, which require the complete request (including body) to be mirrored for meaningful validation.

---

Kudos to @elivlo for originally identifying this undefined behavior.

[arjunxplorer]

Hi @BorysekOndrej ,

I'd like to work on this.

I dug into the current API + conformance coverage a bit, and I think the issue is more as an underspecification problem than a clear bug against an existing MUST.

Today, RequestMirror is described as mirroring “requests,” and the docs lean on the blue/green / production-traffic use case. That strongly implies the full request (including body), but nothing in the godocs, user guide, or GEP actually says so. On the test side, HTTPRouteRequestMirror only checks that a mirrored request shows up on the mirror backend for the expected path (via echo logs), and the cases are effectively body-less. So an implementation can pass while only forwarding headers which matches what was called out for AgentGateway, and is pretty awkward if you’re trying to validate POST/PUT behavior on the mirrored service.

What I’d like to do:

1. Clarify the intended normative behavior (my read is that the body should be included, possibly with a note about buffering / size limits).
2. Update the API docs + request-mirroring guide accordingly.
3. Extend conformance so we send a request with a body and assert the mirror backend receives it.

@snorwin would love your take on this especially whether body mirroring should be required for HTTPRouteRequestMirror vs something softer / separately gated, and how you’d want that reflected in conformance.
Happy to adjust the approach based on that before opening a PR.

Thank you!!

[bittrance]

I think mirroring the body risks opening up an attack amplifier, where a malicious actor uploads a large payload and the receiver pays twice. An attacker can also create requests that never complete in order to exhaust sockets or SNATs, which would also get amplified. The same amplifications can be achieved during header transfer, but that scenario can be mitigated by the implementation having an arbitrary (large) hard-coded max size or duration. Such mitigations are hard to apply to the body where there are legitimate use cases for any maximum, regardless of value.

I therefore think that the default should be not to mirror the body.

Having said that, it makes perfect sense to be able to mirror the body, probably with max duration and size.

[youngnick]

Good points, thanks @bittrance. I agree that at best, mirroring the body should be an option, that's off by default. As it's a new behavior, it would probably need an Extended feature and associated conformance tests too.

Note that in the past, mirroring conformance tests can be quite tricky to get right and slow, so it's worth thinking about that for anyone who picks this work up.

[elivlo]

@bittrance I understand where you are coming from, but I have to disagree here. Creating a mirror is an intentional action. So while the attack-amplification argument is technical correct, nobody forces me to mirror requests in the first place. The same logic, I could just as well ask why the body is even transferred in the HTTPRoute by default at all. Those requests could never complete either.

I know that mirroring the body is not trivial but there are plenty projects with a working implementation already.

I do like your idea of making it configurable. That would not introduce breaking changes in the spec. But defaulting to false would mean gateways that currently mirror the full body would have to start dropping it. This is effectively a hidden breaking change for any HTTPRoute with RequestMirror that relies on full body mirroring today.

[youngnick]

Unfortunately, @elivlo, this is a rock and a hard place in terms of API.

We can't require implementations to mirror the body, because they may not be right now.

If we do make it configurable, and default it to either "on" or "off", then we risk breaking people.

The only option I can see is to make an "ImplementationDefault" setting that says "bodies will be sent or not, based on how the implementation is configured elsewhere".

Then, you can say "Body mirroring is On", and if the implementation can't do it, it can tell you so. Similarly for "off".

It sucks that we're going to end up with this "ImplementationDefault" thing all over the API for things that we haven't defined yet (we had a similar problem with telemetry config).