Use forwarding proxy mode for plaintext endpoints in CRT S3 client - #7321
Open
MingWangSong wants to merge 1 commit into
Open
Use forwarding proxy mode for plaintext endpoints in CRT S3 client#7321MingWangSong wants to merge 1 commit into
MingWangSong wants to merge 1 commit into
Conversation
The CRT derives the proxy connection type from whether TLS options were supplied for the main connection, not from the scheme actually in use. Because S3NativeClientConfiguration always builds a TlsContext and S3CrtAsyncHttpClient passes it to the native client unconditionally, a plaintext http:// endpoint was still reached through a CONNECT tunnel. Proxies that only support forwarding mode reject those requests. Pin the proxy connection type to Forwarding when endpointOverride uses the http scheme, mirroring the scheme check that the CRT's own HttpClientConnectionManager already performs before handing a TLS context to the native layer. Only the combination of a configured proxy and a plaintext endpoint override is affected; https endpoints, endpoints without an override, and proxy-less configurations continue to resolve through Legacy as before. Not passing the TlsContext was evaluated first and does not work: aws-c-s3 still tunnels, so the connection type has to be set explicitly. Fixes aws#7320
MingWangSong
force-pushed
the
fix-crt-plaintext-endpoint-proxy-forwarding
branch
from
August 27, 2026 01:46
93d67b6 to
e8e6e08
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Fixes #7320.
When an HTTP proxy is configured, the CRT-based S3 client always establishes the proxy connection through a
CONNECTtunnel, even whenendpointOverrideuses the plaintexthttp://scheme. Proxies that only support forwarding mode — a common corporate configuration, since tunnels defeat traffic inspection — reject these requests with:The emitted request is self-contradictory: the client sends
CONNECT host:80, so it clearly knows the target is plaintext (hence port 80), yet still negotiates a tunnel as if TLS were in use.This is inconsistent with the rest of the SDK. Given identical configuration:
http://endpointhttps://endpointApacheHttpClient(syncS3Client)GET http://host/...CONNECT host:443S3AsyncClient.crtBuilder())CONNECT host:80CONNECT host:443It is also inconsistent with the CRT client's own behaviour without a proxy, where an
http://endpoint is correctly reached in plaintext.Root cause.
aws-c-httpderives the proxy connection type from whether TLS options were supplied for the main connection:HttpProxyOptionsdefaultsconnectionTypetoLegacyandCrtConfigurationUtils.resolveProxynever overrides it, so that rule always applies. MeanwhileS3NativeClientConfigurationbuilds aTlsContextunconditionally andS3CrtAsyncHttpClienthands it to the native client unconditionally — so the main connection always "has TLS options", regardless of the endpoint scheme. In effect, "the client holds a TLS context" is being treated as "this connection uses TLS".Note that
aws-crt-java's ownHttpClientConnectionManageralready guards this correctly:Modifications
S3NativeClientConfigurationnow pins the proxy connection type toForwardingwhenendpointOverrideuses thehttpscheme, mirroring the scheme check the CRT already performs elsewhere.HttpProxyConnectionType.ForwardingandHttpProxyOptions.setConnectionType()are already public inaws-crt-java.endpointOverride. Every other path is untouched:https://endpoints, endpoints without an override, and configurations without a proxy all keep resolving throughLegacyexactly as before.tlsContexthandling is deliberately left alone. Not passing theTlsContextwas evaluated first and does not work —aws-c-s3still tunnels. The connection type has to be set explicitly.Testing
Unit tests added to
S3NativeClientConfigurationTest:build_proxyConnectionType_followsEndpointScheme— parameterized overhttp://,HTTP://(case-insensitivity),https://, and no endpoint override, assertingForwardingonly for the plaintext cases.build_whenPlaintextEndpointAndNoProxy_shouldNotSetProxyOptions— guards the null-proxy path.End-to-end verification against a stub proxy that records the first line it receives:
connectionTypetlsContextLegacy(before this change)CONNECT host:80LegacyCONNECT host:80Forwarding(after this change)GET http://host:80/...Confirmed
Forwardingcombined with a non-nullTlsContextworks and does not trigger the "configuration error" mentioned in theAWS_HPCT_HTTP_FORWARDdoc comment — that refers to TLS on the tunnel destination, not to the client holding a context.Also verified that
https://endpoints continue to tunnel, and that requests without a proxy are unaffected.Client-level verification against the stub proxy, before and after the change:
http://CONNECT host:80GET http://host:80/bucket?list-type=2https://CONNECT host:443CONNECT host:443(unchanged)http://CONNECT host:80CONNECT host:80(unchanged — out of scope for this PR)Downstream verification. Also verified through a real downstream consumer — Alluxio's S3 under-filesystem connector, which constructs its CRT client via
S3AV2UnderFileSystem.createAmazonS3CRTAsync(...)from mount-level configuration. Running that code path unchanged, only swapping the SDK jar:http://CONNECT host:80http://GET http://host:80/bucket?list-type=2https://CONNECT host:443(unchanged)mvn installforservices/s3and its dependencies succeeds,S3NativeClientConfigurationTestpasses (9 tests), andcheckstyle:checkreports 0 violations.Types of changes
Checklist
mvn installsucceedsscripts/new-changescript and following the instructions. Commit the new file created by the script in.changes/next-releasewith your changes.License