g3proxy: Ability to change ClientHello sent to upstream server

[mspublic]

When using g3proxy it is sometime detected by websites running on cloudflare and other sites using TLS fingerprinting such as https://github.com/salesforce/ja3. Having the ability to modify the ClientHello will make the proxy less detectable/less fingerprintable and more usable in enterprise environments.

The ideal “solution” would be to be able to set a ja3 fingerprint and have the proxy send it.

I have not found a way to modify it via OpenSSL but it appears rustls may give access to the ClientHello https://docs.rs/reqwest/latest/reqwest/struct.ClientBuilder.html#method.use_preconfigured_tls

Here is a bit more info and code examples:

https://medium.com/cu-cyber/impersonating-ja3-fingerprints-b9f555880e42

https://github.com/refraction-networking/utls

https://github.com/Danny-Dasilva/CycleTLS

https://github.com/LyleMi/ja3proxy

https://github.com/Kolosok86/http-tls-proxy

[zh-jq-b]

Rustls doesn't support the customization of ClientHello Message at the time of now, see rustls/rustls#1564.

To implement this, we have to parse the client hello message sent by the tls providers and then rewrite them at the underlying stream layer, which may have some limitation for adding unknown ciphersuites and extensions.

For OpenSSL, we will add a new SslConnector in g3-openssl crate in the following days, then we can experiment with client hello rewriting there.

[mspublic]

My mistake. I got reqwest and rustls mixed up.

I look forward to the change! Thanks for the fast reply. G3 is great work!

[zh-jq-b]

Change the client message in the underlying layer is not working, as OpenSSL will check record MAC.
I have added extension reordering ability in branch https://github.com/bytedance/g3/tree/openssl-ja3, and the test failed with OpenSSL send BadRecordMAC alert message to target site.

[mspublic]

Sorry for delay - I did not receive a notification.

Thanks for the update! I will review the code. I think this will become a big hinderance to enterprise usage - many sites using fingerprinting/cloudflare etc will block requests that appear to be plain OpenSSL as they think it's a bot. This is one of the major issues with Squid - detected as a bot. Sites such as https://chat.openai.com/ don't work. It makes deployment into most enterprises a problem.

I see you have been working on adding BoringSSL support - this may be the way to go. It seems to be leveraged by other projects to impersonate Chrome - https://dev.to/gssvv/making-tls-client-with-chrome-like-ssl-handshake-rust-boring-ssl-h2-n63

It seems to be a good path to go down as openssl seems a bit strict on the ability to modify.

[zh-jq-b]

I see you have been working on adding BoringSSL support - this may be the way to go. It seems to be leveraged by other projects to impersonate Chrome - https://dev.to/gssvv/making-tls-client-with-chrome-like-ssl-handshake-rust-boring-ssl-h2-n63

The missing seems to be

set_grease_enabled
enable_ocsp_stapling
enable_signed_cert_timestamps

Can you try adding this to the rel/boringssl branch and test it? I won't available for this feature this week.

[zh-jq-b]

For OCSP stapling:

• https://knowledge.digicert.com/quovadis/ssl-certificates/ssl-general-topics/what-is-ocsp-stapling
• OpenSSL ways: https://www.openssl.org/docs/man1.1.1/man3/SSL_set_tlsext_status_ocsp_resp.html
• BoringSSL deprecated ways: https://commondatastorage.googleapis.com/chromium-boringssl-docs/ssl.h.html#SSL_set_tlsext_status_type
• BoringSSL new ways: https://commondatastorage.googleapis.com/chromium-boringssl-docs/ssl.h.html#SSL_enable_ocsp_stapling

[zh-jq-b]

For GREASE:

• https://www.rfc-editor.org/rfc/rfc8701.html
• OpenSSL missing: Implement RFC 8701 (was draft-ietf-tls-grease) openssl/openssl#9660
• BoringSSL ways: https://commondatastorage.googleapis.com/chromium-boringssl-docs/ssl.h.html#SSL_CTX_set_grease_enabled

[zh-jq-b]

For SCT:

• https://datatracker.ietf.org/doc/html/rfc6962#section-3.2
• OpenSSL ways (via OCSP): https://www.openssl.org/docs/man1.1.1/man3/SSL_enable_ct.html
• BoringSSL ways: https://commondatastorage.googleapis.com/chromium-boringssl-docs/ssl.h.html#SSL_CTX_enable_signed_cert_timestamps

[Millione]

now supported by rustls/rustls#1730

[zh-jq-b]

OCSP Stapling, SCT, GREASE have been added as config options, and you can test them when build with AWS-LC or BoringSSL. And the extension order is random.

The still missing extensions when comparing with chromium are:

• application-settings
• compress-certificate
• encrypted-client-hello

[zh-jq-b]

ALPS:

• RFC(https://datatracker.ietf.org/doc/html/draft-vvv-tls-alps-01)
• BoringSSL(https://commondatastorage.googleapis.com/chromium-boringssl-docs/ssl.h.html#SSL_add_application_settings)

ECH:

• RFC(https://datatracker.ietf.org/doc/html/draft-ietf-tls-esni-17)
• BoringSSL(https://commondatastorage.googleapis.com/chromium-boringssl-docs/ssl.h.html#SSL_set1_ech_config_list)

[mspublic]

This is great progress! Thank you @zh-jq-b! I finally have some time to do some more testing of this.

[mspublic]

I have been testing with BoringSSL enabled and it has been working great! I find that during regular use I run into less issues using the BoringSSL feature than standard OpenSSL (which is typical of OpenSSL usage in general).

I did notice that it isn't doing TLS extension permutation which is a newer feature to prevent fingerprinting https://chromestatus.com/feature/5124606246518784.

It should be a pretty easy addition. The code is enabled the same way GREASE is. I was going to take a stab at it by reusing the grease code and submitting a PR to as well variant-ssl too. Is that ok? Or would you prefer to?

Thoughts?

https://github.com/google/boringssl/blob/db614a5677d90e48cfb2c0f8197f1b5168fceea5/ssl/ssl_lib.cc#L3015

void SSL_CTX_set_permute_extensions(SSL_CTX *ctx, int enabled) {
  ctx->permute_extensions = !!enabled;
}

void SSL_set_permute_extensions(SSL *ssl, int enabled) {
  if (!ssl->config) {
    return;
  }
  ssl->config->permute_extensions = !!enabled;
}

[zh-jq-b]

I did notice that it isn't doing TLS extension permutation which is a newer feature to prevent fingerprinting https://chromestatus.com/feature/5124606246518784.

It should be a pretty easy addition. The code is enabled the same way GREASE is. I was going to take a stab at it by reusing the grease code and submitting a PR to as well variant-ssl too. Is that ok? Or would you prefer to?

An extra config option should be used and the default value should be set to false.

I haven't look into the details but I remember that the tls ext ordering is already randomized when GREASE is enabled. Do you know what's the difference?

[zh-jq-b]

permute_extensions added in commit 27e34e0