feat: Make Rustls the default TLS provider.

[calavera]

This switches the default TLS provider to Rustls. It keeps the native-tls feature with the same configuration it had before as the default TLS provider.

I'm new to the codebase, so let me know all the things I've missed. I've done the minimal work to see what the level or effort was to change the TLS provider. Tests seem to pass.

Todo:

• Integrate with rustls-platform-verifier. See Consider switching to rustls as the default #2025 (comment). WIP
• Update documentation
• Other??

@seanmonstar I'll really appreciate the feedback and guidance. This was the number 1 problem people bumped into when using Reqwest on AWS Lambda functions written in Rust when I maintained the Rust Runtime for Lambda.

Fixes #2723

[calavera]

I'm working through those CI errors.

• The TlsBackend enum requires some changes to work properly.
• The MSRV for tokio-rustls seems to be 1.71.0.
• AWS-LC-RS doesn't seem to build in nightly 😔 https://github.com/seanmonstar/reqwest/actions/runs/15984639773/job/45087493094?pr=2752#step:4:423

[seanmonstar]

Awesome, thanks for starting this! <3

The MSRV for tokio-rustls seems to be 1.71.0.

Sigh, I could look into asking if an older version can be supported, but that's unlikely. We can also setup our MSRV job to pin rustls to a version that works. reqwest tries to provide a working option, but cannot promise all dependencies won't do their own thing.

AWS-LC-RS doesn't seem to build in nightly

Hm, that might need us to update a minimal version of a dependency?

[calavera]

@seanmonstar

Sigh, I could look into asking if an older version can be supported, but that's unlikely. We can also setup our MSRV job to pin rustls to a version that works. reqwest tries to provide a working option, but cannot promise all dependencies won't do their own thing.

The Rustls version that seems to be compatible with 1.64.0 is https://crates.io/crates/rustls/0.23.19, it's 7 months old.
The Tokio-Rustls version that's compatible with that is https://crates.io/crates/tokio-rustls/0.26.0, which is a year old.

Should I update the dependencies in the MSRV job with the other updates?

Hm, that might need us to update a minimal version of a dependency?

I'll look into this today.

[seanmonstar]

The Rustls version that ...

I just looked at them, and actually, that seems fine. In that, in our Cargo.toml, we just depend on the minimum version we need, which perhaps can be 0.23.19. Anyone updating through a normal cargo update will automatically get the later versions, but because we allow the older ones, someone can pin in their Cargo.lock. And then in our msrv job, we add cargo update -p rustls --precise 0.23.19 and cargo update -p tokio-rustls --precise 0.26.0.

[calavera]

I don't even think the dependency versions in Cargo.toml need to change at all:

rustls = { version = "0.23.4", optional = true, default-features = false, features = ["std", "tls12"] }
tokio-rustls = { version = "0.26", optional = true, default-features = false, features = ["tls12"] }

I've just updated the precise dependencies it needs to comply with MSRV in CI. If someone is still using 1.64.0, it should still work.

[ducaale]

quinn has non-optional dependency on ring which is causing the http3 test server to panic:

thread '<unnamed>' panicked at /home/ducaale/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/rustls-0.23.28/src/crypto/mod.rs:249:14:
no process-level CryptoProvider available -- call CryptoProvider::install_default() before this point

thread 'http3_request_full' panicked at tests/support/server.rs:283:10:
called `Result::unwrap()` on an `Err` value: Any { .. }

One workaround it is to manually install the aws_lc_rs provider, but it would be nice if ring was an optional dependency in quinn

// reqwest/tests/support/server.rs

// see https://github.com/rustls/rustls/issues/1938#issuecomment-2567934864 for why result is ignored
let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); // added this line

let mut tls_config = rustls::ServerConfig::builder()
    .with_no_client_auth()
    .with_single_cert(vec![cert], key)
    .unwrap();

[calavera]

@ducaale those are "Workspace dependencies", it's not a non-optional dependency. Quinn does have features to enable both provider Ring and AWS-LC optionally. I've fixed the problem in dd9e110

See the features in https://github.com/quinn-rs/quinn/blob/845e68daa79278beddd30eafa30c6b2c750ed22d/quinn/Cargo.toml#L20

[djc]

Thanks for working on this. Maybe it would be good to make the hostname-ignoring verifier a Cargo feature, with an appropriately signalling name like dangerous-rustls-ignore-hostname (while only activating when also enabled via API)?

[djc]

Suggest getting rid of the native-roots option, rustls-platform-verifier is superior to it.

[calavera]

I still need to figure out how rustls-platform-verifier would fit in here. I don't think that removing options that are currently present is a good solution, as it could break people that use them for whatever reason. My current goal is to make this work with the features Reqwest has at the moment, and then extend it later.

[djc]

Is it worth having this? If a downstream is not setting any __* options this doesn't happen, right?

[calavera]

It could happen if someone cherry picks features without knowing what they enabled. Since this check was already here, I rather keep it.

[seanmonstar]

I think it can still happen, if a user enabled a *-no-provider feature and didn't "install" a default using rustls directly, right?

[djc]

Fair!

[djc]

rustls-platform-verifier has been following along with rustls's own MSRV, which is 1.71 IIUC. Is that not also an issue for reqwest here?

rustls has advanced its MSRV within the 0.23.x release series, so downstream users who are sensitive to MSRV bumps can just use an older rustls (though this seems like a pretty bad idea). On the other hand, rustls-platform-verifier has made several semver-incompatible releases while going through the same MSRV bump.

[complexspaces]

After a quick check, it seems non-trivial to downgrade rustls-platform-verifier's MSRV to match reqwests and also support the latest version of crates without causing problems. There's both crate duplication issues and API incompatibilities supporting the whole range of rustls 0.23.x.

[seanmonstar]

can just use an older rustls (though this seems like a pretty bad idea)

That's the reality some people live in, and that's why I try to support Debian stable. I've found personally that it's not hard at all to keep an older MSRV within the library itself, but controling dependencies is rough. Where it's possible to pin in a CI job, I do that to ensure that I haven't forced the MSRV on users, they can make the decision themselves whether using an older version is good for them.

Regarding the integration with rustls-platform-verifier, I think you'd need to decide whether you want to support it when this change lands or not.

@calavera First of all, amazing work. Really, thank you! ❤️

To your question, I have concerns, myself. I wouldn't want to recommend by default trusting the OS verifier stack, it's not often a good part of the OS. Some OSes have marked it deprecated (such as Apple), and applications like Chrome and Edge don't use them anymore (1). They provide their own verifier, with the ability to check the OS keystore for any additionally installed certificates. That's what I would want to use for reqwest's default.

I do wonder if, in the meantime, we can just make the default an under-specified "it verifies things", that we could improve when there's something like what I just described available. We probably could make all of that a separate PR, either way.

What do you think?

[complexspaces]

Some OSes have marked it deprecated (such as Apple)

FWIW this is not true. Nothing that is used for chain verification in Apple's Security.framework is deprecated. Apple deprecated their SecureTransport.framework APIs, which are entirely different (they provide an entire TLS stack, like rustls does). I am not aware of any OS that has deprecated this functionality either.

[ctz]

To your question, I have concerns, myself. I wouldn't want to recommend by default trusting the OS verifier stack, it's not often a good part of the OS. Some OSes have marked it deprecated (such as Apple)

I'm not sure that is right, but equally I'm not an Apple expert. The thing Apple recently deprecated was "Secure Transport" (an SSL API) and the verifier API is in Security framework. I understand that Apple also have a policy that this API must be used on iOS apps.

and applications like Chrome and Edge don't use them anymore (1). They provide their own verifier, with the ability to check the OS keystore for any additionally installed certificates. That's what I would want to use for reqwest's default.

The closest we have to that is rustls-native-certs.

It's worth noting that it falls short of what you get when depending on the platform verifier (on good platforms¹) or what a browser does. Equalising that at the level of a library is pretty challenging, and platforms/browsers can do this because they are long-lived applications where it is universally accepted that they are online most of the time, and can both update themselves and download fresh data.

That's a long way of saying that using rustls-native-certs means there's no reasonable way of doing revocation, tracking CA distrusts, or checking certificate transparency proofs. rustls-platform-verifier does give a route to achieving those things (depending on the platform.)

Footnotes

1. The platforms vary, of course. Linux is an notably weak in this area. ↩

[seanmonstar]

Fair, I conflated Apple's verifier code with SecureTransport.

[calavera]

I do wonder if, in the meantime, we can just make the default an under-specified "it verifies things", that we could improve when there's something like what I just described available. We probably could make all of that a separate PR, either way.

What do you think?

That makes sense to me. I’d prefer to put it in a separate PR for sure. We could start by leaving the default verifier as is, and add rustls-platform-verifier as an optional feature for people that want to rely on it, disabling apis that are not currently supported in that crate.

[seanmonstar]

That sounds good to me.

As I mentioned in the implementation issue, I've gone ahead and made a branch to hold all these steps (rustls-default), so we can merge and have some incremental commits, but still keep master ready to release fixes for now. I've changed the target for this PR, and I can merge into in a little bit, if there's nothing else you wanted to touch first.

[calavera]

I can merge into in a little bit, if there's nothing else you wanted to touch first.

I’m good. Thanks for making that branch. I’ll open a new PR against it soon.