Skip to content

Kestrel inconsistent certificate chain handling between endpoint and default configuration #60709

Closed
@jnjudge1

Description

@jnjudge1

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

On Kestrel server, there is inconsistent behavior configuring certificates per endpoint versus in the “Default” configuration. The discrepancy occurs when specifying a certificate file containing a chain of certificates where the chain is not included in the system's trust store.

I tested this on Windows, Mac, and Ubuntu using certificate files of the form:

server_certificate.crt:
-----BEGIN CERTIFICATE-----
<leaf>
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
<intermediate>
-----END CERTIFICATE-----
server_key.key
-----BEGIN EC PRIVATE KEY-----
<key>
-----END EC PRIVATE KEY-----

The files were generated using step CLI, and none of the root or intermediate certificates were in the system trust store initially. For each of the following, I removed the certificates from the trust store before each test.

On Windows:

If a certificate file is specified in an endpoint's configuration section, Kestrel will:

  • Output the full certificate chain in its TLS response
  • Add the intermediate certificate to the Windows certificate store if it is not already present.
appsettings.json:
{
  "Kestrel": {
    "Endpoints": {
      "UsingEndpointConfiguration": {
        "Url": "https://localhost:5000",
        "Certificate": {
          "Path": "server_certificate.crt",
          "KeyPath": "server_key.key"
        }
      },
      "UsingDefaultConfiguration": {
        "Url": "https://localhost:5001",
      }
    },
    "Certificates": {
      "Default":{
        "Path":"server_certificate.crt",
        "KeyPath": "server_key.key"
      }
    }
  }
}

openssl s_client -connect localhost:5000 -showcerts and openssl s_client -connect localhost:5001 -showcerts correctly responds with a full certificate chain as specified in server_certificate.crt.

The intermediate will also be added to the Windows Certificate Store if it is not present.

If the certificate file is specified only in the default configuration section:

  • The chain is not presented if it is not already in the Windows certificate store.
  • The intermediate certificate is not added to the store.
{
  "Kestrel": {
    "Endpoints": {
      "UsingDefaultConfiguration": {
        "Url": "https://localhost:5000"
      }
    },
    "Certificates": {
      "Default":{
        "Path":"server_certificate.crt",
        "KeyPath": "server_key.key"
      }
    }
  }
}

openssl s_client -connect localhost:5000 -showcerts only responds with the leaf certificate specified in server_certificate.crt, and the intermediate is not added to the certificate store.

On MacOS:

If a certificate file is specified in an endpoint's configuration section, Kestrel will:

  • Output the full certificate chain in its TLS response for that endpoint. We will receive the full chain from the following configuration.
appsettings.json:
{
  "Kestrel": {
    "Endpoints": {
      "UsingEndpointConfiguration": {
        "Url": "https://localhost:5000",
        "Certificate": {
          "Path": "server_certificate.crt",
          "KeyPath": "server_key.key"
        }
      }
    }
  }
}

If the certificate file is specified in the default configuration section:

  • The certificate chain is not presented by endpoints using the default configuration, regardless of if it is specified for other endpoints.
appsettings.json:
{
  "Kestrel": {
    "Endpoints": {
      "UsingEndpointConfiguration": {
        "Url": "https://localhost:5000",
        "Certificate": {
          "Path": "server_certificate.crt",
          "KeyPath": "server_key.key"
        }
      },
      "UsingDefaultConfiguration": {
        "Url": "https://localhost:5001",
      }
    },
    "Certificates": {
      "Default":{
        "Path":"server_certificate.crt",
        "KeyPath": "server_key.key"
      }
    }
  }
}

openssl s_client -connect localhost:5001 -showcerts will only present us the leaf certificate.

Behavior on Ubuntu:

  • The behavior observed on Ubuntu is the same as on MacOS.

Relevant Code Snippet

This behavior appears to be caused by this line in TlsConfigurationLoader.cs:

 var (defaultCert, _ /* cert chain */) = _certificateConfigLoader.LoadCertificate(defaultCertConfig, "Default");

which does not process the full certificate chain when it is used in the default configuration section. As a result, the subsequent call to SslStreamCertificateContext.Create does not add the full certificate chain:

_serverCertificateContext = SslStreamCertificateContext.Create(certificate, additionalCertificates: options.ServerCertificateChain);

On Mac and Ubuntu this means the chain is never presented on endpoints using the default configuration.

On Windows, SslStreamCertificateContext.Create adds intermediates specified in the call to the Windows Certificate Store if they are not already present. This means if the certificate chain has been specified on an endpoint level configuration, it will be added to the Windows trust store, and subsequently any endpoints specified using the default configuration will also have their full chain presented.

However, if the file is only specified on the default configuration, the chain will be thrown out and never added to the trust store from this call, so any endpoints using the default configuration will only present their leaf certificates.

Expected Behavior

The behavior when specifying the certificate file in an individual endpoint’s configuration should have identical behavior to endpoints using the default configuration. The full certificate chain should be included for endpoints using the default configuration when possible.

Steps To Reproduce

  1. Generate some certificates using step CLI or similar CA such as Hashicorp Vault, where intermediates are not in the system trust store
  2. Create a minimal dotnet app, for example: repo
  3. Run the app, then openssl s_client -connect localhost:5000 -showcerts, and observe the certificate chain

Exceptions (if any)

No response

.NET Version

9.0.103

Anything else?

IDE: VSCode 1.97.2

Tools

step cli: https://github.com/smallstep/cli

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-networkingIncludes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions