Breaking change in .NET 9 behavior for the ability to allow duplicate attributes in a PFX

[ppekrol]

Description

We are aware that .NET 9 brought significant number of API (and behavior) changes to the framework. One issue that we are struggling is that some of the certificates that we are dealing with have duplicate attributes.

There was a discussion about this topic here: #103924

It appears that Pkcs12LoaderLimits has a property called AllowDuplicateAttributes, but it is internal. Shouldn't this be public?

We are aware that there is a 'Pkcs12LoaderLimits.DangerousNoLimits', but we do not want to disable everything, just have an option to change this one limit.

At the moment we are using reflection to set it:

        var allowDuplicateAttributes = new Pkcs12LoaderLimits(Pkcs12LoaderLimits.Defaults);
        var allowDuplicateAttributesProperty = allowDuplicateAttributes.GetType().GetProperty("AllowDuplicateAttributes", BindingFlags.Instance | BindingFlags.NonPublic);
        Debug.Assert(allowDuplicateAttributesProperty != null, "AllowDuplicateAttributes != null");
        allowDuplicateAttributesProperty.SetValue(allowDuplicateAttributes, true);
        var certificate = X509CertificateLoader.LoadPkcs12FromFile(Path.Combine(""), password: null, loaderLimits: allowDuplicateAttributes);

Reproduction Steps

var cert = X509CertificateLoader.LoadPkcs12FromFile("pathToCertFileWithDuplicateAttributes");

System.Security.Cryptography.X509Certificates.Pkcs12LoadLimitExceededException
HResult=0x80131501
Message=The PKCS#12/PFX violated the 'AllowDuplicateAttributes' limit.
Source=System.Security.Cryptography
StackTrace:
at System.Security.Cryptography.X509Certificates.X509CertificateLoader.RejectDuplicateAttributes(AttributeAsn[] bagAttributes, HashSet`1 duplicateAttributeCheck)
at System.Security.Cryptography.X509Certificates.X509CertificateLoader.ProcessSafeContents(ReadOnlyMemory`1 contentData, Pkcs12LoaderLimits loaderLimits, Nullable`1& workRemaining, BagState& bagState)
at System.Security.Cryptography.X509Certificates.X509CertificateLoader.ReadCertsAndKeys(BagState& bagState, ReadOnlyMemory`1 data, ReadOnlySpan`1& password, Pkcs12LoaderLimits loaderLimits)
at System.Security.Cryptography.X509Certificates.X509CertificateLoader.LoadPkcs12(ReadOnlyMemory`1 data, ReadOnlySpan`1 password, X509KeyStorageFlags keyStorageFlags, Pkcs12LoaderLimits loaderLimits)
at System.Security.Cryptography.X509Certificates.X509CertificateLoader.LoadPkcs12(Byte[] data, String password, X509KeyStorageFlags keyStorageFlags, Pkcs12LoaderLimits loaderLimits)

Expected behavior

Have the ability to override default Pkcs12LoaderLimits and set the AllowDuplicateAttributes to desired value

Actual behavior

Exception is thrown, no ability to override it beside using reflection.

Regression?

Was working in .NET 8

Known Workarounds

        var allowDuplicateAttributes = new Pkcs12LoaderLimits(Pkcs12LoaderLimits.Defaults);
        var allowDuplicateAttributesProperty = allowDuplicateAttributes.GetType().GetProperty("AllowDuplicateAttributes", BindingFlags.Instance | BindingFlags.NonPublic);
        Debug.Assert(allowDuplicateAttributesProperty != null, "AllowDuplicateAttributes != null");
        allowDuplicateAttributesProperty.SetValue(allowDuplicateAttributes, true);
        var certificate = X509CertificateLoader.LoadPkcs12FromFile(Path.Combine("pathToCertWithDuplicateAttributes"), password: null, loaderLimits: allowDuplicateAttributes);

Configuration

No response

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[bartonjs]

Where did you get a certificate with duplicate attributes, and what attributes are duplicated?

[ppekrol]

Something that we will be looking into shortly, we are generating client certificates here: https://github.com/ravendb/ravendb/blob/1c37d8e3792de0321bcd826fc9dfc4ada4b2f72a/src/Raven.Server/Utils/CertificateUtils.cs#L307
but we cannot turn back time and numerous client access certificates were created already, so beside fixing that on our side, it would be great to have an ability to go back to prev (.NET 8 and before) behavior IMO.

[bartonjs]

Oh, right, this isn't a "certificate" with duplicate extensions (which you actually appear to have for CA certs), it's a PFX SafeBag that has duplicate attributes in it.

The main trouble, as the discussion outlined, is that if we allow you to set it to true we need to decide what that means. We didn't. So using AllowDuplicates=true without using DangerousNoLimits might have behavioral differences on Windows.

If this comes down to "Bouncy Castle always gets this wrong and we somehow never noticed", we'd need to think about something different; but if this is a localized thing I don't know that we want to put a lot of effort into normalizing those paths. (And it would be a nontrivial amount of effort, not just "expose the boolean")

[hiddenshadow21]

Hi,

We created scenario showing the problem (see app attached in the zip).
Self-signed certificates are working correctly - the scenario 1.

In scenario 2 we are creating CA and server certificate. Loading these certificates is working correctly as well.

In scenario 3 we are creating whole chain - CA, server, and client certificates. As a last step we are attaching issuer certificate to generated client to have correct chain using bouncyCastle Pkcs12Store:

        Pkcs12Store store = new Pkcs12StoreBuilder().Build();
        var serverCert = DotNetUtilities.FromX509Certificate(certificateHolder.Certificate);

        store.Load(new MemoryStream(certBytes), Array.Empty<char>());
        store.SetCertificateEntry(serverCert.SubjectDN.ToString(), new X509CertificateEntry(serverCert));

        var memoryStream = new MemoryStream();
        store.Save(memoryStream, Array.Empty<char>(), GetSeededSecureRandom());

While debugging we found that loading our client.pfx fails duplicate check for OID 2.16.840.1.113894.746875.1.1.
This one is added automatically, under the hood, when saving PKCS12 store with certificate that does not have private key attached and has an EKU (see source code here).
In our case it adds two values since our server certificate has 2 EKUs.

Any idea how can we make it compliant with .Net 9?

PfxLimitsNet9.zip

[ppekrol]

@bartonjs Did you had a chance to take a peek what @hiddenshadow21 wrote? We are not sure if this is us, Bouncy Castle or .NET 9 at the moment, so it is hard for us to judge how to fix the issue. By glance it looks like this is something that Bouncy Castle is doing by design?