`public_key` certificate decode APIs ignore trailing bytes after a complete DER certificate

[tynus2]

Describe the bug

public_key:der_decode/2 and public_key:pkix_decode_cert/2 return success for a complete DER certificate that has extra bytes appended after the top-level TLV. The leftover bytes are discarded and are not returned to the caller.

The low-level TLV decoder does see them: asn1rt_nif:decode_ber_tlv/1 returns {Tlv, Rest} with byte_size(Rest) > 0. The generated 'OTP-PUB-KEY':decode/2 path then keeps only the TLV (element(1, ber_decode_nif(Data))).

This matches the ASN.1 compiler's documented default (undec_rest is off, so trailing bytes are discarded). I am reporting it at the public_key layer because those APIs are documented as decoding a DER-encoded certificate / entity, return only the decoded object, and give the caller no leftover length to check.

This is not a crash, and I am not claiming a signature-forgery exploit or a demonstrated ssl bug.

To Reproduce

Save as trailing_cert.escript and run escript trailing_cert.escript:

#!/usr/bin/env escript
%%! -noshell

main(_) ->
    application:ensure_all_started(crypto),
    application:ensure_all_started(public_key),
    Cert = binary:decode_hex(
        <<"3082018C3081F6A003020102020100300D06092A864886F70D01010B0500"
          "30003020180F32303139303631393038353535395A170D33363034313230"
          "35333431325A300030819F300D06092A864886F70D010101050003818D00"
          "30818902818100C59DDB82F9BF59A7272E6DC66E5A507D455677AA211BE0"
          "A77C6C6EF67BDA8E1DB8B62F37B6A3BF0619D87F36857B36051FA39FC186"
          "94A60B6D056A4C3C2252F7F34C95AD86EE15ADBAA0A03A584B18342A8990"
          "98F69C5A791AFB4291C19AE1D0AB857AC3EEFBBCA92E561F4CDBD8DAB92A"
          "7F4EEF05BAC19C54F37832A65517690203010001A3153013301106096086"
          "480186F84201010404030206C0300D06092A864886F70D01010B05000381"
          "8100B1F6EA202E7D8D1AACEB0720C0E4E34DA4CEDC456A3432544AF03B33"
          "4048C44E812236F01EFE5245C0963533F3BAADFC2B0AAB1436290D088D6C"
          "381E1CCB38E3A41FEBEC29C61DF27977254D6DCF490FF0FA6DDD772F63E8"
          "4116247F2084C0BF2AA3540E952DBFCFFD3AC9BEA4F9CBAFDD9FC422CD72"
          "B603B813FBAE28D86201">>),
    Trailing = <<Cert/binary, 0>>,
    io:format("OTP ~s  public_key ~s~n",
              [erlang:system_info(otp_release), code:lib_dir(public_key)]),
    check("valid certificate", Cert),
    check("same certificate plus one trailing 00 byte", Trailing),
    halt(0).

check(Label, Bin) ->
    {_, Rest} = asn1rt_nif:decode_ber_tlv(Bin),
    io:format("~n== ~s ==~nbytes=~p leftover=~p~n",
              [Label, byte_size(Bin), byte_size(Rest)]),
    lists:foreach(
      fun({Name, Fun}) ->
              case catch Fun() of
                  {'EXIT', Reason} ->
                      io:format("  REJECT ~s: ~p~n", [Name, Reason]);
                  _Value ->
                      io:format("  ACCEPT ~s~n", [Name])
              end
      end,
      [{"public_key:der_decode('Certificate', Bin)",
        fun() -> public_key:der_decode('Certificate', Bin) end},
       {"public_key:der_decode('OTPCertificate', Bin)",
        fun() -> public_key:der_decode('OTPCertificate', Bin) end},
       {"public_key:pkix_decode_cert(Bin, otp)",
        fun() -> public_key:pkix_decode_cert(Bin, otp) end},
       {"public_key:pkix_decode_cert(Bin, plain)",
        fun() -> public_key:pkix_decode_cert(Bin, plain) end}]).

Any other complete DER certificate plus <<0>> also reproduces it.

Observed output:

OTP 27  public_key /usr/lib/erlang/lib/public_key-1.17.1.1

== valid certificate ==
bytes=400 leftover=0
  ACCEPT public_key:der_decode('Certificate', Bin)
  ACCEPT public_key:der_decode('OTPCertificate', Bin)
  ACCEPT public_key:pkix_decode_cert(Bin, otp)
  ACCEPT public_key:pkix_decode_cert(Bin, plain)

== same certificate plus one trailing 00 byte ==
bytes=401 leftover=1
  ACCEPT public_key:der_decode('Certificate', Bin)
  ACCEPT public_key:der_decode('OTPCertificate', Bin)
  ACCEPT public_key:pkix_decode_cert(Bin, otp)
  ACCEPT public_key:pkix_decode_cert(Bin, plain)

The same four APIs also accept the certificate plus four trailing bytes (<<0,1,2,3>>).

Expected behavior

For APIs documented as decoding one DER-encoded certificate / entity, leftover bytes after the complete top-level object should be treated as an error, or at least returned so the caller can reject them.

Current docs:

• der_decode/2: “Decodes a public-key ASN.1 DER encoded entity.”
• pkix_decode_cert/2: “Decodes an ASN.1 DER-encoded PKIX certificate.”

https://www.erlang.org/doc/apps/public_key/public_key.html

If ignoring trailing bytes is intentional, the docs should say so, because these functions currently expose neither leftover bytes nor a consumed length.

Affected versions

• OTP 27.3.4.6 (erts-15.2.7.4)
• public_key 1.17.1.1
• asn1 5.3.4.2
• OS: Ubuntu 26.04 LTS, x86_64
• Install: distro Erlang package (no custom configure flags, no local source patches)

The generated BER decode dispatcher that discards leftover bytes is still present on current master:

https://github.com/erlang/otp/blob/master/lib/asn1/src/asn1ct_gen.erl

(element(1, ber_decode_nif(Data)) unless the module is compiled with undec_rest)

Additional context

Root cause, as I read it:

1. asn1rt_nif:decode_ber_tlv/1 returning {Tlv, Rest} is expected.
2. 'OTP-PUB-KEY':decode/2 is generated without undec_rest, so it does element(1, ber_decode_nif(Data)) and drops Rest.
3. pkix_decode_cert(Bin, plain) calls der_decode('Certificate', Bin).
4. pkix_decode_cert(Bin, otp) goes through pubkey_cert_records:decode_cert/1 → 'OTP-PUB-KEY':decode('OTPCertificate', ...).

This is the ASN.1 compiler default. From the undec_rest option in asn1ct:

By default when decoding, any bytes following the end of an ASN.1 data structure are discarded.

I would not treat that default as a bug in generic BER decode/2. The gap is that public_key's complete-object certificate APIs inherit it and hide leftover from the caller.

Related observation, not an exploit: pkix_decode_cert(Bin, otp) followed by pkix_encode('OTPCertificate', Cert, otp) produces a 400-byte encoding from both the 401-byte and 404-byte inputs (trailing bytes are dropped by re-encoding).

What this is not:

• Not a claim that asn1rt_nif:decode_ber_tlv/1 is wrong.
• Not a claim that every generated BER decode/2 must start rejecting leftover.
• Not a demonstrated signature forgery. pkix_verify/2 extracts the signed TBS from the decoded prefix; appending bytes after a valid certificate does not change that TBS in the tests above.
• Not a crash or DoS.
• I have not shown a broken ssl handshake.

Suggested change, either:

1. Reject non-empty leftover in der_decode/2 and pkix_decode_cert/2 (compile the relevant modules with undec_rest and error on Rest =/= <<>>, or check leftover via asn1rt_nif:decode_ber_tlv/1).
2. Or document that these APIs decode the first complete object and discard following bytes, and consider a strict flag / leftover return for callers that need a complete-buffer check.

I would prefer (1) for the certificate APIs.