crypto: `dh.c` fails to compile on macOS arm64 due to `ErlNifUInt64`/`uint64_t` pointer mismatch

[ruslandoga]

👋

Describe the bug

Building Erlang/OTP 27.3.4.6 from source on macOS arm64 fails while compiling lib/crypto/c_src/dh.c.

dh_generate_key_nif() declares len as ErlNifUInt64, then passes &len to OSSL_PARAM_construct_uint64(), which expects a uint64_t *.

On macOS arm64, Apple Clang resolves these as different pointer types:

• ErlNifUInt64 * → unsigned long *
• uint64_t * → unsigned long long *

The compiler emits an -Wincompatible-pointer-types diagnostic as an error, aborting the build.

The same mismatch is visible on current master:

• ErlNifUInt64 len
• OSSL_PARAM_construct_uint64("priv_len", &len)

OpenSSL declares the second argument as uint64_t *:

• OSSL_PARAM_construct_uint64

To Reproduce

On an Apple Silicon Mac:

MISE_ERLANG_COMPILE=true \
mise install --force --verbose erlang@27.3.4.6

mise invokes kerl 4.4.0 to build OTP from source.

Compilation then fails with:

dh.c:93:63: error: incompatible pointer types passing 'ErlNifUInt64 *' (aka 'unsigned long *') to parameter of type 'uint64_t *' (aka 'unsigned long long *') [-Wincompatible-pointer-types]
   93 |         params[i++] = OSSL_PARAM_construct_uint64("priv_len", &len);
      |                                                               ^~~~
/opt/homebrew/opt/openssl/include/openssl/params.h:82:67: note: passing argument to parameter 'buf' here
   82 | OSSL_PARAM OSSL_PARAM_construct_uint64(const char *key, uint64_t *buf);
      |                                                                   ^
1 error generated.

Environment:

macOS:          15.7.7 (24G720)
Architecture:   arm64
Apple Clang:    17.0.0 (clang-1700.6.3.2)
Clang target:   arm64-apple-darwin24.6.0
OpenSSL:        3.6.3 from Homebrew
mise:           2026.8.6
kerl:           4.4.0

Expected behavior

Erlang/OTP should compile successfully without suppressing incompatible-pointer-type diagnostics.

Affected versions

Reproduced with Erlang/OTP 27.3.4.6.

Source inspection shows the same ErlNifUInt64 * to uint64_t * mismatch in the current maint-27, maint-28, maint, and master branches.

Additional context

The two types have the same size on this macOS arm64, but they are distinct C types and their pointer types are not compatible.

A local workaround is to downgrade this diagnostic during the source build:

CFLAGS="-O2 -g -Wno-error=incompatible-pointer-types"

A type-safe fix could retain ErlNifUInt64 for enif_get_uint64(), copy the value into a separate uint64_t variable, and pass the address of that variable to OSSL_PARAM_construct_uint64(). This would also preserve the Windows behavior addressed by commit ea0acb49c00d.