Cannot compile on MacOS #376
Replies: 21 comments 2 replies
|
Have you tried setting C_INCLUDE_PATH (and perhaps LIBRARY_PATH) environment variables? |
|
|
Only the "dynamic" feature allows you to dynamically build against a pre-compiled openssl version, if you exclude that feature, as you did, only a static build is produced, for which you need to provide the path to the sources in KRYOPTIC_OPENSSL_SOURCES, as the error clearly indicates. |
UpdateInvestigation showed that 143 │ * Check of sane values.
144 │ */
145 │ /* Can't go higher than the current version. */
146 │ # if OPENSSL_API_LEVEL > (OPENSSL_VERSION_MAJOR * 10000 + OPENSSL_VERSION_MINOR * 100)
147 │ # error "OPENSSL_API_COMPAT expresses an impossible API compatibility level (too high)"
148 │ # endif
149 │ /* OpenSSL will have no version 2.y.z */
150 │ # if OPENSSL_API_LEVEL < 30000 && OPENSSL_API_LEVEL >= 20000
151 │ # error "OPENSSL_API_COMPAT expresses an impossible API compatibility level (b/w 20K and 30K)"
152 │ # endif
153 │ /* Below 0.9.8 is unacceptably low */
154 │ # if OPENSSL_API_LEVEL < 908
155 │ # error "OPENSSL_API_COMPAT expresses an impossible API compatibility level (too low: 908)"
156 │ # endifand the error Any suggestion? (Note: setting explicitly via |
|
do you also have openssl system headers installed? |
Not sure I understand the question - certainly, I have OpenSSL headers installed in
I rather doubt that. MacOS has LibreSSL installed - but no headers and no user-linkable libraries, AFAIK. So, the main OpenSSL version is Macports-installed And |
|
I would suggest looking at how we build openssl and kryoptic in the pkcs11-provider CI: Specifically I think it is critical to point at the right PKG_CONFIG_PATH with the openssl pkgconfig files, which must point at the correct directories where headers and librarues are. If that doesn't work I can't help further as I am not a Mac OS user. |
|
Actually I got it to compile here locally by setting: I also had In my case openssl was installed in |
|
I don't know: Maybe |
|
OPENSSL_API_COMPAT is a red hering what oprnssl checks for is OPENSSL_API_LEVEL, which can be influenced by OPENSSL_API_COMPAT but also by OPENSSL_CONFIGURED_API I suggest running: this will tell if there is any system header mocking with this define (it is not an environment variable, so what is set in environment variables does not matter). |
|
I think the issue here is just that this code path doesn't use pkg-config to locate OpenSSL, but it should. PR coming up. |
|
With my proposed changes in #368: |
The problem is that Xcode on MacOS (stupidly) done away with Regardless: As you see, OpenSSL headers or library files are not installed/available by the system. Again, is it possible to make |
|
@neverpanic, with your patch (PR #368) |
|
@simo5 one question: I couldn't figure how to use |
|
See https://github.com/latchset/pkcs11-provider/blob/main/tests/setup.sh and https://github.com/latchset/pkcs11-provider/blob/main/tests/kryoptic-init.sh, that should get you started. |
Thanks. Partially on my way. It looks like I need to at least (a) edit |
No, see https://github.com/latchset/pkcs11-provider/blob/main/tests/kryoptic-init.sh#L19-L21.
For this particular script to work, yes. But you can also just set the few variables that kryoptic and pkcs11-provider actually need and skip the rest. For example, just creating the kryoptic.conf, setting |
MacOS dynamic libraries have extension
So, I have to have
First, MacPorts on MacOS does not create this directory. Second, I already have quite a few providers defined for OpenSSL, including |
I'm saying you don't need to use the script, just read, understand, and apply what you've learned from it.
It's best to have a config, yes. See https://github.com/latchset/kryoptic/blob/main/src/config.rs#L265-L285 for methods to avoid setting an environment variable explicitly. https://github.com/latchset/kryoptic/blob/main/src/config.rs#L228-L234 is the structure of the configuration; https://github.com/latchset/kryoptic/blob/main/src/config.rs#L49-L77 explains the structure of the slot object. I guess a pull request to add a more easily readable documentation format for that would be welcome.
I maintain the
You don't have to write the pkcs11-provider configuration into a file, you can also just set the |
Correct: My fault - no idea why it did not show up when I looked for it initially. I've added the following: then attempted to build and test
Basically, all I want to accomplish now is to add Although, once |
Just export $KRYOPTIC to point to your source tree of kryoptic, have either a release or debug version of it compiled, and fix
It has been available for a while from MacPorts, just install the |
Honestly, I'd rather not create a fork, and I don't know how to submit a PR without forking the repo. At least, I cannot create a branch here. Anyway, here's the patch for diff --git a/tests/kryoptic-init.sh b/tests/kryoptic-init.sh
index 6455d82..8cf669a 100755
--- a/tests/kryoptic-init.sh
+++ b/tests/kryoptic-init.sh
@@ -19,6 +19,8 @@ find_kryoptic() {
find_kryoptic \
"${KRYOPTIC}/target/debug/libkryoptic_pkcs11.so" \
"${KRYOPTIC}/target/release/libkryoptic_pkcs11.so" \
+ "${KRYOPTIC}/target/debug/libkryoptic_pkcs11.dylib" \
+ "${KRYOPTIC}/target/release/libkryoptic_pkcs11.dylib" \
/usr/local/lib/kryoptic/libkryoptic_pkcs11.so \
/usr/lib64/pkcs11/libkryoptic_pkcs11.so \
/usr/lib/pkcs11/libkryoptic_pkcs11.so \However, it does not seem to help much - now KRYOPTIC initialization fails: Also, does this help? $ tests/setup.sh kryoptic
+ source /Users/ur20980/src/pkcs11-provider/tests/helpers.sh
++ : .
+++ which openssl3
+++ true
++ OPENSSL=
++ OPENSSL=openssl
++ helper_emit=1
++ sed --version
++ grep -q 'GNU sed'
++ sed_inplace=("-i" "")
++ export sed_inplace
+ _KEYID_COUNTER=0
+ '[' 1 -ne 1 ']'
+ TOKENTYPE=kryoptic
+ SUPPORT_ED25519=1
+ SUPPORT_ED448=1
+ SUPPORT_X25519=1
+ SUPPORT_X448=1
+ SUPPORT_RSA_PKCS1_ENCRYPTION=1
+ SUPPORT_RSA_KEYGEN_PUBLIC_EXPONENT=1
+ SUPPORT_TLSFUZZER=1
+ SUPPORT_ALLOWED_MECHANISMS=0
+ SUPPORT_SYMMETRIC=1
++ opensc-tool -i
++ grep OpenSC
++ sed -e 's/OpenSC 0\.\([0-9]*\).*/\1/'
+ OPENSC_VERSION=26
+ [[ 26 -le 25 ]]
+ [[ '' = \1 ]]
++ cat /proc/sys/crypto/fips_enabled
cat: /proc/sys/crypto/fips_enabled: No such file or directory
+ [[ '' = \1 ]]
+ PINVALUE=0123456789ABCDEFFEDCBA9876543210
+ SUPPORT_SKEY=0
+ openssl skeyutl -h
+ SUPPORT_SKEY=1
+ TMPPDIR=./kryoptic
+ TOKDIR=./kryoptic/tokens
+ '[' -d ./kryoptic ']'
+ mkdir ./kryoptic
+ mkdir ./kryoptic/tokens
+ PINFILE=./kryoptic/pinfile.txt
+ echo 0123456789ABCDEFFEDCBA9876543210
+ '[' kryoptic == softhsm ']'
+ '[' kryoptic == softokn ']'
+ '[' kryoptic == kryoptic ']'
+ source /Users/ur20980/src/pkcs11-provider/tests/kryoptic-init.sh
++ title SECTION 'Searching for Kryoptic module'
++ case "$1" in
++ shift 1
++ echo '########################################'
########################################
++ echo '## Searching for Kryoptic module'
## Searching for Kryoptic module
++ echo ''
++ find_kryoptic /Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.so /Users/ur20980/src/kryoptic/target/release/libkryoptic_pkcs11.so /Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib /Users/ur20980/src/kryoptic/target/release/libkryoptic_pkcs11.dylib /usr/local/lib/kryoptic/libkryoptic_pkcs11.so /usr/lib64/pkcs11/libkryoptic_pkcs11.so /usr/lib/pkcs11/libkryoptic_pkcs11.so /usr/lib/x86_64-linux-gnu/kryoptic/libkryoptic_pkcs11.so
++ for _lib in '"$@"'
++ test -f /Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.so
++ for _lib in '"$@"'
++ test -f /Users/ur20980/src/kryoptic/target/release/libkryoptic_pkcs11.so
++ for _lib in '"$@"'
++ test -f /Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib
++ echo 'Using kryoptic path /Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib'
Using kryoptic path /Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib
++ P11LIB=/Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib
++ return
++ title LINE 'Creating Kryoptic database'
++ case "$1" in
++ shift 1
++ echo 'Creating Kryoptic database'
Creating Kryoptic database
++ cat
++ export KRYOPTIC_CONF=./kryoptic/tokens/kryoptic.conf
++ KRYOPTIC_CONF=./kryoptic/tokens/kryoptic.conf
++ export 'TOKENLABEL=Kryoptic Token'
++ TOKENLABEL='Kryoptic Token'
++ export TOKENLABELURI=Kryoptic%20Token
++ TOKENLABELURI=Kryoptic%20Token
++ pkcs11-tool --module /Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib --init-token --label 'Kryoptic Token' --so-pin 0123456789ABCDEFFEDCBA9876543210
Using slot 0 with a present token (0x0)
Token successfully initialized
++ pkcs11-tool --module /Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib --so-pin 0123456789ABCDEFFEDCBA9876543210 --login --login-type so --init-pin --pin 0123456789ABCDEFFEDCBA9876543210
Using slot 0 with a present token (0x0)
User PIN successfully initialized
++ export 'TOKENCONFIGVARS=export KRYOPTIC_CONF=./kryoptic/tokens/kryoptic.conf'
++ TOKENCONFIGVARS='export KRYOPTIC_CONF=./kryoptic/tokens/kryoptic.conf'
++ export TESTPORT=34000
++ TESTPORT=34000
++ export SUPPORT_ALLOWED_MECHANISMS=1
++ SUPPORT_ALLOWED_MECHANISMS=1
++ '[' -z '' ']'
++ export SUPPORT_ML_DSA=1
++ SUPPORT_ML_DSA=1
++ '[' -z '' ']'
++ export SUPPORT_ML_KEM=1
++ SUPPORT_ML_KEM=1
+ [[ '' = \1 ]]
+ [[ 1 = \1 ]]
+ [[ 1 = \0 ]]
+ SEEDFILE=./kryoptic/noisefile.bin
+ dd if=/dev/urandom of=./kryoptic/noisefile.bin bs=2048 count=1
+ RAND64FILE=./kryoptic/64krandom.bin
+ dd if=/dev/urandom of=./kryoptic/64krandom.bin bs=2048 count=32
+ P11DEFLOGIN=("--login" "--pin=${PINVALUE}")
+ title LINE 'Generate openssl config file'
+ case "$1" in
+ shift 1
+ echo 'Generate openssl config file'
Generate openssl config file
+ export PKCS11_PROVIDER_MODULE=/Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib
+ PKCS11_PROVIDER_MODULE=/Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib
+ export PKCS11_PROVIDER_DEBUG=file:./kryoptic/p11prov-debug.log
+ PKCS11_PROVIDER_DEBUG=file:./kryoptic/p11prov-debug.log
+ export OPENSSL_CONF=./kryoptic/openssl.cnf
+ OPENSSL_CONF=./kryoptic/openssl.cnf
+ sed -e 's|@libtoollibs@||g' -e 's|@testsblddir@|.|g' -e 's|@testsdir@|./kryoptic|g' -e 's|@SHARED_EXT@||g' -e 's|@PINFILE@|./kryoptic/pinfile.txt|g' -e 's|##TOKENOPTIONS||g' /Users/ur20980/src/pkcs11-provider/tests/openssl.cnf.in
+ SERIAL=0
+ title LINE 'Creating new Self Sign CA'
+ case "$1" in
+ shift 1
+ echo 'Creating new Self Sign CA'
Creating new Self Sign CA
+ get_next_keyid
+ local id_val=0
+ (( _KEYID_COUNTER+=1 ))
++ printf %04x 0
+ KEYID=0000
++ printf %%%02x%%%02x 0 0
+ URIKEYID=%00%00
+ CACRTN=caCert
+ ptool --keypairgen --key-type=RSA:2048 --id=0000 --label=caCert
+ CMDOPTS=(--module="${P11LIB}" --token-label="${TOKENLABEL}")
+ '[' -n --login ']'
+ CMDOPTS+=("${P11DEFLOGIN[@]}")
+ CMDOPTS+=("$@")
+ pkcs11-tool --module=/Users/ur20980/src/kryoptic/target/debug/libkryoptic_pkcs11.dylib '--token-label=Kryoptic Token' --login --pin=0123456789ABCDEFFEDCBA9876543210 --keypairgen --key-type=RSA:2048 --id=0000 --label=caCert
Key pair generated:
Private Key Object; RSA
label: caCert
ID: 0000
Usage: decrypt, sign
Access: sensitive, always sensitive, never extractable, local
Unique ID: 4e01613e-ff2b-4a73-838e-6db2db9315a4
uri: pkcs11:model=v1;manufacturer=Kryoptic%20Project;serial=005dcaa8f92ae805;token=Kryoptic%20Token;id=%00%00;object=caCert;type=private
Public Key Object; RSA 2048 bits
label: caCert
ID: 0000
Usage: encrypt, verify
Access: local
Unique ID: be39e9d5-3906-41ab-a903-7421d41c684f
uri: pkcs11:model=v1;manufacturer=Kryoptic%20Project;serial=005dcaa8f92ae805;token=Kryoptic%20Token;id=%00%00;object=caCert;type=public
+ crt_selfsign caCert Issuer 0000
+ LABEL=caCert
+ CN=Issuer
+ KEYID=0000
+ (( SERIAL+=1 ))
+ CERTSUBJ=/CN=Issuer/
+ SIGNKEY='pkcs11:object=caCert;token=Kryoptic%20Token;type=private'
+ OPENSSL_CMD='x509
-new -subj "/CN=Issuer/" -days 365 -set_serial "1"
-extensions v3_ca -extfile "./kryoptic/openssl.cnf"
-out "./kryoptic/caCert.crt" -outform DER
-signkey "pkcs11:object=caCert;token=Kryoptic%20Token;type=private"'
+ ossl 'x509
-new -subj "/CN=Issuer/" -days 365 -set_serial "1"
-extensions v3_ca -extfile "./kryoptic/openssl.cnf"
-out "./kryoptic/caCert.crt" -outform DER
-signkey "pkcs11:object=caCert;token=Kryoptic%20Token;type=private"'
+ helper_output=
+ [[ '' = \1 ]]
+ echo '# r x509
-new -subj "/CN=Issuer/" -days 365 -set_serial "1"
-extensions v3_ca -extfile "./kryoptic/openssl.cnf"
-out "./kryoptic/caCert.crt" -outform DER
-signkey "pkcs11:object=caCert;token=Kryoptic%20Token;type=private" '
+ echo ' openssl x509
-new -subj "/CN=Issuer/" -days 365 -set_serial "1"
-extensions v3_ca -extfile "./kryoptic/openssl.cnf"
-out "./kryoptic/caCert.crt" -outform DER
-signkey "pkcs11:object=caCert;token=Kryoptic%20Token;type=private" '
openssl x509
-new -subj "/CN=Issuer/" -days 365 -set_serial "1"
-extensions v3_ca -extfile "./kryoptic/openssl.cnf"
-out "./kryoptic/caCert.crt" -outform DER
-signkey "pkcs11:object=caCert;token=Kryoptic%20Token;type=private"
++ eval openssl x509 -new -subj '"/CN=Issuer/"' -days 365 -set_serial '"1"' -extensions v3_ca -extfile '"./kryoptic/openssl.cnf"' -out '"./kryoptic/caCert.crt"' -outform DER -signkey '"pkcs11:object=caCert;token=Kryoptic%20Token;type=private"'
+++ openssl x509 -new -subj /CN=Issuer/ -days 365 -set_serial 1 -extensions v3_ca -extfile ./kryoptic/openssl.cnf -out ./kryoptic/caCert.crt -outform DER -signkey 'pkcs11:object=caCert;token=Kryoptic%20Token;type=private'
Could not open file or uri for loading private key from pkcs11:object=caCert;token=Kryoptic%20Token;type=private
C0305C4EF87F0000:error:80000002:system library:file_open:No such file or directory:providers/implementations/storemgmt/file_store.c:264:calling stat(pkcs11:object=caCert;token=Kryoptic%20Token;type=private)
C0305C4EF87F0000:error:1608010C:STORE routines:inner_loader_fetch:unsupported:crypto/store/store_meth.c:363:No store loader found. For standard store loaders you need at least one of the default or base providers available. Did you forget to load them? Info: Global default library context, Scheme (pkcs11 : 0), Properties (<null>)
+ __out= |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Env
MacOS Sequoia 15.7.1, Xcode-26.1. OpenSSL-3.5.4 installed by Macports (this matters).
Problem
Fails to compile - unable to find OpenSSL header files:
OpenSSL headers live in
/opt/local/include/openssl/, libraries - in/opt/local/lib/(symlinked from/opt/local/libexec/openssl3/):There needs to be a way to point
kryopticbuild process at the location of the OpenSSL binary installation.Currently set env vars do that, but
kryopticdoes not look at them:$ env | grep OPENSSL OPENSSL_LIB_DIR=/opt/local/lib OPENSSL_INCLUDE_DIR=/opt/local/include OPENSSL_APP=/opt/local/libexec/openssl3/bin/openssl OPENSSL_MODULES=/opt/local/libexec/openssl3/lib/ossl-modules OPENSSL_CFLAGS=-O3 -std=gnu18 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/opt/local/include OPENSSL_ROOT_DIR=/opt/local/libexec/openssl3 OPENSSL_CONF=/opt/local/etc/openssl/openssl.cnf OPENSSL_CXXFLAGS=-std=gnu++20 -O3 -march=native -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/opt/local/include OPENSSL_DIR=/opt/local/libexec/openssl3 $All reactions