Possible ways to implement multiple modules for pkcs11-provider

[Ayke]

Hello there,

I'm thinking forking (possibly contribute to this repo too if it's welcome) pkcs11-provider to implement a version that supports to have multiple modules for pkcs11-provider. In other words, I'm trying to make it work when both SoftHSM and softokn are active and connected to pkcs11-provider, so that user can choose which module to use in pkcs11-uri

Now after days of investigation, it seems to me that inside pkcs11-provider it is almost always assumed that within one P11PROV_CTX there's only one module, so one way to implement it is simply to break this assumption, create a list of modules, only load function lists when uri has been given, and fix every place where it assumes there's just one module.

I was wondering if this is too much of a change, so an alternative seems to be: have a list of P11PROV_CTX instead, and make this list type the new "provider context" (i.e. OpenSSL core will treat this new list type as provider context and give it to provider). In the meantime for EVP_ and other objects, still have original P11PROV_CTX references when needed.

Could you please let me know of your opinions to the implementation? Does any of those sound good? Thank you so much for your time and information in advance!!

Best,
Ayke

[simo5]

We definitely welcome the possibility of loading multiple modules, it hasn't been done so far only because it wasn't needed by our immediate needs, but it is something we've been wanting to look into.

You can easily add an array of P11PROV_MODULE structures in p11prov_ctx, but if different configurations or different quirks are needed that becomes a little more complicated.

Searching for keys can also lead to issues if there is no pkcs11-uri or the provided pkcs11-uri does not contain identifiers needed to selects a single token.
Ordering of the modules would be required and try/retry operations sprinkled in several places, with error caching an management.

Another option we have been contemplated has been to make it possible to load multiple pkcs11-provider instances (one per pkcs11 token) so that they can work in parallel, that could potentially require some small changes in openssl, however using entirely different configurations would be much simpler to manage that way and would require a lot less changes to the code structure.

[Ayke]

Thank you for the info Simo!!

For OpenSSL to load entirely different configurations, please correct me if I'm wrong, it seems problematic to me as in OpenSSL these two "entirely different configured " providers are indistinguishable. Because the items they register are identical (e.g. for encode/decoder their type, format etc. they are the same). If we are to implement such a solution, as I understand it, we'll need to implement sort of an equivalent to a new "attribute" during registration to openssl that is settable from pkcs11-provider config? So that in openssl operation they can tell the difference between one pkcs11-provider instance from another?

In the meantime, for the "array of P11PROV_MODULE" approach, if just in my local setup it's guaranteed that the key retrieval will come with a pkcs11-uri with module info, does that become a valid "locally acceptable" solution?

[simo5]

The idea would be that you have entirely different sections in openssl, one per module.

In terms of types, decoders are fall-through in openssl, so upon parsing the pkcs11-uri in one of the decoders you would be able to return an error that signifies roughly: "I do not understand this type, actually" and fallback to the next provider that does (at least in theory).

However this would require strict identification of what token the URI refers to by always making sure of embedding the token label or serial in the URI, otherwise the buck would stop at the first module, as in absence of label/serial a pkcs11 provider would have to search internally for the object, and you probably do not want to do that at decoding time for various reasons.

[simo5]

However if you want to pursue the multiple tokens in a single provider approach, you need to define how the code will behave if there is no token info in the URI.
For a first implementation it may be ok to state that lacking token info in the URI the code will only search the first token that was registered.

Note that you will also have to define what happens in terms of session objects (as in, which token will handle them), which we have to create from time to time in order to complete some operations. For some things it may be trickier than you think.

[beldmit]

I believe you still can specify different names for several instances of the same OpenSSL providers, but it definitely worth testing, may require different dynamic paths (copies of .so), and even patching OpenSSL itself (I think, it would be approvable)

[Ayke]

I believe you still can specify different names for several instances of the same OpenSSL providers, but it definitely worth testing, may require different dynamic paths (copies of .so), and even patching OpenSSL itself (I think, it would be approvable)

Yes thanks for the help!! We currently can have multiple instances of the same provider, the trouble is just without any modification (currently) OpenSSL function calls wouldn't differ these different instances, so function calls that are meant for one instance may get run in the other instance.

[Jakuje]

Would using of p11-kit-proxy solve your issue for you? It loads any amount of pkcs11 modules and presents them as one for the application, in this case pkcs11-provider.

[Ayke]

Wow that sounds just the thing I need! I'll investigate and try it out! Thank you so much for the info!!!

[Jakuje]

FYI I added the page https://github.com/latchset/pkcs11-provider/wiki/Using-multiple-pkcs11%E2%80%90modules-or-tokens

Let me know if this worked for you, you had some trouble use it or if there is some more information you would like to find on such wiki page (or just edit it as it is wiki).

[Ayke]

Thanks a lot!! I'll help with the wiki too XD