BLS12-381 host functions #156
Conversation
|
We should avoid doing the hostcalls for specific signature schemes like this anymore. Instead, we should do four hostcalls for the underlying slow operations, namely scalar multiplications, multi-scalar multiplications, Miller loops, and final exponentiations. There are no real standards for BLS signatures, so this allows us to support whatever form someone deploys, and also support most snark verifiers, all with only 4 host calls per curve. We'd expose these through the forked sp-ark-[curve] crates, so then your regular rust code would work in the runtime but accelerated. We discuss the dsign rational in https://hackmd.io/@rgbPIkIdTwSICPuAq67Jbw/rkipBanIn We'd eventually add quite a few curves this way: Very standard curves: Several pairing friendly curves: Some jubjub curves for those, maybe some sister curves later. Two cycle curves: secq256k1, ark_grumpkin -- 2 hostcalls each Also maybe some NIST curves: ark-secp256r1 -- 2 hostcalls, requested Additional work RFPs: https://hackmd.io/@rgbPIkIdTwSICPuAq67Jbw/r1dRCb86C |
|
While I strongly agree with the idea of introducing ark-related host calls for computationally heavy operations (essentially using ark-extensions), this RFC focuses on generating and using secrets from the keystore. That cannot be done within the runtime, so dedicated calls are required. The ark-extensions work is mostly about optimizing verification operations, which involve only public keys and do not require keystore access. We could consider introducing a generic call with a parameter to select the scheme, but in any case, this follows the same format used by the other schemes. Also, I am not entirely sure we need dedicated calls for the ECDSA-BLS hybrid scheme. I suppose the runtime could simply call into the host twice, once for each scheme, and then combine the results? |
|
@burdges I cannot say I understand what you're talking about as I'm not familiar with the problem space at all, I'm just specifying interfaces here, but I do understand what "4 hostcalls each" means ;) That's 4 allocations of output buffers on the runtime side, 4 times applying marshaling strategies to every argument (they are not no-op unless the arguments are primitives), and 4 times copying over the buffers here and there. If we can generalize the whole thing, disassemble it into Lego pieces, and create chained solutions out of those pieces, that's great news, but that should be done on the host side anyway to avoid FFI overheads. @davxy as for the ECDSA-BLS, I saw @drskalman implemented the ECDSA-BLS key generation as a host function, and I did the same for the other interfaces. Again, if it can be decomposed into pieces, then the question is: how critical is the performance of this stuff, and does the performance boost justify increased interface complexity? If something is called once per block, it's probably okay to do two separate calls for it; if it's called often, it makes sense to keep it together at the host side. |
|
Alright, you cannot sign anything from block runtime, but this gets used by offchain workers? We've some mechanism that restricts such signing host calls to offchain workers? |
We don't have anything that restricts this. However, calling this while building a block would lead to indeterminism, so this isn't done. |
|
It's restricted to only the relay chain runtime then? |
Parachains could also use this off chain. But maybe I misunderstood you? |
|
I simply meant that parachains cannot invoke this in their PVF. Anyways my original comment was entirely about the verification, not about signing per se. There are many reasons not to expose secrets to off-chain workers, if avoidable, so no objections here. |
|
I'm confused on why we need anything besides these three functions:
We are not using anything else currently in the current code. I understand that we can add more host functions for efficiency, but I don't think we want to go ahead of ourselves especially that we are going to be stuck with them for the rest of our lives. For example if the functions that @burdges suggested goes through then we don't need really need verify function as the host. Moreover, we absolutely don't need So if I make sense, perhaps could we delete and drop all other functions from the RFC beside those three? |
|
@drskalman TBH, my knowledge of asymmetric cryptography ends somewhere around when OpenSSL was originally released back in the late 90s, so don't judge me too strictly. My understanding is that we're generating key pairs to sign something with the private key and later verify that with the public key, so how much sense does it make to have only key generation functions? Also, again, I'm not sure I understand in full what's happening here, but looks like something that at least verifies stuff: https://github.com/paritytech/polkadot-sdk/blob/80ab459c3ec346dd52add01406a8fc7ab28db07f/substrate/primitives/consensus/beefy/src/lib.rs#L167 |
|
@s0me0ne-unkn0wn indeed we sign with the BLS key, however not in the runtime nor as an offchain worker task (in contrast to for example we generate proof of possession in the runtime) so we never need the sign function as a host call. The signature verification is no-std so we can just run the entire verification code inside the runtime. As such we do not necessarily need a host function for verification either. Performing verification in a host call is more efficient but that very well could come from integrating ark_substrate host calls so we don't need one specific for BLS. |
I guess this only comes down to performance requirements, or?
Can you explain this a bit more, please? These ark-substrate host functions just speed up finite field operations or what?
Then let's just stay with that since it fixes your problem and we can do more BLS or FF host calls later for more performance, if needed. Seems to be mixup of two different topics. |
As verification is a pure functin that'll run fine in PVM or WASM, just very slowly.
Those speed up scalar multiplications and pairings, but leave miscelanious field operations in the runtime. This buys a huge speed up for any protocol using that curve, but with relatively few hostcalls. This PR is about something fairly unrelated: Actually using the BLS12-381 keys from the keystore in an off-chain worker. |
@s0me0ne-unkn0wn I agree with @drskalman . Let us focus only on what is strictly required (i.e. PoP generation via keystore for offchain work). We will reduce verification time by offloading it to the host in a more generic way, as proposed in #163 |
7 participants
Rendered
CC @drskalman @davxy