Perturbing/add msm bls

[perturbing]

Description

✨ PR: Add and Test Two MSM Implementations for BLS12-381

Summary

This PR introduces multi-scalar multiplication (MSM) for BLS12-381 elliptic curve points:

---

Changes

⚙️ Internal Module (BLS12_381/Internal.hs)

• Added types for projective/affine points and scalars, including array/block representations
• Added FFI bindings for 6 C functions:
  • blst_{p1s,p2s}_mult_pippenger, blst_{p1s,p2s}_to_affine, and c_blst_{p1s,p2s}_mult_pippenger_scratch_sizeof
• Added helper marshalling functions:
  • withPointArray, withScalarArray, withAffineBlockArrayPtr
• Implemented blsMSM with filtering of points-at-infinity and zero scalars for safety and efficiency

✅ Tests (Test/EllipticCurve.hs)

• Added property tests comparing blsMSM against naive implementations
• Fixed prop_randomFailsFinalVerify to use group-based logic instead of point inequality
• Updated Arbitrary instance for Point
• Increased probability of generating blsZero for better edge-case coverage

Checklist

• Commit sequence broadly makes sense and commits have useful messages
• New tests are added if needed and existing tests are updated
• All visible changes are prepended to the latest section of a CHANGELOG.md for the affected packages.
  New section is never added with the code changes. (See RELEASING.md)
• When applicable, versions are updated in .cabal and CHANGELOG.md files according to the
  versioning process.
• The version bounds in .cabal files for all affected packages are updated.
  If you change the bounds in a cabal file, that package itself must have a version increase. (See RELEASING.md)
• Commits that only contain large amounts of formatting changes were added to .git-blame-ignore-revs
• Self-reviewed the diff

[lehins]

Haven't looked into ffi calls, just at the haskell portion of it.
I'll look into this in more depth once PR is out of draft

[perturbing]

Thank you for taking a first look @lehins, much appreciated!

The FFI is not working yet, I am getting some segmentation faults that I am trying to debug with valgrind.

[perturbing]

Hi, I fixed the bug I encountered with the memory layout (I overlooked how C code wanted the pointer).

I also added a property test, but got some weird behavior where my test's success depends on the running of other tests. See my comments here and here.

[lehins]

I have quickly looked through the PR, but haven't spotted exactly what is wrong with this PR, but there is definitely something seriously wrong with memory management in this PR. All the non-deterministic test failures, which should not be the case with functionality that pretends to be "pure" serve as a good indicator that the functionality has a bug. Moreover, the fact that unrelated tests are affected is another strong indicator that something is seriously wrong!

I'll try to dig deeper into this functionality some time this week. Maybe I can help you get to the bottom of this.

[perturbing]

Hi, I am ready for another round of review (see also the PR description for an overview of the changes).

Thanks in advance, much appreciated.

I think it's best to not yet merge this PR, best to first let @kwxm implement it in plutus via an SRP stanza.

[tdammers]

Looks mostly good to me.

[tdammers]

Why the null terminator? AFAICT, all code that consumes this API uses an explicit length to determine the end of the array; the null terminator is never used.

[perturbing]

Initially, I had it without, but the C code does use it (I discovered this by looking at the Rust implementation of the blst bindings here).

And to test it without, I ran this CI test, which failed on all windows machines.

[perturbing]

And to add to that, there are more things that are implicit in for the blst lib that I discovered. For example, for windows builds, given one pair for an input (so one point + scalar), the msm call will also fail. Which Is why I added this

    [(scalar, pt)] -> do
      i <- scalarToInteger scalar
      return (blsMult pt i)

And to test it, see this and this failure on all windows tests.

[lehins]

That is pretty surprising to see null termination. However, unlike usage of null termination for binary data or strings using, for an array of pointers I think it is fairly safe. What BLST designers should have done, is to produce some sort of error when number of pointers does not match where null termination is placed, since both carry the same information and there is a chance for a discrepancy.

It would be good to learn the expected semantics here. For example this would sound sensible, but I can't find any information on it:

• when there is a null pointer in a position that resulted in less points than was expected that it is safe, since one of those points could contain a null pointer
• when number of points (numScalars) supplied is reached and the next element in the array is not a null pointer than it is an error.

On a separate subject same comment as I made on withPointArray applies here too

[lehins]

I had some questions and minor suggestions, but other than that it is looking good.

Needs a rebase on master. Also could you please add all visible changes to the Changelog and bump minor version in both cabal file and in the latest section in the changlog file to 2.2.3.0.

[lehins]

Slight simplification that avoids redundant iteration over the list and plays better with list fusion than zip does.

Suggested change

	let accumulate [] = do
	poke (ptr `advancePtr` numPoints) nullPtr
	go numPoints (PointArrayPtr (castPtr ptr))
	accumulate ((ix, point) : rest) =
	withPoint point $ \(PointPtr pPtr) -> do
	poke (ptr `advancePtr` ix) pPtr
	accumulate rest
	in accumulate (zip [0 ..] points)
	let accumulate curPtr [] = do
	poke curPtr nullPtr
	go numPoints (PointArrayPtr (castPtr ptr))
	accumulate curPtr (point : rest) =
	withPoint point $ \(PointPtr pPtr) -> do
	poke curPtr pPtr
	accumulate (curPtr `plusPtr` sizeReference) rest
	in accumulate ptr points

Nice comment about zipWithM_ not being a valid abstraction here! 👍

[lehins]

That is pretty surprising to see null termination. However, unlike usage of null termination for binary data or strings using, for an array of pointers I think it is fairly safe. What BLST designers should have done, is to produce some sort of error when number of pointers does not match where null termination is placed, since both carry the same information and there is a chance for a discrepancy.

It would be good to learn the expected semantics here. For example this would sound sensible, but I can't find any information on it:

• when there is a null pointer in a position that resulted in less points than was expected that it is safe, since one of those points could contain a null pointer
• when number of points (numScalars) supplied is reached and the next element in the array is not a null pointer than it is an error.

On a separate subject same comment as I made on withPointArray applies here too

[lehins]

I'd recommend avoiding construction of scalar if it is top be discarded:

Suggested change

	scalar <- scalarFromInteger s
	-- Here we filter out pairs that will not contribute to the result.
	-- This is also for safety, as the c_blst_to_affines C call
	-- will fail if the input contains the point at infinity.
	-- see https://github.com/supranational/blst/blob/165ec77634495175aefd045a48d3469af6950ea4/src/multi_scalar.c#L11C32-L11C37
	-- We also filter out the zero scalar, as for any point pt
	-- we have:
	--
	-- pt ^ 0 = id
	--
	-- Which yields no contribution to summation, and
	-- thus we can skip the point and scalar pair. This filter
	-- saves us an extra input to the more expensive exponential
	-- operation.
	if blsIsInf pt || scalar == zeroScalar
	then return acc
	else return ((scalar, pt) : acc)
	-- Here we filter out pairs that will not contribute to the result.
	-- This is also for safety, as the c_blst_to_affines C call
	-- will fail if the input contains the point at infinity.
	-- see https://github.com/supranational/blst/blob/165ec77634495175aefd045a48d3469af6950ea4/src/multi_scalar.c#L11C32-L11C37
	if blsIsInf pt
	then pure acc
	else do
	scalar <- scalarFromInteger s
	-- We also filter out the zero scalar, as for any point pt
	-- we have:
	--
	-- pt ^ 0 = id
	--
	-- Which yields no contribution to summation, and
	-- thus we can skip the point and scalar pair. This filter
	-- saves us an extra input to the more expensive exponential
	-- operation.
	if scalar == zeroScalar
	then return acc
	else return ((scalar, pt) : acc)

Also, in a one-to-one conversion we could avoid construction of zeroScalar, by checking s == 0 instead, but I suspect because of modular arithmetic there could be other values of Integer that would map to scalarFromInteger 0. If I am wrong and there are no other values that map to 0 except 0, then I'd suggest this instead:

I'd recommend avoiding construction of scalar if it is top be discarded:

Suggested change

	scalar <- scalarFromInteger s
	-- Here we filter out pairs that will not contribute to the result.
	-- This is also for safety, as the c_blst_to_affines C call
	-- will fail if the input contains the point at infinity.
	-- see https://github.com/supranational/blst/blob/165ec77634495175aefd045a48d3469af6950ea4/src/multi_scalar.c#L11C32-L11C37
	-- We also filter out the zero scalar, as for any point pt
	-- we have:
	--
	-- pt ^ 0 = id
	--
	-- Which yields no contribution to summation, and
	-- thus we can skip the point and scalar pair. This filter
	-- saves us an extra input to the more expensive exponential
	-- operation.
	if blsIsInf pt || scalar == zeroScalar
	then return acc
	else return ((scalar, pt) : acc)
	-- Here we filter out pairs that will not contribute to the result.
	-- This is also for safety, as the c_blst_to_affines C call
	-- will fail if the input contains the point at infinity.
	-- see https://github.com/supranational/blst/blob/165ec77634495175aefd045a48d3469af6950ea4/src/multi_scalar.c#L11C32-L11C37
	-- We also filter out the zero scalar, as for any point pt
	-- we have:
	--
	-- pt ^ 0 = id
	--
	-- Which yields no contribution to summation, and
	-- thus we can skip the point and scalar pair. This filter
	-- saves us an extra input to the more expensive exponential
	-- operation.
	if blsIsInf pt || s == 0
	then pure acc
	else do
	scalar <- scalarFromInteger s
	pure ((scalar, pt) : acc)

[lehins]

I suggest having a list of tuples supplied, instead of two separate lists, because there are no surprises when these two lists have different length. (i.e. burden of correct zipping is put onto the caller of the function)

Suggested change

	blsMSM :: forall curve. BLS curve => Int -> [Integer] -> [Point curve] -> Point curve
	blsMSM :: forall curve. BLS curve => Int -> [(Integer, Point curve)] -> Point curve

[lehins]

foldM is a left foldlM, so this operation will reverse the order when compared to the input.

> foldM (\acc x -> (x:acc) <$ print x) [] [1,2,3,4]
1
2
3
4
[4,3,2,1]

My question is this indeed desired? If it isn't I'd suggest switching to foldrM

Considering that tests successfully check against the same naive implementation, I suspect that it all works out only because MSM operation is commutative. I am not sure if it is dangerous to rely on this property, but at the very least it surprising to see the order of input to be reversed.

[lehins]

You don't want to generate two lists of different length just to immediately discard elements from the one that is longer.

Suggested change

	, testProperty "MSM matches naive approach" $ \((ps, ss) :: ([BLS.Point curve], [BigInteger])) ->
	let pairs = [(p, i) | (BigInteger i, p) <- zip ss ps]
	, testProperty "MSM matches naive approach" $ \(pairs' :: [(BLS.Point curve, BigInteger)]) ->
	let pairs = [(p, i) | (BigInteger i, p) <- pairs']

[lehins]

TIL there is a word in English refert 😄

[lehins]

This will give an idea of how well both cases of blsMSM function are tested.

Suggested change

	(ps', ss') = unzip pairs
	in BLS.blsMSM 10 ss' ps'
	(ps', ss') = unzip pairs
	threshold = 10
	in classify (length pairs <= threshold) "Below threshold" $
	BLS.blsMSM threshold ss' ps'

[lehins]

Redundant newline

Suggested change