Skip to content

crypto/bn256/google: canonicalize coordinates before testing them - #35559

Open
4waan wants to merge 4 commits into
ethereum:masterfrom
4waan:bn256-google-canonical-coordinates
Open

crypto/bn256/google: canonicalize coordinates before testing them#35559
4waan wants to merge 4 commits into
ethereum:masterfrom
4waan:bn256-google-canonical-coordinates

Conversation

@4waan

@4waan 4waan commented Aug 20, 2026

Copy link
Copy Markdown

big.Int coordinates in the google backend are lazily reduced.. Add and Double write values outside [0, P) and Negative writes a negative yEach commit makes each one of 4 predicates (that read those values as if they were canoncial) agree with the representation it is actually handed.

MakeAffine inverted an unreduced z. ModInverse returns nil when no inverse exists. On the curve that nil went straight into big.Int.Mul and panicked; on the twist the same z has a zero norm, so Invert returned zero and the point at infinity silently became the affine point 0 : 0 : 1. Reducing z first removes the case instead of detecting it: after reduction z ∈ [1, P) at the inversion, coprime to prime P. That holds for any input, including a caller who mutates the big.Ints G1.CurvePoints hands out, which a nil check would only catch after the fact.

MakeAffine returned non-canonical coordinates. The z == 1 path returned without touching x and y, so G1.Neg of an affine point produced a representation MakeAffine would not normalize, and comparing two mathematically equal points by their coordinates was unsound. Fixing it at MakeAffine rather than at Negative covers Add and Double as well. That path now also writes t ( twistPoint.Add never sets it so a point arriving already affine can carry a stale t) and the Miller loop reads t.

gfP2.IsOne tested y through Bits which reports the magnitude so -1 read as 1.

gfP2.Invert discarded the ModInverse result. This one was correct by accident, and the commit says so rather than claiming a bug... the inverse fails only when the norm x²+y² is zero, and P ≡ 3 mod 4 makes -1 a non-residue, so a zero norm forces both components of a to zero and the multiplications wipe the stale value either way. Returning zero explicitly stops that resting on a coincidence, and matches what cloudflare's exponentiation-based inverse and gnark produce for 0⁻¹.

Testing

canonical_test.go adds five tests. Four fail against current master, checked per test in isolation rather than as a batch; TestGFp2InvertZero passes both ways and is there to pin the 0⁻¹ = 0 convention against a future refactor of Invert.

(independent of #38885 and branched from master) that one fixes reachable aliasing in Double, this one touches MakeAffine, IsOne and Invert.. two can be in review in parallel

4waan added 4 commits August 21, 2026 01:00
MakeAffine reads z three ways that all assume a canonical value: Bits
reports the magnitude, IsInfinity only recognises an exact zero, and
ModInverse returns nil for a z that is a non-zero multiple of P. Add and
Double leave z unreduced, so reduce it first. On the curve that nil went
straight into big.Int.Mul and panicked; on the twist such a z has a zero
norm, so Invert returns zero and the point at infinity silently became
the affine point 0 : 0 : 1.

Unreachable today: Add reduces z and Double doubles a reduced value, so
z stays in [0, 2P-2] and even, and z = P is ruled out because P is odd.
Reducing holds regardless, including for a caller that mutates the
big.Ints G1.CurvePoints hands out. No consensus impact either way, since
crypto/bn256 resolves to gnark on amd64/arm64.
The z == 1 path returned without touching x and y, which the group law
leaves as unreduced as z was: Add and Double do not reduce them and
Negative leaves y negative. G1.Neg of an affine point therefore produced
a representation MakeAffine would not normalize, so comparing two
mathematically equal points by their coordinates was unsound. Reduce
them on the way out, which covers every source rather than just Negative.

Set t as well. twistPoint.Add never writes it, so a point that reaches
MakeAffine already affine can carry a stale t, and the Miller loop does
read t. curvePoint.t is never read and is set for consistency.
IsOne tests y through big.Int.Bits, which reports the magnitude, so -1
read as 1. Both callers pass reduced values today, gfP12.IsOne because
it calls Minimal first and twistPoint.MakeAffine because it now reduces
z, but a false IsOne on a twist z skips normalization and a false IsOne
in GT decides PairingCheck. Make the test itself sound.
Invert discarded the return value, so on failure inv kept whatever the
pool last held. The result was still correct by accident: ModInverse
fails only when the norm x²+y² is zero, and P ≡ 3 mod 4 makes -1 a
non-residue, so a zero norm forces both components of a to zero and the
multiplications below wipe inv either way. Return zero explicitly rather
than resting on that. Zero is also what cloudflare's exponentiation-based
inverse and gnark produce for 0⁻¹, so the three backends agree.
@4waan
4waan requested review from fjl and gballet as code owners August 20, 2026 19:52
@4waan
4waan force-pushed the bn256-google-canonical-coordinates branch from d842634 to 2f512c7 Compare August 21, 2026 05:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant