feat(detector): flag BLS12-381 proof points reaching a pairing without a subgroup check

[Alheriii]

Closes #629

Summary

Adds bls_subgroup_check (SANCT_BLS_SUBGROUP_UNCHECKED): flags BLS12-381 pairing usage where a proof point reaches the pairing call with nothing having established subgroup or curve membership.

Why it matters

The pairing e: G1 × G2 → GT is defined on the prime-order subgroups, not the full curve. BLS12-381 has a cofactor in both groups — h₁ ≈ 2⁶⁴ in G1, h₂ ≈ 2³¹⁸ in G2 — so the curve holds plenty of points outside the subgroup a proof system reasons about. Pairing one of them evaluates the map outside the domain the soundness argument covers:

Malleability. Given a valid proof point P, an attacker can often produce P + T for a small-order T such that the verification equation still holds. The proof serializes differently but verifies identically — so any replay defence keyed on the proof bytes (a used-proof set, a nullifier derived from the encoding) is bypassed with a fresh-looking submission.

Forgery. Off-subgroup, the algebraic relations the verifier checks no longer pin the witness down. This is the mechanism behind the small-subgroup attacks that made subgroup checks mandatory in the CFRG BLS signature standard, and the same reasoning carries to any pairing-based verifier.

The check costs a scalar multiplication per point. That is precisely why implementations reach for the _unchecked variants, and why the bug survives review — skipping it is safe only for points the contract produced itself, never for anything a caller supplied.

What it reports

Two shapes, both putting an unvalidated point in front of a pairing:

1. _unchecked constructors and deserializers — deserialize_uncompressed_unchecked, from_compressed_unchecked, new_unchecked, … In arkworks these are exactly the entry points that skip the subgroup check.
2. A point arriving already typed — G1Affine, G2Affine, G1Projective, G2Projective as a parameter. Deserialization happened elsewhere, so nothing in this function establishes membership.

False positives

Held down three ways:

• File-level gate before parsing. The rule returns early unless the file actually uses BLS12-381, so an _unchecked constructor in unrelated code is invisible to it.
• A pairing must be reached. pairing, miller_loop, final_exponentiation or verify — deserializing a point for storage makes no verification claim and is not flagged.
• Any membership check silences it. Including the correct arkworks idiom: unchecked deserialization followed by an explicit is_in_correct_subgroup_assuming_on_curve(). clear_cofactor() / mul_by_cofactor() count too, since mapping into the subgroup is a valid remedy where rejecting is not appropriate.

#[cfg(test)] modules are skipped and inline // sanctifier:ignore[SANCT_BLS_SUBGROUP_UNCHECKED] is honoured, matching the surrounding rules.

Documentation

docs/detectors/bls_subgroup_check.md covers the cofactor figures, both attack consequences, the vulnerable shape, and the fix — including that both conditions are needed, since is_in_correct_subgroup_assuming_on_curve assumes on-curve, as the name says.

References, per the acceptance criterion:

• CFRG BLS signatures draft §5.2 — KeyValidate / subgroup validation
• Sean Bowe — BLS12-381: New zk-SNARK Elliptic Curve Construction — cofactor structure of G1 and G2
• 0xPARC ZK Bug Tracker — missing curve/subgroup checks
• Trail of Bits — Breaking the Shield — proof malleability from unchecked group elements
• arkworks Validate::Yes vs Validate::No — what the _unchecked variants actually skip

Rows added to the detector catalog and the finding-code table — tests/detector_docs_coverage.rs enforces both.

Acceptance criteria

• Flags missing subgroup checks — both the unchecked-deserialization and caller-supplied-point shapes
• Documented with references

Tests

7 unit tests: both flagged shapes, both accepted check forms, the non-BLS and never-pairs cases, and suppression. cargo fmt --all -- --check clean, cargo clippy -p sanctifier-core --all-targets clean, cargo test -p sanctifier-core passing.

Pre-existing failures, unrelated to this PR: memory::tests::memory_guard_rejects_above_zero_limit and memory::tests::memory_tracker_samples_increase_peak fail on an unmodified checkout of main on macOS (memory sampling). Verified by stashing this change and re-running — same two failures.

Local runs used --no-default-features because the default smt feature needs z3.h present; that is a local toolchain gap, not a change here.

Conflicts

This and #946 (for #628) share four files by necessity — the detector registry (rules/mod.rs), finding_codes.rs, and the two docs tables. Rather than both appending to the same place, each PR's insertions are anchored far apart in every shared file: this one registers after edge_amount and inserts near BALANCE_EQUALITY; the other registers after vk_provenance and inserts near PROOF_LENGTH_UNVALIDATED, tens of lines away in each case, so no two hunks share context.

Both merge orders were tested locally against main — clean either way, with fmt, clippy, the docs-coverage test and 294 passing tests on the combined result.

[definurse13]

Force-pushed a fix for the failing CI.

differential_test::rule_to_code_map_covers_all_default_rules requires every rule in the default registry to appear in sanctifier_rule_to_code in tests/fixtures/corpus/differential-corpus.json, and I had only covered the other three registration touchpoints (rules/mod.rs, finding_codes.rs, and the two docs tables). Added:

"bls_subgroup_check": "SANCT_BLS_SUBGROUP_UNCHECKED",
"SANCT_BLS_SUBGROUP_UNCHECKED": "SANCT_BLS_SUBGROUP_UNCHECKED"

All 13 test targets pass locally now, including differential_test and detector_docs_coverage. I had missed it because cargo test -p sanctifier-core stops at the first failing target and the lib target fails on macOS with two pre-existing memory::tests failures, so the integration targets never ran — --no-fail-fast surfaces them.

The corpus entry is anchored next to balance_equality, ~50 lines from where #946 adds its entry next to vk_provenance, so this does not become a new conflict point between the two PRs. Both merge orders re-verified clean.

The workflow runs on the new commits are sitting at action_required and need a maintainer to approve them.