Skip to content

feat: let a consumer install the per-run hash seed - #119

Open
LukaszRozmej wants to merge 8 commits into
mainfrom
feat/seed-hashes
Open

feat: let a consumer install the per-run hash seed#119
LukaszRozmej wants to merge 8 commits into
mainfrom
feat/seed-hashes

Conversation

@LukaszRozmej

@LukaszRozmej LukaszRozmej commented Sep 6, 2026

Copy link
Copy Markdown
Member

Changes

The zkEVM build starts with fixed hash seeds and previously exposed no way for a guest to replace them. Add UInt256.SeedHashes(in UInt256 seed) to install a full-width 256-bit seed in both standard and zkEVM builds. Int256.GetHashCode delegates to UInt256 and follows the same seed.

  • Both builds replace their hash state when called. Standard builds initially draw cryptographic randomness per process; zkEVM builds initially use constants. Install the seed before populating hash-keyed containers: reseeding invalidates their stored hashes and is not synchronized with concurrent hashing.
  • Keep mutable seeds in the nested RunSeed type so UInt256's own statics remain immutable after initialization. Both builds cache the AES round keys there rather than rebuilding them per hash.
  • The AES path uses the two raw 128-bit seed halves across its rounds. The scalar path mixes all four 64-bit seed limbs into the key before widening multiplication and folding. There is no 32-bit seed expansion or XOR reduction of the seed halves.
  • Three properties of a bare low ^ high fold made collisions constructible rather than searchable once the seed is public, and all three are addressed: MultiplyFold carries its factors past the product (low ^ high ^ a ^ b), the scalar mixer folds each limb pair through MumFold rather than MultiplyFold, and FoldHash closes with a further MumFold against a constant no key can reach. See below.
  • Standard hosts without AES now use the multiply mixer instead of XxHash3. The zkEVM scalar path also uses this mixer, with software widening multiplication where hardware intrinsics are unavailable. Hash values change on every build and every path. These are in-memory hashes with no persisted or cross-version meaning.
  • Remove the unused GetXxHashCode, its dedicated test file, two unused imports, and the System.IO.Hashing package reference and central version. The packed library now has no consumer NuGet dependencies; SourceLink remains a private build dependency.

Payload-derived seeding is motivated by the proposed EIP-8025 update and zkevm-standards#41. The guest's new_payload_request_root is public and identical across retries and provers for the same payload. Hash-flooding protection depends on the seed not being known before an adversary chooses keys; this API does not provide private entropy or cryptographic authentication.

Mixer changes, and why a public seed needs them

A per-payload seed removes the precomputed colliding set EIP-8025 is about, but the guest's seed is
public, so it does not remove a set built for the payload in hand. Three properties of low ^ high made
that set free to construct rather than something to search for.

A zero product erased its factors. low ^ high is zero whenever either factor is, and the scalar
mixer XORs the seed into the key limbs before multiplying the pairs. So a key matching the seed in one
limb erased the limb folded with it, and matching one limb of each half collapsed every key onto a
single value. MultiplyFold carries the factors past the product, which is why the carry lives there
rather than in the mixer: fixing only the halves leaves MumFold(a, b) == 0 reachable through
a == 0x9E3779B97F4A7C15, collapsing every key with that low half onto hash 0 regardless of its high
half.

The product is commutative. Folding a pair without position-separating constants gave a half the
same value when its two seed-masked words were exchanged — a colliding pair for every key, requiring no
cancellation at all. MumFold's asymmetric constants separate the positions, so the limb pairs go
through it.

The carried factors chained. Carrying the factors past the product removes the constant output a
zero factor gave, but not the family: low ^ high ^ a ^ b with a zero factor returns the other factor
verbatim, so the fold becomes the identity rather than a constant, and three of those compose. With
K1 = 0x9E3779B97F4A7C15 and K2 = 0xBF58476D1CE4E5B9, u0 = s0^K1 and u2 = s2^K1 zero each half's
first factor, u1 = s1^K2^K1 carries the low half's output onto the outer fold's constant, and the hash
reduces to FoldHash(u3^s3) — so u3 = s3 ^ (t | t<<32) gives hash 0 for every t. The AES path is
reachable the same way: a known seed means known round keys, so picking the round key freely and
inverting both rounds reaches any mixer input, including one whose low word zeroes the final fold's
first factor.

An avalanche finaliser does not answer this. splitmix64's is a bijection with a cheap inverse, so an
adversary picks a preimage of any value that folds to zero. FoldHash therefore closes with another
MumFold, against a constant no key can reach: its factor is a fixed non-zero value, so the fold cannot
degenerate to the identity, and inverting it is the same problem the mixer already rests on. Cost is one
widening multiply per hash.

Measured on the zkEVM build with intrinsics disabled, so GetHashCode() takes the scalar mixer, 512
keys per case under one installed seed:

keys low ^ high with the carry
one limb matches the seed, its partner free 1 distinct 512
one limb of each half matches, partners free 1 distinct 512
a half's two seed-masked words exchanged 512 collisions 0
control: all four limbs free 512 512
single-bit input flips 64 / 64 64 / 64

Since the halves now fold through MumFold, the value that zeroes a factor is seed ^ constant rather
than the seed itself; the first two rows describe the input as it was before that routing change, and
the regression tests use the value that zeroes a factor today.

The chained family, 4096 keys per case, before and after the closing fold:

keys carry only with the closing fold
scalar mixer, a zeroed factor at every fold 1 distinct 4096
standard x64 AES path, same chain via round inversion 1 distinct 4096

What remains is the generic bound of a 32-bit hash, which is not specific to this mixer. With the seed
known, searching for a colliding set costs roughly 2^(32(k-1)/k) draws for a k-way set; measured on the
same build, a pair took a mean of 98,366 draws over 20 trials, three-way 2.8 M, four-way 31.6 M and
five-way 86.0 M. Flooding a bucket needs many keys in it, so that cost climbs steeply — which is the
difference the three fixes above make, since each property handed out a set of unbounded size for free.
This is an argument about three structural shortcuts, not a cryptanalysis of the multiply structure.

Code generation

Review measurements at 39276e3 found the standard AES wrapper at 109 bytes without a stack frame, versus 98 bytes with a stack slot in the base. Each round-key load is a 16-byte memory operand; the second round consumes the other seed half. Using Multiply64 removes the previous stack store/load around multiplication. Both scalar variants inline without calls (134 bytes for the hardware multiply form, 363 bytes for the software form). These are code-generation observations, not end-to-end throughput measurements, and they predate both fold changes: the carry adds two XOR instructions per fold, and the closing fold adds one widening multiply plus its XORs per hash.

Testing

UInt256HashSeedTests covers replacement, distribution, and every seed bit both individually and
paired with its counterpart 128 bits away. Existing hash tests exercise the public path and the scalar
mixer. The fixture draws a fresh random seed at teardown, since SeedHashes is write-only and a seed
left installed there would otherwise be inherited by whatever runs next.

Each fix is pinned by a test that fails without it:

  • MultiplyHash_ZeroingAFoldFactorDoesNotEraseItsPartner constructs the cancellation family under the
    value that zeroes a factor and asserts it distributed. It replaces
    MultiplyHash_MatchingTheSeedDoesNotCancelALimb, which pinned nothing once the halves went through
    MumFold: a factor is key ^ seed ^ constant, so key[i] == seed[i] left it at K1/K2 and it
    could not be zero — reverting the carry left all four of its cases green. The replacement fails on all
    four limbs without the carry.
  • MultiplyHash_ExchangingAHalfsWordsChangesTheHash covers the commutativity half.
  • MultiplyHash_ChainingZeroedFoldFactorsDoesNotCollapse covers the chain, and fails with the closing
    fold removed.
  • MultiplyHash_MatchesBigIntegerReference models the composition explicitly, which is what pins the
    halves going through MumFold and the closing fold being there at all.

Full suite, Release on x64, in the four configurations CI covers:

Build Intrinsics Passed Failed Skipped
Standard on 588,571 0 0
Standard off 588,570 0 1
zkEVM on 588,571 0 0
zkEVM off 588,570 0 1

The skip is the AES-only distribution test.

dotnet pack succeeds. Inspection of the resulting package confirms both impl/std and impl/zkevm assemblies and an empty consumer dependency group.

Consumer

Nethermind #13166 installs the full payload root into its own span mixers and uses UInt256Comparer for guest storage-slot maps. It still uses the published int256 package and does not yet call this API. A subsequent package upgrade and explicit UInt256.SeedHashes(in root) call are needed to seed the package's default comparer too. The version bump remains a separate PR.

The zkEVM build has no entropy source, so its `UInt256.GetHashCode` seeds are compile-time
constants. Every guest of a given version therefore buckets a given key identically for ever:
a colliding key set can be found offline against the published binary and replayed against
every prover, turning constant-time lookups linear. EIP-8025 asks a guest to mix a per-payload
value - `new_payload_request_root` - into its hash function, and there was no way to hand one
to this library.

`UInt256.SeedHashes(uint)` is that way. Both builds honour it and both replace every seed they
hash with, so the two differ only in where they start: the zkEVM build from constants, the
standard build from the seed it already draws per process. `Int256.GetHashCode` delegates here,
so it follows.

Two details worth the review:

- The seeds move to a nested `RunSeed` type. Mutating them then leaves `UInt256`'s own statics
  immutable after their constructor, which is what lets NativeAOT freeze them - and on the
  standard build it also takes the RNG call out of `UInt256`'s own class constructor.
- The 32-bit seed is spread over 64 bits by splitmix64's finalizer, chained once per derived
  seed, so one changed seed bit moves a whole seed word rather than half of one, and no two run
  seeds can land on the same derived one.
LukaszRozmej added a commit to NethermindEth/nethermind that referenced this pull request Sep 6, 2026
The type test inside GenericEqualityComparer<T> put one key type's problem in a comparer that
serves every key type. UInt256Comparer is that comparer instead: it hashes a slot key through
the run-seeded mixer, and GetOptimized() hands it to the guest and null to the host, the same
shape GenericEqualityComparer.GetOptimized already uses.

PersistentStorageProvider had a private comparer doing exactly this, so it now shares this one
and the duplicate is gone.

Also marks where UInt256.SeedHashes belongs once Nethermind.Numerics.Int256 ships it
(NethermindEth/int256#119), which is what makes the routing above a second line rather than
the only one.
Accept a UInt256 seed and mix its four limbs before the scalar hash loses input bits, preventing the old CRC collision family from surviving every seed. Preserve the AES and standard XxHash paths, deriving their seeds from the new input.

Document that callers must install a private, cryptographically random 256-bit seed before creating hash-keyed collections and replace the full seed between runs.

Add regression coverage for CRC collision families, cross-seed cancellation sets, each seed limb, distribution, and an independent BigInteger reference. Remove the unused CRC mixer and its superseded tests.
Comment thread src/Nethermind.Int256/UInt256.cs Outdated
`low ^ high` of a widening product is zero whenever either factor is, and
`GetMultiplyHashCode` XORs the seed into the key limbs before multiplying the pairs. So a key
matching the seed in one limb erased the limb folded with it, and matching one limb of each
half collapsed every key onto a single value. Reseeding is not an answer: the guest's seed is
`new_payload_request_root`, which is public and known to whoever assembles the payload, so the
set is constructible for the payload in hand rather than precomputed against the binary.

`MultiplyFold` now carries its factors past the product, which also covers `MumFold` and the
AES final fold - fixing only the halves would leave `MumFold(a, b) == 0` reachable through
`a == 0x9E3779B97F4A7C15`, collapsing every key with that low half onto hash 0.

Separately, the product is commutative, so folding a pair without position-separating constants
gave a half the same value when its two seed-masked words were exchanged - a colliding pair for
every key, needing no cancellation at all. The pairs now fold through `MumFold`, whose
asymmetric constants separate the positions.

Measured on the guest build with intrinsics disabled, 512 keys per case: one limb matched, its
partner free, 1 distinct hash before and 512 after; one limb of each half matched, 1 before and
512 after; a half's words exchanged, 512 collisions before and 0 after. Control and single-bit
input avalanche are unchanged.

`MultiplyHash_ReseedingBreaksCollisionSet` pinned the family, so it becomes
`MultiplyHash_MatchingTheSeedDoesNotCancelALimb`, asserting the same constructed set is
distributed under its own seed. The exchange case and an updated BigInteger reference follow.
@LukaszRozmej

Copy link
Copy Markdown
Member Author

@claude review - is the issue solved now?

LukaszRozmej and others added 2 commits September 7, 2026 16:09
Carrying the factors past the product removed the *constant* output a zero factor used to
give, but not the family: `low ^ high ^ a ^ b` with a zero factor returns the other factor
verbatim, so the fold becomes the identity rather than a constant. Chaining that collapses
the whole mixer. With `K1 = 0x9E3779B97F4A7C15` and `K2 = 0xBF58476D1CE4E5B9`, taking
`u0 = s0^K1` and `u2 = s2^K1` zeroes each half's first factor, `u1 = s1^K2^K1` carries the
low half's output onto the outer fold's constant, and the hash reduces to `FoldHash(u3^s3)` -
so `u3 = s3 ^ (t | t<<32)` gives hash `0` for every `t`. Measured on the scalar mixer, 4096
such keys: 1 distinct hash. The AES path is reachable the same way, because a known seed
means known round keys: pick the round key freely, invert both rounds to the `mixed` whose
low word zeroes the final fold's first factor, and 4096 forged keys also gave 1 distinct
hash on the standard x64 build.

An avalanche finaliser does not answer this - splitmix64's is a bijection with a cheap
inverse, so an attacker picks a preimage of any zero-folding value. `FoldHash` therefore
closes with another `MumFold`, against a constant no key can reach. Its factor is a fixed
non-zero value, so the fold does not degenerate and inverting it is the same problem the
mixer already rests on: the two families above now measure 4096 distinct hashes each, and
what remains is a 32-bit hash's generic search. Cost is one widening multiply per hash.

`MultiplyHash_MatchingTheSeedDoesNotCancelALimb` never exercised the cancellation its docs
described: once the halves went through `MumFold`, a factor is `key ^ seed ^ constant`, so
pinning `key[i] == seed[i]` left it at `K1`/`K2` and it could not be zero. Reverting the
factor carry left all four cases green. It becomes
`MultiplyHash_ZeroingAFoldFactorDoesNotEraseItsPartner`, using the value that does zero a
factor, and now fails on all four limbs without the carry.
`MultiplyHash_ChainingZeroedFoldFactorsDoesNotCollapse` pins the closing fold, and the
BigInteger reference models it.

Also: the fixture drew a fresh random seed at teardown rather than leaving a known one
installed process-wide, since `SeedHashes` is write-only and the fixture's seed would
otherwise be inherited by whatever ran next; the zkEVM build caches its AES round keys as
the standard build does, instead of rebuilding both vectors from the mutable static on every
hash; and `MumFold` returns `ulong`, dropping the casts at its call sites.
# Conflicts:
#	src/Nethermind.Int256/UInt256.std.cs
#	src/Nethermind.Int256/UInt256.zkevm.cs
@kamilchodola

Copy link
Copy Markdown
Contributor

@LukaszRozmej fixed conflicts and did a codex review as follows:

Findings
[Robustness rule: shared mutable state] UInt256.std.cs:17-22, same in the zkEVM file. SeedHashes writes three statics (a 32-byte struct and two vectors) with no synchronisation, while GetHashCode reads them on any thread. A hash computed concurrently with a reseed can combine limbs of the old and new seed, producing a value neither seed reproduces. The XML docs disclose this, so it may be an accepted exception, but the rule requires reporting it. A cheap fix that also removes the tearing entirely: keep the three values in one immutable sealed class instance and swap a single static reference. Readers then pay one pointer load and always see a consistent seed.

[Test quality: hot path with no benchmark] UInt256.cs:1343-1352. GetHashCode gains a widening multiply per call, and the seeds move from static readonly primitives, which Tier-1 JIT folds into immediates and a constant vector load, to mutable statics that must be loaded on every hash. The PR reports code size only and says so. There is no GetHashCode benchmark in the benchmark project. UInt256 is the key type of the storage-cell dictionaries in Nethermind, so a BenchmarkDotNet number for the AES path and the scalar path, before and after, belongs in the PR before the version bump ships it.

Summary
Findings: Robustness rule: 1, Test quality: 1
Total: 2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants