Skip to content

Commit 230a36b

Browse files
Merge pull request #355 from marius-bughiu/fix/issue-297-unsigned-hasher-names
refactor(hashing): name the algorithm in the unsigned hashers
2 parents 1debf52 + 1a63a55 commit 230a36b

26 files changed

Lines changed: 601 additions & 169 deletions

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,15 @@ All notable changes to Celerity are documented here. This project follows [Keep
3030
- A `--shard-dry-run` switch on the benchmarks runner that resolves a shard's class list without measuring anything. Closes [#300](https://github.com/marius-bughiu/Celerity/issues/300).
3131
- `scripts/benchmark_comment.js` — the pull-request benchmark comment, moved out of `benchmarks.yml` so the rule deciding which rows count as a regression is runnable and testable, with a `--self-test` wired into `ci.yml`. Closes [#351](https://github.com/marius-bughiu/Celerity/issues/351).
3232
- That comment now publishes the run's own **observed spread** — the p50, p90 and p95 of |Δ| across every paired row — so a flag can be read against the run it arrived in instead of an assumed floor. Closes [#351](https://github.com/marius-bughiu/Celerity/issues/351).
33+
- **`UInt32WangNaiveHasher`** and **`UInt64Murmur3Hasher`** in `Celerity.Hashing` — the algorithm-named replacements for `UInt32Hasher` / `UInt64Hasher`. Same hash values; every integer hasher now names its tier, so `uint` and `ulong` read the same way `int` and `long` always have. [`docs/migration.md`](docs/migration.md) covers the rename. Closes [#297](https://github.com/marius-bughiu/Celerity/issues/297).
34+
35+
### Changed
36+
37+
- The `uint` / `ulong` hasher benchmark arms are named after their algorithm rather than `_Default`, so the dashboard labels those bars like the `int` and `long` ones. The renamed rows start a new series on the tracked chart. Closes [#297](https://github.com/marius-bughiu/Celerity/issues/297).
38+
39+
### Deprecated
40+
41+
- **`UInt32Hasher` and `UInt64Hasher`** are `[Obsolete]` and forward to `UInt32WangNaiveHasher` / `UInt64Murmur3Hasher`. The bare name meant the *cheapest* mixer for `uint` and the *strongest* for `ulong`, so a caller moving between the two widths changed hash strength, not just key width. Both still ship — existing code compiles with a warning, and `UInt64Hasher` keeps `IHashProvider64<ulong>` so a sketch parameterized on it does not lose its 64-bit path — and both will be removed in a future major version. Closes [#297](https://github.com/marius-bughiu/Celerity/issues/297).
3342

3443
### Fixed
3544

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -666,13 +666,13 @@ Once the collection is settled, pick a hasher for your key shape. Defaults are g
666666
| Key type | Default | When to escalate |
667667
|---|---|---|
668668
| `int` / `long` | `Int32WangNaiveHasher` / `Int64WangNaiveHasher` (built into `IntDictionary` / `LongDictionary`) | Uniform / trusted keys (dense sequential IDs) → *drop* to `Int32IdentityHasher` / `Int64IdentityHasher` (the zero-work floor — no mixing, nothing beats it on speed). Clustered keys → `Int32WangHasher``Int32Murmur3Hasher` (the Wang full finalizer is a cheaper middle tier than Murmur3). |
669-
| `uint` / `ulong` | `UInt32Hasher` (cheap XOR-fold) / `UInt64Hasher` (`fmix64`) | `uint`: → `UInt32WangHasher``UInt32Murmur3Hasher`. `ulong`: drop to `UInt64WangHasher` / `UInt64WangNaiveHasher` when the `fmix64` multiplies cost more than they buy on uniform keys. |
669+
| `uint` / `ulong` | `UInt32WangNaiveHasher` (cheap XOR-fold) / `UInt64Murmur3Hasher` (`fmix64`) | `uint`: → `UInt32WangHasher``UInt32Murmur3Hasher`. `ulong`: drop to `UInt64WangHasher` / `UInt64WangNaiveHasher` when the `fmix64` multiplies cost more than they buy on uniform keys. |
670670
| `string` (ASCII) | `StringFnV1AHasher` (folds the low byte per char) | Non-ASCII or long keys → `StringFnV1AFullHasher` / `StringFnV1A64Hasher`. Clustered keys → strong-avalanche `StringMurmur3Hasher`, `StringXxHash3Hasher`, etc. |
671671
| `string` (untrusted input) | `DefaultHasher<string>` (BCL Marvin32, per-process-randomized) | A **keyed** PRF — `StringSipHash13Hasher` (Rust's default), `StringSipHash24Hasher`, `StringHalfSipHash24Hasher`, or `StringHighwayHash64Hasher` — but only resists hash-flooding if seeded with a *secret, per-process-random* key; with a fixed seed it is deterministic, not DoS-resistant (see caveat below). |
672672
| `Guid` | `GuidHasher` ||
673673
| Any other type | `DefaultHasher<T>` (delegates to `EqualityComparer<T>.Default`) | Replace with a hand-written struct hasher if profiling shows `Hash` on the hot path. |
674674

675-
> **Counting past ~10^8 distinct elements? Pick a 64-bit hasher.** The probabilistic sketches (`HyperLogLog`, `BloomFilter`, `CuckooFilter`, `XorFilter`, `CountMinSketch`) never store the element, so two elements that hash alike are indistinguishable forever — and a 32-bit hash reaches only 2^32 ≈ 4.3 billion values, no matter how it is widened. Hashers that carry genuine 64-bit entropy implement **`IHashProvider64<T>`** and the sketches route through it automatically: `Int64WangHasher` / `Int64Murmur3Hasher` (`long`), `UInt64WangHasher` / `UInt64Hasher` (`ulong`), `GuidHasher` (`Guid`), and the nine 64-bit `string` hashers (`StringXxHash64Hasher`, `StringXxHash3Hasher`, `StringCityHash64Hasher`, `StringMetroHash64Hasher`, `StringHighwayHash64Hasher`, `StringSipHash13Hasher`, `StringSipHash24Hasher`, `StringFnV1A64Hasher`, `StringFnV164Hasher`). They already computed 64 bits internally and folded them away, so `Hash64` costs nothing extra. With a 32-bit-only hasher the sketches still work — `HyperLogLog` applies the classical large-range correction so its estimate stays honest — but the entropy floor is real. See [`IHashProvider64<T>`](docs/api/hashing.md#ihashprovider64t).
675+
> **Counting past ~10^8 distinct elements? Pick a 64-bit hasher.** The probabilistic sketches (`HyperLogLog`, `BloomFilter`, `CuckooFilter`, `XorFilter`, `CountMinSketch`) never store the element, so two elements that hash alike are indistinguishable forever — and a 32-bit hash reaches only 2^32 ≈ 4.3 billion values, no matter how it is widened. Hashers that carry genuine 64-bit entropy implement **`IHashProvider64<T>`** and the sketches route through it automatically: `Int64WangHasher` / `Int64Murmur3Hasher` (`long`), `UInt64WangHasher` / `UInt64Murmur3Hasher` (`ulong`, as does the deprecated `UInt64Hasher` alias — prefer the named type), `GuidHasher` (`Guid`), and the nine 64-bit `string` hashers (`StringXxHash64Hasher`, `StringXxHash3Hasher`, `StringCityHash64Hasher`, `StringMetroHash64Hasher`, `StringHighwayHash64Hasher`, `StringSipHash13Hasher`, `StringSipHash24Hasher`, `StringFnV1A64Hasher`, `StringFnV164Hasher`). They already computed 64 bits internally and folded them away, so `Hash64` costs nothing extra. With a 32-bit-only hasher the sketches still work — `HyperLogLog` applies the classical large-range correction so its estimate stays honest — but the entropy floor is real. See [`IHashProvider64<T>`](docs/api/hashing.md#ihashprovider64t).
676676

677677
The value of a struct hasher is **distribution quality (avalanche), determinism, and the zero-cost devirtualized generic***not* raw hashing speed. For `int` keys especially, `GetHashCode()` is already the identity (zero work), so no mixing hasher beats it on speed; `Int32IdentityHasher` / `Int64IdentityHasher` expose that zero-work floor explicitly so you can *skip* mixing when keys are already uniform, and you escalate to a mixer only when distribution (not speed) demands it.
678678

ROADMAP.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -241,6 +241,8 @@ The 2026-Q3 review surveyed the shipped surface against the current .NET 8/9/10
241241

242242
- `SegmentTree<T, TMonoid>` — range aggregates over an arbitrary associative fold. The gap was written down in the library's own documentation: the `FenwickTree<T>` section of the API reference closed by saying a segment tree "are the next step (not shipped)". Fenwick is constrained to `INumber<T>` for a structural reason, not a stylistic one — its range query is the *difference* of two prefix folds, so the operation must have an inverse — which left the entire non-invertible half of the range-query space (min, max, gcd, bitwise and/or, any user-written fold) unreachable, with no BCL counterpart either. Status: `done` — `IMonoid<T>` ships as a `struct` type parameter alongside five built-in folds, so `Combine` inlines rather than costing a virtual call per level. Three calls are worth recording. First, the layout is the flat **`2n`** array, not the power-of-two-padded `4n` one that is usually recommended: the objection to `2n` is that the leaves sit in a rotated order at non-power-of-two lengths, so an internal node can span a wrapped range — but a query that walks outward from both ends into two separate accumulators never combines such a node into the wrong side, and an exhaustive differential sweep over every length and every range under a **non-commutative** fold pins that. A commutative fold cannot observe the difference, which is why min/max/sum could not be the oracle and the fuzz target and the differential suite both run "first non-zero wins" and string concatenation instead. The one visible consequence is that `Aggregate` is a query rather than a root read. Second, **lazy propagation was left out** rather than half-shipped: range updates need a second monoid describing how updates compose plus a distributive law relating the two, which is a different type with a different contract, and it is stated as an exclusion on every doc surface. Third, `T` is deliberately **unconstrained** — a `string`-concatenation monoid is a legitimate fold and the tree's own storage does not care — where the sibling `FenwickTree<T>` is `struct, INumber<T>`. The kill criterion (≥10x over the array scan on interleaved update + range-min at 100k) was measured after implementation and cleared at **14.8x**, with **81x** on a query batch against a pre-built tree; at 1k it is only 1.4x, and the README and API reference both lead with that rather than quoting the headline alone. The floating-point caveat on `MinMonoid` / `MaxMonoid` (the identity is the largest / smallest *finite* value, and a `NaN` resolves by operand position) is documented on the type, in the API reference and in the tests. Tracked in [#348](https://github.com/marius-bughiu/Celerity/issues/348).
243243

244+
- The bare `UIntNN Hasher` name meant opposite tiers of the escalation ladder in the two unsigned widths: `UInt32Hasher` was the cheap XOR-fold while `UInt64Hasher` was the strong Murmur3 `fmix64` finalizer, so a caller who benchmarked on `uint` and then moved to `ulong` keys by analogy silently changed hash *strength*, not just key width. Hasher selection is the main knob this library exposes and the signed families never had the problem, because they name the algorithm in the type. Status: `done` — option 1 of the issue (rename for explicitness, old names kept as `[Obsolete]` aliases until a future major version): `UInt32WangNaiveHasher` and `UInt64Murmur3Hasher` ship, and the aliases forward to them rather than repeating the mixer, so the pairs cannot drift and no hash value moved. Two calls are worth recording. First, `UInt64Hasher` keeps its `IHashProvider64<ulong>` implementation rather than being reduced to the 32-bit surface: dropping it would push an existing sketch back to the 2^32 entropy floor `IHashProvider64<T>` exists to escape, silently, as a side effect of a *naming* change. Second, the identity tier is still signed-only, and that is now recorded as an open gap rather than a decision: the first draft argued an unsigned key reaches the zero-work floor with a cast at the call site, which is false for the primary use case — the collections constrain `THasher` to `IHashProvider<TKey>` and invoke `Hash` internally, so no `IHashProvider<uint>` identity hasher means no zero-work floor for a `uint`-keyed collection at all. Filed as [#357](https://github.com/marius-bughiu/Celerity/issues/357) rather than settled inside a naming change. The regression guard is the family-wide `IntegerHasherFamilyNamingTests`, which fails on a new bare-named integer hasher or a width missing a tier; it also pins the one unsigned/signed pair that is deliberately *not* bit-identical, the 32-bit naive fold, whose shift is arithmetic on `int` and logical on `uint`. Tracked in [#297](https://github.com/marius-bughiu/Celerity/issues/297).
245+
244246
Two areas were judged real but deliberately deferred rather than rostered: a `Celerity.Statistics` package (DDSketch / reservoir sampling / running moments — a coherent fourth axis, but two new packages in one cycle is too much at once), and a batch of fuzz-target and AOT-smoke-coverage gaps (real, but low expected defect yield; better folded into whichever collection PR lands next than pursued on their own).
245247

246248
## Non-goals

docs/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ This folder contains reference documentation for the Celerity high-performance c
1313
## API reference
1414

1515
- [Collections](api/collections.md) — dictionaries (`CelerityDictionary`, `RobinHoodDictionary`, `SwissDictionary`, `HashCachingDictionary`, `PooledCelerityDictionary`, `IntDictionary`, `LongDictionary`, `SmallDictionary`, `EnumMap`, `FrozenCelerityDictionary`), sets (`CeleritySet`, `SwissSet`, `RobinHoodSet`, `HashCachingSet`, `PooledCeleritySet`, `IntSet`, `LongSet`, `SmallSet`, `EnumSet`, `FrozenCeleritySet`), multi-collections (`CelerityMultiMap`, `CelerityMultiSet`), probabilistic / bit collections (`BitSet`, `BloomFilter`, `CuckooFilter`, `XorFilter`, `HyperLogLog`, `CountMinSketch`, `TopKSketch`), caches (`LruCache`), sequences (`Deque`), and union-find (`DisjointSet`).
16-
- [Hashing](api/hashing.md)`IHashProvider<T>` interface and built-in hashers (`Int32WangNaiveHasher`, `Int32Murmur3Hasher`, `Int64WangHasher`, `Int64Murmur3Hasher`, `UInt32Hasher`, `UInt64Hasher`, `GuidHasher`, the `String*` hasher family, `DefaultHasher<T>`), and the `HashQualityEvaluator`.
16+
- [Hashing](api/hashing.md)`IHashProvider<T>` interface and built-in hashers (`Int32WangNaiveHasher`, `Int32Murmur3Hasher`, `Int64WangHasher`, `Int64Murmur3Hasher`, `UInt32WangNaiveHasher`, `UInt64Murmur3Hasher`, `GuidHasher`, the `String*` hasher family, `DefaultHasher<T>`), and the `HashQualityEvaluator`.
1717
- [Sorting](api/sorting.md) — non-comparison sorts and selection over primitive keys (`RadixSort`, `CountingSort`, `PartialSort`).
1818
- [Utilities](api/utilities.md)`FastUtils` helper methods.
1919
- [Native AOT & trimming](aot.md) — AOT / trim compatibility and how it is enforced.

docs/aot.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ The design choices that make Celerity fast also make it AOT-friendly:
1515
Compatibility is not a one-time claim; it is checked on every build and in CI.
1616

1717
1. **Static analyzers.** The library project sets `<IsAotCompatible>true</IsAotCompatible>`, which marks the assembly trimmable and turns on the trim, AOT, and single-file Roslyn analyzers. Any reflection or trim-unsafe pattern introduced into the library becomes a build warning at compile time.
18-
2. **End-to-end publish smoke test.** [`src/Celerity.AotSmokeTest`](../src/Celerity.AotSmokeTest) is a console app (`<PublishAot>true</PublishAot>`) that constructs every collection shape (`IntDictionary`, `LongDictionary`, `CelerityDictionary`, `IntSet`, `LongSet`, `CeleritySet`) and a representative spread of hashers (`GuidHasher`, `StringMurmur3Hasher`, `StringFnV1AHasher`, `DefaultHasher<T>`, `UInt32Hasher`, `UInt64Hasher`, the Murmur3 / Wang integer hashers), runs the core operations against them, and asserts the results. The `aot-publish` job in [`.github/workflows/ci.yml`](../.github/workflows/ci.yml) Native-AOT-publishes this app on every push and pull request — once per shipped target framework (`net8.0`, `net9.0`, `net10.0`) via a matrix — and then runs each resulting native binary — a non-zero exit code fails the build. This forces the AOT compiler to compile every generic instantiation down to native code and proves the collections behave correctly under AOT, not just that the analyzers are satisfied.
18+
2. **End-to-end publish smoke test.** [`src/Celerity.AotSmokeTest`](../src/Celerity.AotSmokeTest) is a console app (`<PublishAot>true</PublishAot>`) that constructs every collection shape (`IntDictionary`, `LongDictionary`, `CelerityDictionary`, `IntSet`, `LongSet`, `CeleritySet`) and a representative spread of hashers (`GuidHasher`, `StringMurmur3Hasher`, `StringFnV1AHasher`, `DefaultHasher<T>`, the Murmur3 / Wang / naive-fold integer hashers across `int` / `uint` / `long` / `ulong`), runs the core operations against them, and asserts the results. The `aot-publish` job in [`.github/workflows/ci.yml`](../.github/workflows/ci.yml) Native-AOT-publishes this app on every push and pull request — once per shipped target framework (`net8.0`, `net9.0`, `net10.0`) via a matrix — and then runs each resulting native binary — a non-zero exit code fails the build. This forces the AOT compiler to compile every generic instantiation down to native code and proves the collections behave correctly under AOT, not just that the analyzers are satisfied.
1919

2020
## Publishing a Native AOT app that uses Celerity
2121

0 commit comments

Comments
 (0)