From a222e68b52ddc81070b4d6d0b7d9a01d90d50cc5 Mon Sep 17 00:00:00 2001 From: Marius Bughiu Date: Sun, 26 Jul 2026 16:24:53 +0300 Subject: [PATCH 1/3] feat(collections): span-keyed lookups on the string-keyed collections + StringInternTable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds ISpanHashProvider to Celerity.Hashing and, on top of it, allocation-free ReadOnlySpan lookups across every string-keyed collection, plus the type that pattern makes possible: StringInternTable. Before this, a parser holding a slice of its input buffer had to call new string(span) to probe any Celerity string-keyed type — one allocation plus a copy per lookup, on the hot path of exactly the workloads those types exist for. .NET 9 closed that gap for the BCL Dictionary with GetAlternateLookup; net8.0 is this library's floor, so it had no answer at all. - ISpanHashProvider (int Hash(ReadOnlySpan)) is a sibling interface, not an overload on IHashProvider: a ref struct could not be a generic type argument before allows ref struct, so the span has to be a method parameter. All 23 String*Hasher types implement it, each sharing one body between the two overloads so they cannot drift. - Span TryGetValue / ContainsKey / Contains on FrozenCelerityDictionary, FrozenCeleritySet, CelerityDictionary, CeleritySet and Trie. On the four hashed types these are extension methods carrying the extra constraint on the method, so no shipped type's own constraints change and the JIT still devirtualizes the hash call. - StringInternTable / StringInternTable: GetOrAdd(ReadOnlySpan) returns the canonical string and allocates only on a miss. Full parity rollout in this commit: dedicated + cross-collection tests, a hasher parity suite, a CsCheck differential, a Celerity.Fuzz target, an AOT smoke-test block, a tracked benchmark plus SpanLookup rows on two existing ones, dashboard wiring, API reference and README sections, CHANGELOG, ROADMAP. Closes #311 Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 7 + README.md | 11 +- ROADMAP.md | 2 +- docs/api/collections.md | 138 ++++++ docs/api/hashing.md | 59 +++ src/Celerity.AotSmokeTest/Program.cs | 65 +++ .../FrozenCelerityDictionaryBenchmark.cs | 45 ++ src/Celerity.Benchmarks/Program.cs | 1 + .../StringInternTableBenchmark.cs | 158 +++++++ src/Celerity.Benchmarks/TrieBenchmark.cs | 48 +++ src/Celerity.Fuzz/Differential.cs | 67 +++ src/Celerity.Hashing/ISpanHashProvider.cs | 44 ++ src/Celerity.Hashing/StringAdler32Hasher.cs | 16 +- .../StringCityHash64Hasher.cs | 46 +- src/Celerity.Hashing/StringCrc32Hasher.cs | 17 +- src/Celerity.Hashing/StringDjb2AHasher.cs | 16 +- src/Celerity.Hashing/StringDjb2Hasher.cs | 16 +- src/Celerity.Hashing/StringElfHasher.cs | 17 +- src/Celerity.Hashing/StringFnV164Hasher.cs | 30 +- src/Celerity.Hashing/StringFnV1A64Hasher.cs | 30 +- src/Celerity.Hashing/StringFnV1AFullHasher.cs | 16 +- src/Celerity.Hashing/StringFnV1AHasher.cs | 15 +- src/Celerity.Hashing/StringFnV1Hasher.cs | 16 +- .../StringHalfSipHash24Hasher.cs | 19 +- .../StringHighwayHash64Hasher.cs | 36 +- .../StringJenkinsOaatHasher.cs | 16 +- .../StringMetroHash64Hasher.cs | 36 +- src/Celerity.Hashing/StringMurmur2Hasher.cs | 16 +- src/Celerity.Hashing/StringMurmur3Hasher.cs | 16 +- src/Celerity.Hashing/StringSdbmHasher.cs | 16 +- src/Celerity.Hashing/StringSipHash13Hasher.cs | 34 +- src/Celerity.Hashing/StringSipHash24Hasher.cs | 34 +- src/Celerity.Hashing/StringXxHash32Hasher.cs | 18 +- src/Celerity.Hashing/StringXxHash3Hasher.cs | 50 ++- src/Celerity.Hashing/StringXxHash64Hasher.cs | 36 +- .../Collections/SpanLookupTests.cs | 311 ++++++++++++++ .../StringInternTableDifferentialTests.cs | 127 ++++++ .../Collections/StringInternTableTests.cs | 326 ++++++++++++++ .../Hashing/SpanHashParityTests.cs | 209 +++++++++ .../Collections/CelerityDictionary.cs | 41 ++ src/Celerity/Collections/CeleritySet.cs | 38 ++ .../Collections/FrozenCelerityDictionary.cs | 41 ++ src/Celerity/Collections/FrozenCeleritySet.cs | 38 ++ .../Collections/SpanLookupExtensions.cs | 185 ++++++++ src/Celerity/Collections/StringInternTable.cs | 404 ++++++++++++++++++ src/Celerity/Collections/Trie.cs | 56 ++- web/dev/bench/detail.html | 1 + web/dev/bench/index.html | 8 +- web/index.html | 1 + 49 files changed, 2914 insertions(+), 79 deletions(-) create mode 100644 src/Celerity.Benchmarks/StringInternTableBenchmark.cs create mode 100644 src/Celerity.Hashing/ISpanHashProvider.cs create mode 100644 src/Celerity.Tests/Collections/SpanLookupTests.cs create mode 100644 src/Celerity.Tests/Collections/StringInternTableDifferentialTests.cs create mode 100644 src/Celerity.Tests/Collections/StringInternTableTests.cs create mode 100644 src/Celerity.Tests/Hashing/SpanHashParityTests.cs create mode 100644 src/Celerity/Collections/SpanLookupExtensions.cs create mode 100644 src/Celerity/Collections/StringInternTable.cs diff --git a/CHANGELOG.md b/CHANGELOG.md index e5f99343..1c1df54a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,13 @@ All notable changes to Celerity are documented here. This project follows [Keep ### Added +- **`ISpanHashProvider`** in `Celerity.Hashing` — a `Hash(ReadOnlySpan)` contract, implemented by all 23 built-in `String*Hasher` types, that returns exactly what `Hash(string)` returns for the same characters. It is a sibling interface rather than an overload on `IHashProvider` because a `ref struct` cannot be a generic type argument on `net8.0`, this library's floor. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- **Span-keyed lookups on the string-keyed collections** — `TryGetValue(ReadOnlySpan, out TValue?)` / `ContainsKey(ReadOnlySpan)` / `Contains(ReadOnlySpan)` on `FrozenCelerityDictionary`, `FrozenCeleritySet`, `CelerityDictionary`, `CeleritySet`, and `Trie`. A tokenizer, CSV/log reader, or route dispatcher holding a slice of its input buffer no longer pays a `new string(span)` allocation and copy per probe. Results match the `string` overloads exactly (ordinal comparison); an empty span means `""`, never the `null` key. Available on all three target frameworks, including the `net8.0` floor where the BCL has no equivalent. On the four hashed collections these are extension methods, so no existing type's constraints change. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- **`StringInternTable` / `StringInternTable`** in `Celerity.Collections` — a canonicalizing token table probed with a `ReadOnlySpan`: `GetOrAdd` returns the one shared `string` for those characters and allocates **only on a miss**, so a 10M-cell parse over 100 distinct tokens creates 100 strings instead of 10,000,000. The pre-.NET-9 BCL cannot express this — `HashSet.TryGetValue` takes a `string`, so you must allocate before you can discover you already had it — and `string.Intern` is process-wide, never collected, and still needs a `string`. Implements `IReadOnlyCollection`; not thread-safe. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- **`SpanHashParityTests`** — a family-wide roster test plus a per-hasher `Hash(s) == Hash(s.AsSpan())` assertion over every length class, checked both as a whole string and as a slice of a larger buffer, since a divergence there would silently report a stored key as absent. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- **`SpanLookupTests`, `StringInternTableTests`, `StringInternTableDifferentialTests`**, and a `StringInternTable` target in the `Celerity.Fuzz` differential harness — cross-collection coverage asserting each span path against its own type's string overload, plus seeded differential runs against a `Dictionary` oracle. `Celerity.AotSmokeTest` exercises every new span instantiation and the intern table under a Native AOT publish. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- **`StringInternTableBenchmark`** (registered in the CI-tracked suite) plus `SpanLookup` rows on `FrozenCelerityDictionaryBenchmark` and `TrieBenchmark`, with the matching dashboard cards. Allocation is the headline: the BCL arms allocate one string per occurrence, the Celerity arms one per distinct token. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- API reference sections for `ISpanHashProvider`, span-keyed lookups, and `StringInternTable`, plus README entries in the collections list, the "Choosing a collection" decision table, and the hasher-picking notes. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). - **`StringKeyProbeBenchmark`** — the tracked suite's first `string`-keyed dictionary and set rows (lookup hit, lookup miss, set `Contains`), shown as the **String-keyed probe** card on the benchmark dashboard. Closes [#308](https://github.com/marius-bughiu/Celerity/issues/308). - **`ReferenceKeyProbeTests`** — cross-collection coverage for vacant-slot detection on the open-addressed collections with reference-type keys. Closes [#308](https://github.com/marius-bughiu/Celerity/issues/308). - **`BTreeDictionary` and `BTreeSet`** (with `BTreeDictionary` / `BTreeSet` aliases and the `DefaultComparer` struct comparer) in `Celerity.Collections` — the library's first sorted map and set, and the B-tree the BCL lacks. Up to 31 keys per node keep a lookup `log₃₂(n)` node visits deep instead of chasing the `log₂(n)` pointers a red-black tree costs, and both add the ordered surface a hash table cannot answer: `Min` / `Max`, lower / upper bound, `EnumerateRange` in `O(log n + k)`, and in-order enumeration. They win on the interleaved insert + lookup + range-scan workload and on memory, and lose slightly on a delete-dominated one. Not thread-safe. Closes [#305](https://github.com/marius-bughiu/Celerity/issues/305). diff --git a/README.md b/README.md index 5082543a..a5736a6f 100644 --- a/README.md +++ b/README.md @@ -86,6 +86,11 @@ The mutable sets (`CeleritySet`, `SwissSet`, `RobinHoodSet`, `HashCachingSet`, ` - `Trie` — ordered **prefix tree** mapping string keys to values. `GetByPrefix` lists every entry whose key starts with a prefix in `O(prefix + matches)`, and `TryGetLongestPrefix` finds the longest stored key that is a prefix of a query in `O(query)`. The trie the BCL lacks — autocomplete, longest-prefix routing, and ordered (ascending-ordinal) iteration, where a `Dictionary` has no prefix index and must scan every key and run `StartsWith`. Exact `Add` / `TryGetValue` favour a `Dictionary` (one hash vs a character walk); the trie earns its place on the prefix operations. Implements `IReadOnlyDictionary`. +**Span-keyed string lookups** + +- `StringInternTable` / `StringInternTable` — a **canonicalizing token table** probed with a `ReadOnlySpan`: `GetOrAdd` returns the one shared `string` for those characters and allocates **only on a miss**. A 10M-cell parse over 100 distinct tokens creates 100 strings instead of 10,000,000. The collection you cannot build on the pre-.NET-9 BCL — `HashSet.TryGetValue` makes you allocate the string *before* you can discover you already had it, and `string.Intern` is process-wide, never collected, and still needs a `string`. Implements `IReadOnlyCollection`. +- The same span-keyed probes ship on `FrozenCelerityDictionary`, `FrozenCeleritySet`, `CelerityDictionary`, `CeleritySet`, and `Trie` — so a tokenizer, CSV/log reader, or route dispatcher holding a slice of its input buffer never has to call `new string(span)` per lookup. Works on all three target frameworks, including the `net8.0` floor where the BCL has no equivalent (.NET 9 added `Dictionary.GetAlternateLookup`). See [span-keyed lookups](docs/api/collections.md#span-keyed-lookups). + **Sorted (ordered) collections** - `BTreeDictionary` / `BTreeDictionary` — a **sorted map backed by a B-tree**: up to **31 keys per node** in flat arrays, so a lookup visits `log₃₂(n)` nodes instead of chasing `log₂(n)` pointers — roughly **4 cache misses instead of ~20 at `n = 1M`**. Adds the ordered surface a hash table cannot answer: `Min`, `Max`, `TryGetLowerBound` / `TryGetUpperBound`, `EnumerateRange` in `O(log n + k)`, and in-order enumeration. The B-tree the BCL lacks — `SortedDictionary<,>` is a red-black tree with one heap object per entry, and `SortedList<,>` memmoves the tail on every insert. Implements `IDictionary` and `IReadOnlyDictionary`. @@ -580,6 +585,8 @@ Each type buys a different tradeoff. Find your workload below; if it isn't here, | **Incremental connectivity / connected components** — union equivalence classes and ask whether two elements are in the same group (Kruskal MST, clustering, image segmentation, undirected cycle detection, "are these accounts linked?") | `DisjointSet` | Union-find with **union by size** + **path halving**: near-`O(1)` amortized `Union` / `Find` / `Connected`, `O(α(n)) ≤ 4`. Runs a stream of merges + connectivity queries in near-linear total time, where the BCL substitutes are super-linear — a `Dictionary>` set-merge is `O(n²)` to coalesce `n` singletons, and a per-query BFS/DFS is `O(V+E)` every query. Grows only by merging (no un-union); it is not an `ISet` — for element membership with add/remove/set-algebra use `CeleritySet` or `HashSet`. | | **Priority queue whose priorities change** — a best-so-far frontier you relax (Dijkstra / Prim / A\*), or an event scheduler that reschedules / cancels pending items | `IndexedPriorityQueue` | Addressable binary min-heap with an element→slot index: `Update` (decrease-/increase-key) and `Remove` an arbitrary element in `O(log n)`, `Contains` / `TryGetPriority` in `O(1)`. The BCL `PriorityQueue<,>` can do none of these — its only substitute is lazy deletion, which grows the heap by one entry per update. Each element is a key (appears once); custom `IComparer` for a max-heap. For plain enqueue/dequeue with duplicate elements, the BCL `PriorityQueue<,>` is simpler. | | **Prefix / autocomplete / longest-prefix** over string keys — list everything under a prefix, find the most specific stored key that prefixes a query, or iterate keys in order (typeahead, route/dispatch tables, tokenizer / dictionary matching, namespace listing) | `Trie` | Ordered prefix tree: `GetByPrefix` yields every entry under a prefix in `O(prefix + matches)` and in ascending key order, `TryGetLongestPrefix` finds the longest stored prefix of a query in `O(query)`, and enumeration is sorted for free — none of which a `Dictionary` can do without an `O(n)` scan + `StartsWith`. For **pure exact-key** `Add` / `TryGetValue` / `Remove` a `Dictionary` (one hash vs a per-character walk) is faster; the trie earns its place only when you use the prefix operations. Implements `IReadOnlyDictionary`; not thread-safe. | +| **Many occurrences of few distinct strings**, produced as slices of a buffer you already hold — CSV / log / JSON parsing, tokenizers, column stores, header dispatch: you want one canonical `string` per distinct token, not one per occurrence | `StringInternTable` | Probed with a `ReadOnlySpan`: `GetOrAdd` returns the shared instance and **allocates only on a miss**, so a 10M-cell parse over 100 distinct tokens creates 100 strings, not 10,000,000. `HashSet` cannot express this before .NET 9 — its `TryGetValue` takes a `string`, so you must allocate the string *before* you can discover you already had it. `string.Intern` is process-wide, never collected, and still needs a `string`; this table's lifetime is yours and `Clear` releases it. On .NET 9+ `Dictionary.GetAlternateLookup` is comparable; this works on `net8.0` too. Not thread-safe. | +| **Look a string key up from a `ReadOnlySpan`** you already hold (route dispatch, header lookup, parse-then-map) without allocating a `string` per probe | span overloads on `FrozenCelerityDictionary` / `FrozenCeleritySet` / `CelerityDictionary` / `CeleritySet` / `Trie` | `TryGetValue(ReadOnlySpan, …)` / `ContainsKey` / `Contains` probe the table directly, deleting the `new string(span)` allocation and copy per lookup. Available whenever the hasher implements `ISpanHashProvider` — every built-in `String*Hasher` does. Same results as the `string` overloads (ordinal comparison); an empty span means `""`, never the `null` key. See [span-keyed lookups](docs/api/collections.md#span-keyed-lookups). | | **Sorted keys** — you need the entries in comparer order, or the ordered questions a hash table cannot answer: smallest / largest key, "first key at or after *x*", "every key in `[a, b)`" (time-series by timestamp, order books, LSM-style memtables, sweep-line events, interval endpoints) | `BTreeDictionary` / `BTreeSet` | B-tree with up to 31 keys per node in flat arrays: a lookup visits `log₃₂(n)` nodes instead of chasing `log₂(n)` pointers (~4 cache misses instead of ~20 at `n = 1M`), an in-order walk streams contiguous arrays rather than successor pointers, and allocation is one node per 31 entries instead of one object per entry. The BCL has no B-tree: `SortedDictionary<,>` / `SortedSet<>` are red-black trees, `SortedList<,>` is `O(n)` per middle insert, and `OrderedDictionary<,>` (.NET 9) is *insertion*-ordered, not sorted. Wins on the **interleaved insert + lookup + range-scan** load; for a few dozen entries a `SortedList<,>` is hard to beat, and if you never need order a hash table answers in `O(1)`. | | **Prefix / range sums over a sequence you keep mutating** — running aggregates, rank / order-statistics counters (inversions, "how many ≤ x seen"), cumulative-frequency tables | `FenwickTree` | Binary Indexed Tree (`T : INumber`): **point update** and **prefix / range sum** both `O(log n)`, in one array with no per-node overhead. The BCL has no prefix-sum structure; a plain array forces `O(n)` per query (recompute the slice) *or* `O(n)` per update (fix the suffix). Wins precisely when updates and partial-sum queries interleave. If the data is immutable after build, a one-shot precomputed prefix-sum array answers in `O(1)` with less code; if you only update and never query a partial sum, a raw array is simpler. | | Need a stable iteration order or multi-threaded access | `BTreeDictionary<,>` / `BTreeSet<>` for sorted order, `Trie` for ordered string keys; BCL `ConcurrentDictionary<,>` for concurrency | Celerity is single-threaded, and the **hash-based** collections leave iteration order unspecified. The ordered collections do promise order by contract: the B-trees iterate in comparer order, `Trie` in ascending ordinal key order. | @@ -605,6 +612,8 @@ The value of a struct hasher is **distribution quality (avalanche), determinism, > **Fixed-seed hashers are not a HashDoS defence.** `string.GetHashCode()` is already a purpose-built **Marvin32** with per-process random seeding; a hardcoded-seed Murmur3 / FNV / xxHash is *not* more flood-resistant — usually **less**, because an attacker who knows the fixed algorithm and seed can precompute colliding keys offline. What stops hash-flooding is a **keyed** PRF with a *secret, per-process-random* key, not merely picking a "stronger" fixed hash. For untrusted `string` keys, the BCL `string.GetHashCode()` (`DefaultHasher`) is the safe default; reach for the keyed SipHash / HighwayHash hashers only when you also supply a secret seed. The fixed-seed hashers' real strength is **reproducibility** (same code across processes and runtimes), which `GetHashCode()` deliberately does not give you. +> **Probing from a `ReadOnlySpan`? Any `String*Hasher` will do.** All 23 implement **`ISpanHashProvider`** — a `Hash(ReadOnlySpan)` overload that returns exactly what `Hash(string)` returns for the same characters — which is what unlocks the [span-keyed lookups](docs/api/collections.md#span-keyed-lookups) and `StringInternTable`. The two overloads share one body, so they cannot drift; a custom string hasher only needs to implement the interface to work the same way. See [`ISpanHashProvider`](docs/api/hashing.md#ispanhashprovider). + The hashing library also ships classic / compatibility hashes (djb2, sdbm, ELF/PJW, CRC-32, Adler-32, FNV-1, MurmurHash2, CityHash, MetroHash, xxHash32/64) for matching an external system's key distribution — see [`docs/api/hashing.md`](docs/api/hashing.md) for the complete list, costs, and avalanche notes, and use `HashQualityEvaluator` (below) to compare candidates on your own keys. ## Benchmarks @@ -731,7 +740,7 @@ Celerity is **Native AOT and trimming compatible** — no reflection, runtime co ## API at a glance -The dictionaries mirror the parts of `Dictionary` most callers reach for: indexer get/set, `ContainsKey`, `TryGetValue`, `Add`, `TryAdd`, `Remove` (both overloads), `Clear`, `EnsureCapacity` / `TrimExcess`, `Count`, `Keys`, `Values`, `GetEnumerator()`. They implement `IReadOnlyDictionary` and accept an `IEnumerable>` at construction. The sets expose `Add`, `TryAdd`, `Contains`, `Remove`, `Clear`, `EnsureCapacity` / `TrimExcess`, `Count`, and a struct enumerator. `EnsureCapacity(n)` pre-grows the table once for a known-size bulk insert (no incremental rehashes); `TrimExcess()` rehashes back down to fit `Count`. The zero / `default(TKey)` key (or element) is stored out-of-band so it never collides with the empty-slot sentinel. +The dictionaries mirror the parts of `Dictionary` most callers reach for: indexer get/set, `ContainsKey`, `TryGetValue`, `Add`, `TryAdd`, `Remove` (both overloads), `Clear`, `EnsureCapacity` / `TrimExcess`, `Count`, `Keys`, `Values`, `GetEnumerator()`. The string-keyed types additionally take a `ReadOnlySpan` on `TryGetValue` / `ContainsKey` / `Contains`, so a caller holding a slice of a buffer never allocates a `string` to probe. They implement `IReadOnlyDictionary` and accept an `IEnumerable>` at construction. The sets expose `Add`, `TryAdd`, `Contains`, `Remove`, `Clear`, `EnsureCapacity` / `TrimExcess`, `Count`, and a struct enumerator. `EnsureCapacity(n)` pre-grows the table once for a known-size bulk insert (no incremental rehashes); `TrimExcess()` rehashes back down to fit `Count`. The zero / `default(TKey)` key (or element) is stored out-of-band so it never collides with the empty-slot sentinel. Full constructors, signatures, exceptions, and per-type examples: **[API reference](docs/README.md)**. diff --git a/ROADMAP.md b/ROADMAP.md index 118af84b..8de80eb6 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -214,7 +214,7 @@ The 2026-Q3 review surveyed the shipped surface against the current .NET 8/9/10 - Fix `HyperLogLog`'s hash-entropy floor. `Hash64` widened a 32-bit `IHashProvider` result, so the reachable hash space was 2^32 — while the type's own docs asserted a 64-bit space and skipped the classical large-range correction on that basis. The bias exceeded the advertised 0.81% standard error from ~1e8 distinct elements, in exactly the regime the type is sold for. Status: `done` — `IHashProvider64` (`ulong Hash64(T key)`) ships as a standalone sibling interface in `Celerity.Hashing`, deliberately *not* deriving from `IHashProvider` so the two contracts stay independent and a 64-bit hasher is never forced to publish a lossy 32-bit fold. Fourteen built-in hashers implement it — `Int64WangHasher`, `Int64Murmur3Hasher`, `UInt64WangHasher`, `UInt64Hasher`, `GuidHasher`, and the nine 64-bit `string` hashers — each of which already computed 64 bits internally and folded them away, so `Hash64` is the same mixer minus the narrowing. The 32-bit-only hashers (`Int32*` / `UInt32*`, the naive folds, `DefaultHasher`) deliberately do not, since a key type narrower than 64 bits has no entropy to publish; a roster test pins that judgement. All five sketches route through it when the hasher provides it, via a compile-time type test the JIT folds away (so neither path allocates or branches) and with existing constructors and type parameters unchanged; on a 32-bit hasher `HyperLogLog` now applies the classical Flajolet large-range correction it previously skipped. `HashQualityEvaluator.Evaluate64` reports distribution over the 64-bit surface. Tracked in [#304](https://github.com/marius-bughiu/Celerity/issues/304). - Implement `IReadOnlySet` on the mutable sets and `IDictionary` on the dictionaries. The sets implement `ISet` and the dictionaries `IReadOnlyDictionary<,>`, but `ISet` does not derive from `IReadOnlySet` — so an ordinary BCL-shaped API taking either interface is a compile error against a Celerity type today. This is the same Guiding Principle #3 gap the 2.2.0 set-algebra work closed, one level up. Status: `planned`. - Delete the per-probe virtual call. The probe loops test for an empty slot with `EqualityComparer.Default.Equals(slot, default(TKey))`, which the JIT devirtualizes for value-type keys but not under `__Canon`-shared reference-type instantiations — one `callvirt` per probe iteration to perform what is a null check. Guiding Principle #2 exists to remove exactly this. Status: `done` — the twelve open-addressed collections now route every vacant-slot test through an internal `EmptySlot.Is` helper whose `typeof(T).IsValueType` guard the JIT folds, so a reference-type instantiation compiles to a plain null test and a value-type one keeps the existing intrinsic comparison unchanged. Behaviour is identical by construction and the whole existing suite passes untouched; `ReferenceKeyProbeTests` pins the substitution against a key type whose `Equals` claims equality with `null`, and the new `StringKeyProbeBenchmark` gives the dashboard its first reference-type-key rows. The follow-up `IEqualityProvider` idea was **not** opened: a `HashCachingDictionary` control arm showed the residual reference-type-key deficit is dominated by re-hashing the key on every probe, not by the remaining equality dispatch — the actionable guidance is to use the hash-caching variants, now documented in [`docs/performance.md`](docs/performance.md#reference-type-keys-cache-the-hash). Tracked in [#308](https://github.com/marius-bughiu/Celerity/issues/308). -- Span-keyed lookups on the string-keyed collections. .NET 9's `GetAlternateLookup>` lets the BCL `Dictionary` probe with a span key and no allocation; Celerity's string-keyed types require a materialized `string`, so the BCL is now *ahead* on the axis this library has invested most in. Status: `planned`. +- Span-keyed lookups on the string-keyed collections. .NET 9's `GetAlternateLookup>` lets the BCL `Dictionary` probe with a span key and no allocation; Celerity's string-keyed types require a materialized `string`, so the BCL is now *ahead* on the axis this library has invested most in. Status: `done` — `ISpanHashProvider` (`int Hash(ReadOnlySpan key)`) ships as a standalone sibling interface in `Celerity.Hashing`, deliberately *not* deriving from `IHashProvider`: that interface is generic in its key type, and a `ref struct` could not be a generic type argument before `allows ref struct` (C# 13 / .NET 9) while `net8.0` remains the floor — expressing the span overload as a non-generic sibling sidesteps that, because the span is a method parameter rather than a type argument. All 23 built-in `String*Hasher` types implement it, each sharing one body between the two overloads so they cannot drift; `SpanHashParityTests` pins `Hash(s) == Hash(s.AsSpan())` per hasher across every length class and as a slice of a larger buffer, since a divergence would silently report a stored key as absent rather than merely being slow. `FrozenCelerityDictionary`, `FrozenCeleritySet`, `CelerityDictionary`, `CeleritySet` and `Trie` gained span `TryGetValue` / `ContainsKey` / `Contains`; on the four hashed types they are extension methods carrying the extra `ISpanHashProvider` constraint on the *method*, so no shipped type's own constraints changed (which would have broken every existing instantiation) and the JIT still devirtualizes through the struct type parameter. `StringInternTable` ships alongside them as the type the pattern makes possible: `GetOrAdd(ReadOnlySpan)` allocates only on a miss, so a 10M-cell parse over 100 distinct tokens creates 100 strings — the one collection the pre-.NET-9 BCL cannot express, since `HashSet.TryGetValue` makes you allocate the string before you can discover you already had it. The optional `ReadOnlySpan` UTF-8 axis and the `#if NET9_0_OR_GREATER` `IAlternateEqualityComparer` plumbing were both left out as the issue's own scoping allowed — neither is needed for the workload win, and each would widen a brand-new public abstraction before it is load-bearing. Tracked in [#311](https://github.com/marius-bughiu/Celerity/issues/311). **The ordered / compressed integer-data lane.** Opened by the sorted-container hole — 38 collections and not one sorted map or set, with `Trie` the only ordered type, and that one string-keyed. The B-trees below close that half; the compressed-integer half is still open. diff --git a/docs/api/collections.md b/docs/api/collections.md index 772be339..066321cd 100644 --- a/docs/api/collections.md +++ b/docs/api/collections.md @@ -144,6 +144,10 @@ public Enumerator GetEnumerator() Returns a struct enumerator that yields `KeyValuePair`. The out-of-band default-key entry is yielded first if present. *Structurally* mutating the dictionary during enumeration — adding a new key, removing a key, or `Clear` — throws `InvalidOperationException` from the next `MoveNext` / `Reset` call, matching BCL `Dictionary<,>` semantics. Overwriting the value of an existing key via the indexer (`dict[existingKey] = newValue`) is *not* a structural change and does not invalidate an active enumerator, so the common "iterate and update values in place" pattern is legal. Iteration order is unspecified and may change between versions. +### Span-keyed lookups (string keys) + +When `TKey` is `string` and the hasher also implements `ISpanHashProvider` (every built-in `String*Hasher` does), `TryGetValue(ReadOnlySpan, out TValue?)` and `ContainsKey(ReadOnlySpan)` probe directly from a slice of a caller-held buffer — no `new string(span)` per lookup. See [span-keyed lookups](#span-keyed-lookups). + ### IReadOnlyDictionary<TKey, TValue?> `CelerityDictionary` implements `IReadOnlyDictionary` via thin explicit interface forwarders on top of the existing struct `KeyCollection` / `ValueCollection` / `Enumerator` types. The zero-allocation `foreach` fast path is preserved; the interface path boxes the enumerator exactly once per `GetEnumerator()` call, matching BCL `Dictionary<,>` behaviour. The out-of-band default-key entry is surfaced through every interface member. @@ -657,6 +661,10 @@ Each throws `ArgumentNullException` when `other` is `null`. The subset / equalit > **`Add` note.** `ISet.Add(T)` returns `bool` (the non-throwing add, equivalent to `TryAdd`). The concrete `public void Add(T)` keeps its throw-on-duplicate behaviour — cast to `ISet`, or use `TryAdd`, when you want the boolean result. `ICollection.Add(T)` ignores duplicates (never throws). +### Span-keyed lookups (string elements) + +When `T` is `string` and the hasher also implements `ISpanHashProvider` (every built-in `String*Hasher` does), `Contains(ReadOnlySpan)` probes directly from a slice of a caller-held buffer — no `new string(span)` per lookup. See [span-keyed lookups](#span-keyed-lookups). + ### Default-element handling `default(T)` is stored out-of-band via a `_hasDefaultValue` flag and never collides with the empty-slot sentinel. Mutating the set during enumeration — including via a mutating set operation such as `UnionWith` — throws `InvalidOperationException` on the next `MoveNext` / `Reset`, matching BCL `HashSet`. @@ -1274,6 +1282,8 @@ var byName = new FrozenCelerityDictionary(new[] Console.WriteLine(byName["alice"]); // 1 ``` +Both the convenience type and this one also carry `TryGetValue(ReadOnlySpan, out TValue?)` and `ContainsKey(ReadOnlySpan)` — see [span-keyed lookups](#span-keyed-lookups). + ## FrozenCeleritySet ```csharp @@ -1391,6 +1401,8 @@ var tags = new FrozenCeleritySet(new[] { "alice", "bob" }); Console.WriteLine(tags.Contains("alice")); // True ``` +Both the convenience type and this one also carry `Contains(ReadOnlySpan)` — see [span-keyed lookups](#span-keyed-lookups). + ## CelerityMultiMap<TKey, TValue, THasher> ```csharp @@ -3662,11 +3674,14 @@ The getter throws `KeyNotFoundException` if `key` is absent (an interior prefix | `void Add(string key, TValue value)` | Adds a key. Throws `ArgumentException` if it already exists. | | `bool TryAdd(string key, TValue value)` | Adds a key, leaving an existing entry unchanged. Returns `false` if already present. | | `bool ContainsKey(string key)` | Whether `key` is a stored key (an interior-only prefix returns `false`). | +| `bool ContainsKey(ReadOnlySpan key)` | The same, from a character span — no `string` is materialized. See [span-keyed lookups](#span-keyed-lookups). | | `bool TryGetValue(string key, out TValue? value)` | Non-throwing exact lookup. | +| `bool TryGetValue(ReadOnlySpan key, out TValue? value)` | The same, from a character span. | | `bool Remove(string key)` | Removes a key, pruning any newly-dead nodes. Returns `false` if absent. | | `bool Remove(string key, out TValue? value)` | `Remove` returning the removed value (`default` when the key was absent). | | `void Clear()` | Removes all keys. | | `bool ContainsPrefix(string prefix)` | Whether any stored key starts with `prefix` (a key equal to the prefix counts). The empty prefix matches iff the trie is non-empty. | +| `bool ContainsPrefix(ReadOnlySpan prefix)` | The same, from a character span. | | `IEnumerable> GetByPrefix(string prefix)` | Every entry whose key starts with `prefix`, in ascending key order (lazy). | | `IEnumerable GetKeysWithPrefix(string prefix)` | The keys of `GetByPrefix`, in ascending order (lazy). | | `bool TryGetLongestPrefix(string query, out string? key, out TValue? value)` | The longest stored key that is a prefix of `query` (an exact match qualifies and is longest). On a miss (`false`), `key` is `null` and `value` is `default`. | @@ -3703,6 +3718,129 @@ if (routes.TryGetLongestPrefix("/api/v1/users/42", out string? route, out string Console.WriteLine($"matched {route} -> {handler}"); // matched /api/v1/users -> users-v1 ``` +## Span-keyed lookups + +Every string-keyed Celerity collection can be probed with a `ReadOnlySpan` — a slice of a buffer the caller already holds — without first materializing a `string`. + +| Type | Span members | +|------|--------------| +| `FrozenCelerityDictionary` | `TryGetValue(ReadOnlySpan, out TValue?)`, `ContainsKey(ReadOnlySpan)` | +| `FrozenCeleritySet` | `Contains(ReadOnlySpan)` | +| `CelerityDictionary` | `TryGetValue(ReadOnlySpan, out TValue?)`, `ContainsKey(ReadOnlySpan)` | +| `CeleritySet` | `Contains(ReadOnlySpan)` | +| `Trie` | `TryGetValue(ReadOnlySpan, out TValue?)`, `ContainsKey(ReadOnlySpan)`, `ContainsPrefix(ReadOnlySpan)` | + +### Why it matters + +Without them, a tokenizer, CSV / log reader, or route dispatcher holding a `ReadOnlySpan` must call `new string(span)` to probe the collection: **one allocation plus a copy per lookup**, on the hot path of exactly the workloads these types exist for. The span overloads delete both. Nothing else changes — stored keys are compared ordinally against the span, which is what `EqualityComparer.Default` does, so a span lookup and the equivalent string lookup always agree. + +> **.NET 9+.** `Dictionary.GetAlternateLookup>()` gives the BCL the same capability on .NET 9 and later. These overloads work on **all three** of Celerity's target frameworks, including the `net8.0` floor where the BCL has no answer. + +### How they are exposed + +On the four hashed collections the span overloads are **extension methods** in `Celerity.Collections` (`SpanLookupExtensions`), because the span probe needs the hasher to implement [`ISpanHashProvider`](hashing.md#ispanhashprovider) as well as `IHashProvider` — and adding that to the class's own constraint would break every existing instantiation. The extra constraint therefore lives on the methods: + +```csharp +where THasher : struct, IHashProvider, ISpanHashProvider +``` + +They bind only when the hasher supplies both, resolve statically (no boxing — the JIT still devirtualizes the hash call through the struct type parameter), and read like instance methods at the call site as long as `Celerity.Collections` is in scope. Every built-in `String*Hasher` implements both. `Trie` takes no hasher, so its span overloads are ordinary instance methods. + +### The empty span + +A span has no `null` state, so an **empty span means the empty string `""`** — an ordinary key — and never the out-of-band `null` key. Look the `null` key up through the `string` overload. + +### Usage example + +```csharp +using Celerity.Collections; +using Celerity.Hashing; + +var routes = new FrozenCelerityDictionary( +[ + new("/api/v1/users", 1), + new("/api/v1/orders", 2), +]); + +ReadOnlySpan line = "GET /api/v1/users HTTP/1.1".AsSpan(); +ReadOnlySpan path = line.Slice(4, 14); + +if (routes.TryGetValue(path, out int handler)) // no string allocated + Dispatch(handler); +``` + +## StringInternTable + +A canonicalizing table of strings: probe it with a `ReadOnlySpan` and it returns the one shared `string` for those characters, allocating **only on a miss**. + +```csharp +public sealed class StringInternTable : StringInternTable + +public class StringInternTable : IReadOnlyCollection + where THasher : struct, IHashProvider, ISpanHashProvider +``` + +### The documented BCL-beating workload + +**A 10M-cell CSV or log parse over ~100 distinct tokens.** The intern table allocates **100 strings instead of 10,000,000** — only the miss path materializes one. Downstream reference equality then works, and the GC never sees the other 9,999,900 copies. + +This is the one collection you cannot build on the pre-.NET-9 BCL: `HashSet.TryGetValue` takes a `string`, so you must **allocate the string before you can discover you already had it** — the very allocation you were trying to avoid. `string.Intern` is not a substitute either: it is process-wide, never collected for the life of the process, and still requires a `string` to hand it. A `StringInternTable` is an ordinary object — its scope is yours, `Clear` releases everything it holds, and dropping the table drops the interned strings with it. + +> **.NET 9+.** `Dictionary.GetAlternateLookup>()` can express the same pattern on .NET 9 and later. This type works on all three of Celerity's target frameworks, including the `net8.0` floor. + +### Default hasher + +The non-generic `StringInternTable` uses `StringFnV1AFullHasher` — cheap, and **full Unicode width**, so tokens that differ only in a character's high byte do not collide. (This deliberately differs from the frozen collections' `StringFnV1AHasher` default, which folds only the low byte: an intern table's inputs are arbitrary parsed text rather than curated identifiers.) A collision is never a correctness problem in either case — the ordinal span comparison resolves it — but it costs probes. Supply any `String*Hasher` through `StringInternTable` for a different speed/quality point. + +### Constructors + +```csharp +public StringInternTable(int capacity = 16, float loadFactor = 0.75f) +``` + +**Throws:** + +- `ArgumentOutOfRangeException` if `capacity` is negative, or `loadFactor` is not in the open interval (0, 1). + +### Methods and properties + +| Member | Description | +|--------|-------------| +| `int Count` | Number of distinct strings interned. | +| `string GetOrAdd(ReadOnlySpan key)` | The canonical instance for those characters, allocating one only on a miss. | +| `string GetOrAdd(string key)` | The canonical instance; on a miss the supplied instance itself becomes canonical, so this never allocates. Throws `ArgumentNullException` on `null`. | +| `bool TryGet(ReadOnlySpan key, out string? value)` | Pure lookup — never interns. `value` is `null` on a miss. | +| `bool Contains(ReadOnlySpan key)` | Whether those characters are already interned. | +| `bool Contains(string key)` | The same. Throws `ArgumentNullException` on `null`. | +| `void Clear()` | Drops every interned string; the backing capacity is preserved. | +| `Enumerator GetEnumerator()` | An allocation-free struct enumerator over the interned strings. Order is unspecified. | + +`GetOrAdd` (when it interns) and `Clear` are structural changes that invalidate an in-flight enumerator; `TryGet` and `Contains` never mutate. + +### Empty-string and `null` handling + +The empty string is an ordinary entry, and an empty span means `""`. `null` is not storable — the `string` overloads reject it. + +### Choosing it + +Reach for `StringInternTable` when you are producing **many occurrences of few distinct strings** from a buffer you already hold: parsers, tokenizers, log ingestion, column stores, header dispatch. If you need to associate a *value* with each token, use a `CelerityDictionary` with the [span lookups](#span-keyed-lookups) instead. It is not thread-safe. + +### Usage example + +```csharp +using Celerity.Collections; + +var interned = new StringInternTable(); + +foreach (ReadOnlySpan cell in SplitCells(line)) +{ + string token = interned.GetOrAdd(cell); // allocates only the first time each token is seen + Consume(token); +} + +Console.WriteLine(interned.Count); // distinct tokens, == strings allocated +``` + ## FenwickTree<T> ```csharp diff --git a/docs/api/hashing.md b/docs/api/hashing.md index 4af991bc..acfb4a28 100644 --- a/docs/api/hashing.md +++ b/docs/api/hashing.md @@ -138,6 +138,65 @@ public struct MyLongHasher : IHashProvider, IHashProvider64 --- +## ISpanHashProvider + +```csharp +public interface ISpanHashProvider +{ + int Hash(ReadOnlySpan key); +} +``` + +The span sibling of `IHashProvider`, so a caller holding a slice of a buffer can hash it — and probe a string-keyed collection with it — without first materializing a `string`. + +**Every built-in `String*Hasher` implements it.** Each already walks the characters, so the span overload *is* the body and the `string` overload delegates to it. The integer and `Guid` hashers do not: a character span is not their key shape. + +### The contract + +For every `string s`, an implementation must satisfy: + +```csharp +hasher.Hash(s) == hasher.Hash(s.AsSpan()) +``` + +This is load-bearing, not cosmetic. The [span lookups](collections.md#span-keyed-lookups) hash a span and then compare the result against keys that were *placed* using the `string` overload. A divergence would not merely be slow — it would report a stored key as absent. Share one body between the two overloads rather than maintaining two copies; `SpanHashParityTests` pins the contract for every built-in hasher across every length class, including as a slice of a larger buffer. + +A span has no `null` state: `default(ReadOnlySpan)` is empty, and empty means the empty string `""`. The `string` overloads keep throwing `ArgumentNullException` on `null` exactly as before. + +### Why it is a separate interface + +`IHashProvider.Hash(T)` is generic in its key type, and a `ref struct` such as `ReadOnlySpan` could not be used as a generic type argument before `allows ref struct` (C# 13 / .NET 9) — while `net8.0` is Celerity's floor. Expressing the span overload as a separate **non-generic** interface sidesteps that entirely, because the span is a method parameter rather than a type argument. The same reasoning gave `IHashProvider64` its independent shape. + +### Implementing it on a custom hasher + +Put the algorithm in the span overload and delegate: + +```csharp +public struct MyStringHasher : IHashProvider, ISpanHashProvider +{ + public int Hash(string key) + { + ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); // one body, so the two cannot drift + } + + public int Hash(ReadOnlySpan key) + { + uint hash = 2166136261u; + foreach (char c in key) + { + hash ^= c; + hash *= 16777619u; + } + return unchecked((int)hash); + } +} +``` + +Implementations must be structs, for the same devirtualization reason as `IHashProvider`. The collections' span overloads are constrained on `where THasher : struct, IHashProvider, ISpanHashProvider`, so a hasher that implements only one of the two simply will not bind there. + +--- + ## Built-in Hashers ### Int32IdentityHasher diff --git a/src/Celerity.AotSmokeTest/Program.cs b/src/Celerity.AotSmokeTest/Program.cs index aaec7ef4..7b04b5d0 100644 --- a/src/Celerity.AotSmokeTest/Program.cs +++ b/src/Celerity.AotSmokeTest/Program.cs @@ -761,6 +761,71 @@ void Check(bool condition, string message) Check(wide.Count == 0 && !wide.ContainsPrefix("key"), "Trie.Clear"); } +// Span-keyed lookups + StringInternTable. These are the paths where the JIT's devirtualization +// of a struct hasher through a *method*-level generic constraint (ISpanHashProvider on the +// SpanLookupExtensions methods, not on the collection's own type parameter) has to survive AOT +// compilation — every instantiation below is one the ILC has to see and generate. +{ + // The key sits inside a larger buffer, exactly as a parser would hold it: nothing here is + // ever turned into a string before the probe. + char[] buffer = "..alpha..beta..".ToCharArray(); + ReadOnlySpan alpha = buffer.AsSpan(2, 5); + ReadOnlySpan beta = buffer.AsSpan(9, 4); + ReadOnlySpan missing = "gamma".AsSpan(); + + var pairs = new[] + { + new KeyValuePair("alpha", 1), + new KeyValuePair("beta", 2), + }; + + var frozenDict = new FrozenCelerityDictionary(pairs); + Check(frozenDict.TryGetValue(alpha, out int fdA) && fdA == 1, "FrozenCelerityDictionary span TryGetValue"); + Check(frozenDict.ContainsKey(beta) && !frozenDict.ContainsKey(missing), "FrozenCelerityDictionary span ContainsKey"); + + var frozenSet = new FrozenCeleritySet(new[] { "alpha", "beta" }); + Check(frozenSet.Contains(alpha) && !frozenSet.Contains(missing), "FrozenCeleritySet span Contains"); + + var dict = new CelerityDictionary(); + dict.Add("alpha", 1); + dict.Add("beta", 2); + Check(dict.TryGetValue(beta, out int dB) && dB == 2, "CelerityDictionary span TryGetValue"); + Check(dict.ContainsKey(alpha) && !dict.ContainsKey(missing), "CelerityDictionary span ContainsKey"); + + var set = new CeleritySet(); + set.Add("alpha"); + Check(set.Contains(alpha) && !set.Contains(missing), "CeleritySet span Contains"); + + var spanTrie = new Trie(); + spanTrie["alpha"] = 1; + Check(spanTrie.TryGetValue(alpha, out int tA) && tA == 1, "Trie span TryGetValue"); + Check(spanTrie.ContainsKey(alpha) && spanTrie.ContainsPrefix("alp".AsSpan()), "Trie span ContainsKey/ContainsPrefix"); + + // Every String*Hasher must answer Hash(s) == Hash(s.AsSpan()) after AOT compilation too — + // the contract the span probes above are built on. + var spanHasher = new StringXxHash3Hasher(); + Check(spanHasher.Hash("alpha") == spanHasher.Hash(alpha), "ISpanHashProvider string/span parity"); + + // StringInternTable: the miss path allocates, every repeat returns the same reference. + var interned = new StringInternTable(); + string first = interned.GetOrAdd(alpha); + string second = interned.GetOrAdd("xxalphaxx".AsSpan(2, 5)); + Check(first == "alpha" && ReferenceEquals(first, second), "StringInternTable canonicalizes by contents"); + Check(interned.Count == 1 && interned.Contains(alpha) && !interned.Contains(missing), "StringInternTable Count/Contains"); + Check(interned.TryGet(alpha, out string? got) && ReferenceEquals(got, first), "StringInternTable.TryGet"); + interned.GetOrAdd(beta); + int internedSeen = 0; + foreach (string _ in interned) internedSeen++; + Check(internedSeen == 2, "StringInternTable struct enumerator"); + interned.Clear(); + Check(interned.Count == 0, "StringInternTable.Clear"); + + // A second hasher instantiation, so the ILC generates more than one closed generic. + var internedStrong = new StringInternTable(); + Check(ReferenceEquals(internedStrong.GetOrAdd(alpha), internedStrong.GetOrAdd(alpha)), + "StringInternTable canonicalizes"); +} + // EnumMap — dense array-backed dictionary for enum keys (the .NET EnumMap). Exercise // the indexer, TryAdd/Add, TryGetValue, Remove, the parallel occupancy vector // (default value distinct from absent), and the ascending-order struct enumerator. diff --git a/src/Celerity.Benchmarks/FrozenCelerityDictionaryBenchmark.cs b/src/Celerity.Benchmarks/FrozenCelerityDictionaryBenchmark.cs index 8ebe6b5e..0074ebae 100644 --- a/src/Celerity.Benchmarks/FrozenCelerityDictionaryBenchmark.cs +++ b/src/Celerity.Benchmarks/FrozenCelerityDictionaryBenchmark.cs @@ -11,6 +11,11 @@ public class FrozenCelerityDictionaryBenchmark private string[] keys = null!; private KeyValuePair[] pairs = null!; + // The same keys laid end to end in one buffer, with the slice bounds of each — the shape a parser or + // router actually holds its keys in, and the input to the SpanLookup category below. + private char[] buffer = null!; + private (int Offset, int Length)[] slices = null!; + private FrozenDictionary frozenDictionary = null!; private FrozenCelerityDictionary frozenCelerity = null!; @@ -22,13 +27,19 @@ public void Setup() { keys = new string[ItemCount]; pairs = new KeyValuePair[ItemCount]; + slices = new (int, int)[ItemCount]; + var text = new System.Text.StringBuilder(); for (int i = 0; i < ItemCount; i++) { // Identifier-shaped, guaranteed-distinct keys. keys[i] = "celerity/key/" + i + "/" + (i * 2654435761u); pairs[i] = new KeyValuePair(keys[i], i); + slices[i] = (text.Length, keys[i].Length); + text.Append(keys[i]).Append(' '); } + buffer = text.ToString().ToCharArray(); + frozenDictionary = pairs.ToFrozenDictionary(p => p.Key, p => p.Value); frozenCelerity = new FrozenCelerityDictionary(pairs); } @@ -70,4 +81,38 @@ public int FrozenCelerityDictionary_Lookup() acc += frozenCelerity[key]; return acc; } + + // ── SpanLookup (the caller holds spans, not strings) ────────────────────── + // The baseline is what a net8.0 caller must do to probe any string-keyed collection: allocate the + // string first. The Celerity arm probes the span directly, so the whole allocation and copy vanish — + // which is the point, and why this category is the one with allocation numbers worth reading. + // (.NET 9's Dictionary.GetAlternateLookup closes this gap on that runtime; this project + // targets net8.0, the floor where the BCL has no answer.) + + [Benchmark(Baseline = true)] + [BenchmarkCategory("SpanLookup")] + public int FrozenDictionary_SpanLookup() + { + int acc = 0; + for (int i = 0; i < slices.Length; i++) + { + (int offset, int length) = slices[i]; + acc += frozenDictionary[new string(buffer, offset, length)]; + } + return acc; + } + + [Benchmark] + [BenchmarkCategory("SpanLookup")] + public int FrozenCelerityDictionary_SpanLookup() + { + int acc = 0; + for (int i = 0; i < slices.Length; i++) + { + (int offset, int length) = slices[i]; + frozenCelerity.TryGetValue(buffer.AsSpan(offset, length), out int value); + acc += value; + } + return acc; + } } diff --git a/src/Celerity.Benchmarks/Program.cs b/src/Celerity.Benchmarks/Program.cs index 72aa01c9..0585bb7a 100644 --- a/src/Celerity.Benchmarks/Program.cs +++ b/src/Celerity.Benchmarks/Program.cs @@ -47,6 +47,7 @@ internal class Program typeof(DisjointSetBenchmark), typeof(IndexedPriorityQueueBenchmark), typeof(TrieBenchmark), + typeof(StringInternTableBenchmark), typeof(FenwickTreeBenchmark), typeof(BTreeDictionaryBenchmark), typeof(BTreeSetBenchmark), diff --git a/src/Celerity.Benchmarks/StringInternTableBenchmark.cs b/src/Celerity.Benchmarks/StringInternTableBenchmark.cs new file mode 100644 index 00000000..239992c4 --- /dev/null +++ b/src/Celerity.Benchmarks/StringInternTableBenchmark.cs @@ -0,0 +1,158 @@ +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Configs; +using Celerity.Collections; + +/// +/// against the BCL shapes a parser would otherwise reach for, over the +/// workload the type exists to serve: a token stream held as spans of one input buffer, with far fewer +/// distinct tokens than occurrences. +/// +/// +/// +/// The headline is the Allocated column, not the nanoseconds — which is why +/// [MemoryDiagnoser] reports bytes here. Every BCL arm must call new string(span) before it +/// can even ask whether it already had that token, so it allocates once per occurrence; +/// is probed with the span and allocates once per distinct token. +/// At the sweep's 64-token universe that is a 15×–1500× difference in strings created, and the gap widens +/// with the stream length. +/// +/// +/// The Dedupe category is the honest end-to-end comparison: all three arms end up holding one +/// canonical instance per distinct token, so they differ only in how much garbage they made getting there. +/// Lookup isolates the probe on an already-warm table, where the intern table's win is just the +/// deleted allocation and copy. +/// +/// +/// The .NET 9+ caveat. .NET 9 shipped +/// Dictionary<string,V>.GetAlternateLookup<ReadOnlySpan<char>>(), which closes +/// this gap for a plain dictionary on that runtime. It is not benchmarked here because this project +/// targets net8.0 only — the floor where the BCL has no answer at all, and the reason the type +/// exists. On .NET 9+ the two approaches are comparable; the library's value there is that the same code +/// works across all three of Celerity's target frameworks. +/// +/// +[MemoryDiagnoser] +[CategoriesColumn] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +public class StringInternTableBenchmark +{ + // Distinct tokens in the stream. Deliberately far smaller than ItemCount: the point of interning is + // that a long stream draws from a small vocabulary (column values, log levels, header names, …). + private const int DistinctTokens = 64; + + // One contiguous buffer holding the whole token stream, exactly as a parser would have it. The + // benchmarks slice it — they never hold the tokens as strings. + private char[] buffer = null!; + private (int Offset, int Length)[] tokens = null!; + + // A warm table / set for the Lookup category, built once in [GlobalSetup]. + private StringInternTable internedFull = null!; + private HashSet hashSetFull = null!; + + [Params(1000, 100_000)] + public int ItemCount; + + [GlobalSetup] + public void Setup() + { + var vocabulary = new string[DistinctTokens]; + for (int i = 0; i < DistinctTokens; i++) + vocabulary[i] = $"token_{i:D4}_value"; + + tokens = new (int, int)[ItemCount]; + var text = new System.Text.StringBuilder(ItemCount * 17); + for (int i = 0; i < ItemCount; i++) + { + string token = vocabulary[i % DistinctTokens]; + tokens[i] = (text.Length, token.Length); + text.Append(token).Append(','); // the separator a real parser would slice around + } + + buffer = text.ToString().ToCharArray(); + + internedFull = new StringInternTable(DistinctTokens); + hashSetFull = new HashSet(DistinctTokens, StringComparer.Ordinal); + foreach (string token in vocabulary) + { + internedFull.GetOrAdd(token); + hashSetFull.Add(token); + } + } + + // ---- Dedupe: walk the stream and end up holding one instance per distinct token ------------------- + + // The pre-.NET-9 BCL answer: materialize every occurrence, then let the set throw the duplicates away. + // Correct, and it allocates one string per occurrence. + [Benchmark(Baseline = true)] + [BenchmarkCategory("Dedupe")] + public int HashSet_Dedupe() + { + var set = new HashSet(StringComparer.Ordinal); + for (int i = 0; i < tokens.Length; i++) + { + (int offset, int length) = tokens[i]; + set.Add(new string(buffer, offset, length)); + } + return set.Count; + } + + // The same shape with a dictionary, which at least hands the canonical instance back — still one + // allocation per occurrence, because the key must exist before it can be looked up. + [Benchmark] + [BenchmarkCategory("Dedupe")] + public int Dictionary_Dedupe() + { + var map = new Dictionary(StringComparer.Ordinal); + for (int i = 0; i < tokens.Length; i++) + { + (int offset, int length) = tokens[i]; + string materialized = new string(buffer, offset, length); + if (!map.TryGetValue(materialized, out _)) + map[materialized] = materialized; + } + return map.Count; + } + + [Benchmark] + [BenchmarkCategory("Dedupe")] + public int StringInternTable_Dedupe() + { + var table = new StringInternTable(); + for (int i = 0; i < tokens.Length; i++) + { + (int offset, int length) = tokens[i]; + table.GetOrAdd(buffer.AsSpan(offset, length)); + } + return table.Count; + } + + // ---- Lookup: probe an already-warm table once per occurrence -------------------------------------- + + [Benchmark(Baseline = true)] + [BenchmarkCategory("Lookup")] + public int HashSet_Lookup() + { + int hits = 0; + for (int i = 0; i < tokens.Length; i++) + { + (int offset, int length) = tokens[i]; + if (hashSetFull.Contains(new string(buffer, offset, length))) + hits++; + } + return hits; + } + + [Benchmark] + [BenchmarkCategory("Lookup")] + public int StringInternTable_Lookup() + { + int hits = 0; + for (int i = 0; i < tokens.Length; i++) + { + (int offset, int length) = tokens[i]; + if (internedFull.Contains(buffer.AsSpan(offset, length))) + hits++; + } + return hits; + } +} diff --git a/src/Celerity.Benchmarks/TrieBenchmark.cs b/src/Celerity.Benchmarks/TrieBenchmark.cs index 19801099..f7d5f255 100644 --- a/src/Celerity.Benchmarks/TrieBenchmark.cs +++ b/src/Celerity.Benchmarks/TrieBenchmark.cs @@ -19,6 +19,11 @@ public class TrieBenchmark private string[] keys = null!; private string[] prefixes = null!; + // The same keys laid end to end in one buffer, with the slice bounds of each — the shape a tokenizer + // actually holds its keys in, and the input to the SpanLookup category below. + private char[] buffer = null!; + private (int Offset, int Length)[] slices = null!; + // Rebuilt per iteration by the [IterationSetup]s below for the build (Add) category. private Trie trie = null!; private Dictionary dict = null!; @@ -38,8 +43,16 @@ public void Setup() prefixes[b] = $"{(char)('a' + b / 26)}{(char)('a' + b % 26)}"; keys = new string[ItemCount]; + slices = new (int, int)[ItemCount]; + var text = new System.Text.StringBuilder(); for (int i = 0; i < ItemCount; i++) + { keys[i] = $"{prefixes[i % PrefixBuckets]}_{i:D8}"; + slices[i] = (text.Length, keys[i].Length); + text.Append(keys[i]).Append(' '); + } + + buffer = text.ToString().ToCharArray(); trieFull = new Trie(); dictFull = new Dictionary(ItemCount); @@ -128,4 +141,39 @@ public long Trie_PrefixMatch() } return acc; } + + // ---- SpanLookup: the caller holds spans, not strings ---------------------------------------------- + // The baseline is what a net8.0 caller must do to probe any string-keyed collection: allocate the + // string first. The trie descends the span directly, so the whole allocation and copy vanish — which + // is why this category is the one with allocation numbers worth reading. It also flips the Lookup + // verdict: the trie's character walk no longer competes against a bare hash, but against a hash plus + // an allocation. (.NET 9's Dictionary.GetAlternateLookup closes this gap on that runtime; + // this project targets net8.0, the floor where the BCL has no answer.) + + [Benchmark(Baseline = true)] + [BenchmarkCategory("SpanLookup")] + public long Dictionary_SpanLookup() + { + long acc = 0; + for (int i = 0; i < slices.Length; i++) + { + (int offset, int length) = slices[i]; + acc += dictFull[new string(buffer, offset, length)]; + } + return acc; + } + + [Benchmark] + [BenchmarkCategory("SpanLookup")] + public long Trie_SpanLookup() + { + long acc = 0; + for (int i = 0; i < slices.Length; i++) + { + (int offset, int length) = slices[i]; + trieFull.TryGetValue(buffer.AsSpan(offset, length), out int value); + acc += value; + } + return acc; + } } diff --git a/src/Celerity.Fuzz/Differential.cs b/src/Celerity.Fuzz/Differential.cs index 1ae3ec0f..3e713a75 100644 --- a/src/Celerity.Fuzz/Differential.cs +++ b/src/Celerity.Fuzz/Differential.cs @@ -45,6 +45,7 @@ public static readonly (string Name, Action Run)[] All = ("CelerityMultiMap", CelerityMultiMapCase), ("FrozenCelerityDictionary", FrozenCase), ("FrozenCeleritySet", FrozenSetCase), + ("StringInternTable", StringInternTableCase), ("BloomFilter", BloomFilterCase), ("CuckooFilter", CuckooFilterCase), ("XorFilter", XorFilterCase), @@ -1237,6 +1238,72 @@ private static void FrozenCase(Random rng) Check(seen == oracle.Count, $"enumeration count {seen} != {oracle.Count}"); } + private static void StringInternTableCase(Random rng) + { + // A tiny, duplicate-rich token universe so hits dominate misses and probe chains stay + // dense; the table starts undersized so the run drives several resizes. + var table = new StringInternTable(capacity: 2); + + // The oracle maps a token's contents to the canonical instance first handed out. + var oracle = new Dictionary(StringComparer.Ordinal); + + int steps = rng.Next(0, 400); + for (int i = 0; i < steps; i++) + { + string token = $"tok_{rng.Next(0, 30)}"; + int op = rng.Next(100); + + if (op < 50) + { + // Intern from a span carved out of a larger buffer — the parser shape. Only a + // miss may materialize a string; a hit must return the instance already held. + string padded = $"<<{token}>>"; + string interned = table.GetOrAdd(padded.AsSpan(2, token.Length)); + + Check(interned == token, $"GetOrAdd(span) returned {interned} for {token}"); + if (oracle.TryGetValue(token, out string? canonical)) + Check(ReferenceEquals(canonical, interned), $"GetOrAdd(span) re-allocated {token}"); + else + oracle[token] = interned; + } + else if (op < 70) + { + string supplied = new string(token.ToCharArray()); + string interned = table.GetOrAdd(supplied); + + if (oracle.TryGetValue(token, out string? canonical)) + Check(ReferenceEquals(canonical, interned), $"GetOrAdd(string) re-allocated {token}"); + else + { + Check(ReferenceEquals(supplied, interned), $"GetOrAdd(string) did not adopt {token}"); + oracle[token] = interned; + } + } + else if (op < 90) + { + bool expected = oracle.TryGetValue(token, out string? canonical); + bool actual = table.TryGet(token.AsSpan(), out string? found); + Check(expected == actual, $"TryGet({token}) {actual} != {expected}"); + Check(!expected || ReferenceEquals(canonical, found), $"TryGet({token}) returned a non-canonical instance"); + Check(table.Contains(token.AsSpan()) == expected, $"Contains(span {token}) != {expected}"); + Check(table.Contains(token) == expected, $"Contains(string {token}) != {expected}"); + } + else + { + int seen = 0; + foreach (string s in table) + { + Check(oracle.TryGetValue(s, out string? canonical) && ReferenceEquals(canonical, s), + $"enumeration yielded a non-canonical or absent {s}"); + seen++; + } + Check(seen == oracle.Count, $"enumeration count {seen} != {oracle.Count}"); + } + + Check(table.Count == oracle.Count, $"Count {table.Count} != {oracle.Count}"); + } + } + private static void FrozenSetCase(Random rng) { // A deliberately tiny, duplicate-rich element universe so the build's dedupe diff --git a/src/Celerity.Hashing/ISpanHashProvider.cs b/src/Celerity.Hashing/ISpanHashProvider.cs new file mode 100644 index 00000000..cbaf4ac9 --- /dev/null +++ b/src/Celerity.Hashing/ISpanHashProvider.cs @@ -0,0 +1,44 @@ +namespace Celerity.Hashing; + +/// +/// Provides a hash function over a of , +/// so a caller holding a slice of an existing buffer can probe a string-keyed collection +/// without first materializing a . +/// +/// +/// +/// The contract. For every string s an implementation must satisfy +/// Hash(s) == Hash(s.AsSpan()). A collection that probes with a span and compares +/// against stored keys relies on this: a divergence would not merely +/// be slow, it would report a stored key as absent. Implementations are expected to share +/// a single body between the two overloads rather than maintain two copies. +/// +/// +/// This interface deliberately does not derive from +/// , mirroring the independent +/// sibling: is generic in +/// its key type, and a ref struct such as could not be +/// used as a generic type argument before allows ref struct (C# 13 / .NET 9) — +/// while net8.0 remains this library's floor. Expressing the span overload as a +/// separate non-generic interface sidesteps that entirely, because the span is a method +/// parameter rather than a type argument. In practice every built-in +/// String*Hasher implements both, so a single struct serves both call shapes. +/// +/// +/// Like , implementations must be value types (structs) so +/// the JIT can devirtualize and inline calls made through a generic type parameter +/// constrained to where THasher : struct, IHashProvider<string>, ISpanHashProvider. +/// +/// +public interface ISpanHashProvider +{ + /// + /// Computes a hash code for the specified character span. + /// + /// The characters to hash. + /// + /// A 32-bit signed integer hash code, equal to the code the same hasher's + /// Hash(string) overload returns for a string with the same contents. + /// + int Hash(ReadOnlySpan key); +} diff --git a/src/Celerity.Hashing/StringAdler32Hasher.cs b/src/Celerity.Hashing/StringAdler32Hasher.cs index 77bbae40..a7d77755 100644 --- a/src/Celerity.Hashing/StringAdler32Hasher.cs +++ b/src/Celerity.Hashing/StringAdler32Hasher.cs @@ -66,7 +66,7 @@ namespace Celerity.Hashing; /// sentinel. /// /// -public struct StringAdler32Hasher : IHashProvider +public struct StringAdler32Hasher : IHashProvider, ISpanHashProvider { /// /// The Adler-32 modulus: 65521, the largest prime below 2^16. @@ -89,7 +89,21 @@ public struct StringAdler32Hasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the standard Adler-32 (RFC 1950 / zlib) of the specified character span + /// over the full little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit Adler-32 of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { uint a = 1u; uint b = 0u; diff --git a/src/Celerity.Hashing/StringCityHash64Hasher.cs b/src/Celerity.Hashing/StringCityHash64Hasher.cs index 9abc5f3c..74291203 100644 --- a/src/Celerity.Hashing/StringCityHash64Hasher.cs +++ b/src/Celerity.Hashing/StringCityHash64Hasher.cs @@ -67,11 +67,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringCityHash64Hasher : IHashProvider, IHashProvider64 +public struct StringCityHash64Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { // CityHash v1.1 mixing constants (Geoff Pike & Jyrki Alakuijala, public domain). private const ulong K0 = 0xC3A5C85C97CB3127UL; @@ -96,7 +96,23 @@ public struct StringCityHash64Hasher : IHashProvider, IHashProvider64 + /// Computes the CityHash64 hash of the specified character span, xor-folded to a + /// signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded CityHash64 hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -117,8 +133,16 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } - int charLen = key.Length; // count of UTF-16 code units (chars) + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { + int charLen = key.Length; // count of UTF-16 code units (chars) ulong byteLen = (ulong)charLen * 2UL; // CityHash operates on the byte length ulong h = byteLen <= 32UL @@ -133,7 +157,7 @@ public ulong Hash64(string key) // ── Length-classed hash bodies (mirror CityHash64's dispatch) ─────────────── - private static ulong HashLen0to16(string key, int charLen, ulong byteLen) + private static ulong HashLen0to16(ReadOnlySpan key, int charLen, ulong byteLen) { if (byteLen >= 8UL) { @@ -168,7 +192,7 @@ private static ulong HashLen0to16(string key, int charLen, ulong byteLen) return K2; } - private static ulong HashLen17to32(string key, int charLen, ulong byteLen) + private static ulong HashLen17to32(ReadOnlySpan key, int charLen, ulong byteLen) { ulong mul = K2 + byteLen * 2UL; ulong a = Lane(key, 0) * K1; @@ -181,7 +205,7 @@ private static ulong HashLen17to32(string key, int charLen, ulong byteLen) mul); } - private static ulong HashLen33to64(string key, int charLen, ulong byteLen) + private static ulong HashLen33to64(ReadOnlySpan key, int charLen, ulong byteLen) { ulong mul = K2 + byteLen * 2UL; ulong a = Lane(key, 0) * K2; @@ -204,7 +228,7 @@ private static ulong HashLen33to64(string key, int charLen, ulong byteLen) return b + x; } - private static ulong HashLong(string key, int charLen, ulong byteLen) + private static ulong HashLong(ReadOnlySpan key, int charLen, ulong byteLen) { // For strings over 64 bytes CityHash hashes the end first, then keeps 56 // bytes of state (v, w, x, y, z) and consumes 64 bytes (thirty-two chars) @@ -251,7 +275,7 @@ private static ulong HashLong(string key, int charLen, ulong byteLen) /// read over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Lane(string key, int i) => + private static ulong Lane(ReadOnlySpan key, int i) => (ulong)key[i] | ((ulong)key[i + 1] << 16) | ((ulong)key[i + 2] << 32) @@ -263,7 +287,7 @@ private static ulong Lane(string key, int i) => /// 16 bits, the next char the high 16 bits. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Block(string key, int i) => + private static ulong Block(ReadOnlySpan key, int i) => (uint)key[i] | ((uint)key[i + 1] << 16); /// CityHash's ShiftMix: val ^ (val >> 47). @@ -296,7 +320,7 @@ private static ulong HashLen16(ulong u, ulong v, ulong mul) /// seeds, returning the two-word result. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static (ulong, ulong) WeakHashLen32WithSeeds(string key, int i, ulong a, ulong b) + private static (ulong, ulong) WeakHashLen32WithSeeds(ReadOnlySpan key, int i, ulong a, ulong b) { ulong w = Lane(key, i); ulong x = Lane(key, i + 4); diff --git a/src/Celerity.Hashing/StringCrc32Hasher.cs b/src/Celerity.Hashing/StringCrc32Hasher.cs index fabac101..3d936bdb 100644 --- a/src/Celerity.Hashing/StringCrc32Hasher.cs +++ b/src/Celerity.Hashing/StringCrc32Hasher.cs @@ -68,7 +68,7 @@ namespace Celerity.Hashing; /// with the empty-slot sentinel. /// /// -public struct StringCrc32Hasher : IHashProvider +public struct StringCrc32Hasher : IHashProvider, ISpanHashProvider { /// The reflected CRC-32 (ISO-HDLC / IEEE 802.3) generator polynomial. private const uint Polynomial = 0xEDB88320u; @@ -114,7 +114,22 @@ private static uint[] BuildTable() public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the standard CRC-32 (ISO-HDLC / IEEE 802.3 / zlib) of the specified + /// character span over the full little-endian UTF-16 byte stream (both bytes of + /// every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit CRC-32 of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { uint[] table = Table; uint crc = 0xFFFFFFFFu; diff --git a/src/Celerity.Hashing/StringDjb2AHasher.cs b/src/Celerity.Hashing/StringDjb2AHasher.cs index 477478be..fc935286 100644 --- a/src/Celerity.Hashing/StringDjb2AHasher.cs +++ b/src/Celerity.Hashing/StringDjb2AHasher.cs @@ -66,7 +66,7 @@ namespace Celerity.Hashing; /// sentinel. /// /// -public struct StringDjb2AHasher : IHashProvider +public struct StringDjb2AHasher : IHashProvider, ISpanHashProvider { /// /// Computes the djb2a (XOR-folding) hash of the specified string over the @@ -84,7 +84,21 @@ public struct StringDjb2AHasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the djb2a (XOR-folding) hash of the specified character span over + /// the full little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit djb2a hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { uint hash = 5381u; foreach (char c in key) { diff --git a/src/Celerity.Hashing/StringDjb2Hasher.cs b/src/Celerity.Hashing/StringDjb2Hasher.cs index 05e033e7..63e1cf6c 100644 --- a/src/Celerity.Hashing/StringDjb2Hasher.cs +++ b/src/Celerity.Hashing/StringDjb2Hasher.cs @@ -51,7 +51,7 @@ namespace Celerity.Hashing; /// the hasher, so this does not collide with the empty-slot sentinel. /// /// -public struct StringDjb2Hasher : IHashProvider +public struct StringDjb2Hasher : IHashProvider, ISpanHashProvider { /// /// Computes Daniel J. Bernstein's djb2 hash of the specified string over the @@ -69,7 +69,21 @@ public struct StringDjb2Hasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes Daniel J. Bernstein's djb2 hash of the specified character span + /// over the full little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit djb2 hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { uint hash = 5381u; foreach (char c in key) { diff --git a/src/Celerity.Hashing/StringElfHasher.cs b/src/Celerity.Hashing/StringElfHasher.cs index 7d3863e4..493355e3 100644 --- a/src/Celerity.Hashing/StringElfHasher.cs +++ b/src/Celerity.Hashing/StringElfHasher.cs @@ -62,7 +62,7 @@ namespace Celerity.Hashing; /// calling the hasher, so this does not collide with the empty-slot sentinel. /// /// -public struct StringElfHasher : IHashProvider +public struct StringElfHasher : IHashProvider, ISpanHashProvider { /// /// Computes the PJW / ELF hash of the specified string over the full @@ -81,7 +81,22 @@ public struct StringElfHasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the PJW / ELF hash of the specified character span over the full + /// little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit ELF hash of (always in the range + /// [0, 0x0FFFFFFF], since the algorithm clears the top nibble) — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { uint hash = 0u; foreach (char c in key) { diff --git a/src/Celerity.Hashing/StringFnV164Hasher.cs b/src/Celerity.Hashing/StringFnV164Hasher.cs index 4abda560..5181527c 100644 --- a/src/Celerity.Hashing/StringFnV164Hasher.cs +++ b/src/Celerity.Hashing/StringFnV164Hasher.cs @@ -65,11 +65,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringFnV164Hasher : IHashProvider, IHashProvider64 +public struct StringFnV164Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { /// /// Computes the FNV-1 64-bit hash of the specified string over the full @@ -87,7 +87,23 @@ public struct StringFnV164Hasher : IHashProvider, IHashProvider64 + /// Computes the FNV-1 64-bit hash of the specified character span, xor-folded to a + /// signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded FNV-1 hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -108,7 +124,15 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { // The FNV-1 64-bit parameters (identical to FNV-1a 64-bit). const ulong fnvPrime = 1099511628211UL; const ulong offsetBasis = 14695981039346656037UL; diff --git a/src/Celerity.Hashing/StringFnV1A64Hasher.cs b/src/Celerity.Hashing/StringFnV1A64Hasher.cs index cca23992..cac080c5 100644 --- a/src/Celerity.Hashing/StringFnV1A64Hasher.cs +++ b/src/Celerity.Hashing/StringFnV1A64Hasher.cs @@ -45,11 +45,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringFnV1A64Hasher : IHashProvider, IHashProvider64 +public struct StringFnV1A64Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { /// /// Computes the FNV-1a 64-bit hash of the specified string over the full @@ -67,7 +67,23 @@ public struct StringFnV1A64Hasher : IHashProvider, IHashProvider64 + /// Computes the FNV-1a 64-bit hash of the specified character span, xor-folded to a + /// signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded FNV-1a hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -88,7 +104,15 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { // The FNV-1a 64-bit parameters. const ulong fnvPrime = 1099511628211UL; const ulong offsetBasis = 14695981039346656037UL; diff --git a/src/Celerity.Hashing/StringFnV1AFullHasher.cs b/src/Celerity.Hashing/StringFnV1AFullHasher.cs index d4ebc4fd..c77a4721 100644 --- a/src/Celerity.Hashing/StringFnV1AFullHasher.cs +++ b/src/Celerity.Hashing/StringFnV1AFullHasher.cs @@ -41,7 +41,7 @@ namespace Celerity.Hashing; /// the hasher, so this does not collide with the empty-slot sentinel. /// /// -public struct StringFnV1AFullHasher : IHashProvider +public struct StringFnV1AFullHasher : IHashProvider, ISpanHashProvider { /// /// Computes the FNV-1a 32-bit hash of the specified string over the full @@ -59,7 +59,21 @@ public struct StringFnV1AFullHasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the FNV-1a 32-bit hash of the specified character span over the full + /// little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit FNV-1a hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { // The FNV-1a 32-bit parameters. const uint fnvPrime = 16777619u; const uint offsetBasis = 2166136261u; diff --git a/src/Celerity.Hashing/StringFnV1AHasher.cs b/src/Celerity.Hashing/StringFnV1AHasher.cs index 262d0711..aeb96b8b 100644 --- a/src/Celerity.Hashing/StringFnV1AHasher.cs +++ b/src/Celerity.Hashing/StringFnV1AHasher.cs @@ -11,7 +11,7 @@ namespace Celerity.Hashing; /// so strings that differ only in high bytes of non-ASCII characters may collide. /// For Unicode-heavy workloads, a full UTF-8 or UTF-16 hash is preferable. /// -public struct StringFnV1AHasher : IHashProvider +public struct StringFnV1AHasher : IHashProvider, ISpanHashProvider { /// /// Computes the FNV-1a 32-bit hash of the specified string. @@ -28,7 +28,20 @@ public struct StringFnV1AHasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the FNV-1a 32-bit hash of the specified character span. + /// + /// The characters to hash. + /// + /// The signed 32-bit FNV-1a hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { // The FNV-1a 32-bit parameters const uint fnvPrime = 16777619; const uint offsetBasis = 2166136261; diff --git a/src/Celerity.Hashing/StringFnV1Hasher.cs b/src/Celerity.Hashing/StringFnV1Hasher.cs index 13ee81e0..31371bfa 100644 --- a/src/Celerity.Hashing/StringFnV1Hasher.cs +++ b/src/Celerity.Hashing/StringFnV1Hasher.cs @@ -53,7 +53,7 @@ namespace Celerity.Hashing; /// the hasher, so this does not collide with the empty-slot sentinel. /// /// -public struct StringFnV1Hasher : IHashProvider +public struct StringFnV1Hasher : IHashProvider, ISpanHashProvider { /// /// Computes the FNV-1 32-bit hash of the specified string over the full @@ -71,7 +71,21 @@ public struct StringFnV1Hasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the FNV-1 32-bit hash of the specified character span over the full + /// little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit FNV-1 hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { // The FNV-1 32-bit parameters (identical to FNV-1a). const uint fnvPrime = 16777619u; const uint offsetBasis = 2166136261u; diff --git a/src/Celerity.Hashing/StringHalfSipHash24Hasher.cs b/src/Celerity.Hashing/StringHalfSipHash24Hasher.cs index bd27a5dd..c2a0c4ca 100644 --- a/src/Celerity.Hashing/StringHalfSipHash24Hasher.cs +++ b/src/Celerity.Hashing/StringHalfSipHash24Hasher.cs @@ -73,7 +73,7 @@ namespace Celerity.Hashing; /// hasher, so this does not collide with the empty-slot sentinel. /// /// -public struct StringHalfSipHash24Hasher : IHashProvider +public struct StringHalfSipHash24Hasher : IHashProvider, ISpanHashProvider { // Canonical HalfSipHash reference key (test-vector key, bytes 00..07), read as // two little-endian 32-bit halves. Fixed because collections build the hasher @@ -103,7 +103,22 @@ public struct StringHalfSipHash24Hasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the HalfSipHash-2-4 hash of the specified character span over its native + /// little-endian UTF-16 byte stream (using this type's fixed built-in key), returning + /// the native 32-bit result as a signed integer. + /// + /// The characters to hash. + /// + /// The signed 32-bit HalfSipHash-2-4 hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { int charLen = key.Length; // count of UTF-16 code units (chars) uint byteLen = (uint)charLen * 2U; // HalfSipHash operates on the byte length @@ -156,7 +171,7 @@ public int Hash(string key) /// HalfSipHash would read over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static uint Word(string key, int i) => + private static uint Word(ReadOnlySpan key, int i) => key[i] | ((uint)key[i + 1] << 16); /// diff --git a/src/Celerity.Hashing/StringHighwayHash64Hasher.cs b/src/Celerity.Hashing/StringHighwayHash64Hasher.cs index efbb0122..c4da046b 100644 --- a/src/Celerity.Hashing/StringHighwayHash64Hasher.cs +++ b/src/Celerity.Hashing/StringHighwayHash64Hasher.cs @@ -80,11 +80,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringHighwayHash64Hasher : IHashProvider, IHashProvider64 +public struct StringHighwayHash64Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { // Fixed 256-bit key: the canonical HighwayHash reference test key (bytes // 00..1f read as four little-endian 64-bit words). Fixed because collections @@ -110,7 +110,23 @@ public struct StringHighwayHash64Hasher : IHashProvider, IHashProvider64 [MethodImpl(MethodImplOptions.AggressiveInlining)] public int Hash(string key) { - ulong h64 = Hash64(key); + ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + + /// + /// Computes the HighwayHash-64 hash of the specified character span (using this type's + /// fixed built-in key), xor-folded to a signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded HighwayHash-64 hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -131,7 +147,15 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { // Four lanes of 64-bit state, held on the stack (no heap allocation). Span v0 = stackalloc ulong[4]; Span v1 = stackalloc ulong[4]; @@ -277,7 +301,7 @@ private static void PermuteAndUpdate( /// [MethodImpl(MethodImplOptions.AggressiveInlining)] private static void UpdateRemainder( - string key, int tailStartChar, int sizeMod32, + ReadOnlySpan key, int tailStartChar, int sizeMod32, Span v0, Span v1, Span mul0, Span mul1) { int sizeMod4 = sizeMod32 & 3; // 0 or 2 on an always-even byte stream @@ -340,7 +364,7 @@ private static void Rotate32By(uint count, Span v1) /// over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Lane(string key, int i) => + private static ulong Lane(ReadOnlySpan key, int i) => (ulong)key[i] | ((ulong)key[i + 1] << 16) | ((ulong)key[i + 2] << 32) @@ -353,7 +377,7 @@ private static ulong Lane(string key, int i) => /// offsets its high byte. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static byte TailByte(string key, int tailStartChar, int b) => + private static byte TailByte(ReadOnlySpan key, int tailStartChar, int b) => (byte)(key[tailStartChar + (b >> 1)] >> ((b & 1) << 3)); /// Rotates a 64-bit value by 32 bits (swaps its two 32-bit halves). diff --git a/src/Celerity.Hashing/StringJenkinsOaatHasher.cs b/src/Celerity.Hashing/StringJenkinsOaatHasher.cs index 0d7f4bf4..ac067e41 100644 --- a/src/Celerity.Hashing/StringJenkinsOaatHasher.cs +++ b/src/Celerity.Hashing/StringJenkinsOaatHasher.cs @@ -50,7 +50,7 @@ namespace Celerity.Hashing; /// does not collide with the empty-slot sentinel. /// /// -public struct StringJenkinsOaatHasher : IHashProvider +public struct StringJenkinsOaatHasher : IHashProvider, ISpanHashProvider { /// /// Computes Bob Jenkins' one-at-a-time hash of the specified string over the @@ -68,7 +68,21 @@ public struct StringJenkinsOaatHasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes Bob Jenkins' one-at-a-time hash of the specified character span over + /// the full little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit one-at-a-time hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { uint hash = 0u; foreach (char c in key) { diff --git a/src/Celerity.Hashing/StringMetroHash64Hasher.cs b/src/Celerity.Hashing/StringMetroHash64Hasher.cs index 94d25869..2b6efbb5 100644 --- a/src/Celerity.Hashing/StringMetroHash64Hasher.cs +++ b/src/Celerity.Hashing/StringMetroHash64Hasher.cs @@ -57,11 +57,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringMetroHash64Hasher : IHashProvider, IHashProvider64 +public struct StringMetroHash64Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { // metrohash64_1 mixing constants (J. Andrew Rogers, public-domain reference). private const ulong K0 = 0xC83A91E1UL; @@ -85,7 +85,23 @@ public struct StringMetroHash64Hasher : IHashProvider, IHashProvider64 + /// Computes the MetroHash64 hash of the specified character span, xor-folded to a + /// signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded MetroHash64 hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -106,8 +122,16 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } - int length = key.Length; // count of UTF-16 code units (chars) + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { + int length = key.Length; // count of UTF-16 code units (chars) ulong byteLength = (ulong)length * 2UL; // MetroHash mixes in the byte length // Seed 0: hash starts at ((0 + k2) * k0) + len. @@ -185,7 +209,7 @@ public ulong Hash64(string key) /// read over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Lane(string key, int i) => + private static ulong Lane(ReadOnlySpan key, int i) => (ulong)key[i] | ((ulong)key[i + 1] << 16) | ((ulong)key[i + 2] << 32) @@ -197,6 +221,6 @@ private static ulong Lane(string key, int i) => /// 16 bits, the next char the high 16 bits. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static uint Block(string key, int i) => + private static uint Block(ReadOnlySpan key, int i) => (uint)key[i] | ((uint)key[i + 1] << 16); } diff --git a/src/Celerity.Hashing/StringMurmur2Hasher.cs b/src/Celerity.Hashing/StringMurmur2Hasher.cs index 9e188a0d..bdd3d2ef 100644 --- a/src/Celerity.Hashing/StringMurmur2Hasher.cs +++ b/src/Celerity.Hashing/StringMurmur2Hasher.cs @@ -56,7 +56,7 @@ namespace Celerity.Hashing; /// sentinel. /// /// -public struct StringMurmur2Hasher : IHashProvider +public struct StringMurmur2Hasher : IHashProvider, ISpanHashProvider { private const uint M = 0x5bd1e995u; private const int R = 24; @@ -78,7 +78,21 @@ public struct StringMurmur2Hasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the MurmurHash2 (32-bit, seed 0) hash of the specified character + /// span over the full little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit MurmurHash2 hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { int length = key.Length; // MurmurHash2 seeds the accumulator with seed ^ byteLength; seed is 0 here, diff --git a/src/Celerity.Hashing/StringMurmur3Hasher.cs b/src/Celerity.Hashing/StringMurmur3Hasher.cs index fd29f91a..97f4a8f8 100644 --- a/src/Celerity.Hashing/StringMurmur3Hasher.cs +++ b/src/Celerity.Hashing/StringMurmur3Hasher.cs @@ -40,7 +40,7 @@ namespace Celerity.Hashing; /// calling the hasher, so this does not collide with the empty-slot sentinel. /// /// -public struct StringMurmur3Hasher : IHashProvider +public struct StringMurmur3Hasher : IHashProvider, ISpanHashProvider { private const uint C1 = 0xcc9e2d51u; private const uint C2 = 0x1b873593u; @@ -61,7 +61,21 @@ public struct StringMurmur3Hasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the MurmurHash3 x86_32 hash of the specified character span over its + /// UTF-16 code units. + /// + /// The characters to hash. + /// + /// The signed 32-bit MurmurHash3 hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { uint h = 0; int length = key.Length; diff --git a/src/Celerity.Hashing/StringSdbmHasher.cs b/src/Celerity.Hashing/StringSdbmHasher.cs index cdcea7cf..3962d448 100644 --- a/src/Celerity.Hashing/StringSdbmHasher.cs +++ b/src/Celerity.Hashing/StringSdbmHasher.cs @@ -59,7 +59,7 @@ namespace Celerity.Hashing; /// calling the hasher, so this does not collide with the empty-slot sentinel. /// /// -public struct StringSdbmHasher : IHashProvider +public struct StringSdbmHasher : IHashProvider, ISpanHashProvider { /// /// Computes the sdbm hash of the specified string over the full little-endian @@ -77,7 +77,21 @@ public struct StringSdbmHasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the sdbm hash of the specified character span over the full + /// little-endian UTF-16 byte stream (both bytes of every character). + /// + /// The characters to hash. + /// + /// The signed 32-bit sdbm hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { uint hash = 0u; foreach (char c in key) { diff --git a/src/Celerity.Hashing/StringSipHash13Hasher.cs b/src/Celerity.Hashing/StringSipHash13Hasher.cs index e10df40c..4287ed68 100644 --- a/src/Celerity.Hashing/StringSipHash13Hasher.cs +++ b/src/Celerity.Hashing/StringSipHash13Hasher.cs @@ -81,11 +81,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringSipHash13Hasher : IHashProvider, IHashProvider64 +public struct StringSipHash13Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { // Canonical SipHash reference key (RFC-draft test-vector key, bytes 00..0f), // read as two little-endian 64-bit halves. Fixed because collections build the @@ -115,7 +115,23 @@ public struct StringSipHash13Hasher : IHashProvider, IHashProvider64 + /// Computes the SipHash-1-3 hash of the specified character span (using this type's + /// fixed built-in key), xor-folded to a signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded SipHash-1-3 hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -136,8 +152,16 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } - int charLen = key.Length; // count of UTF-16 code units (chars) + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { + int charLen = key.Length; // count of UTF-16 code units (chars) ulong byteLen = (ulong)charLen * 2UL; // SipHash operates on the byte length ulong v0 = Init0 ^ K0; @@ -190,7 +214,7 @@ public ulong Hash64(string key) /// would read over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Word(string key, int i) => + private static ulong Word(ReadOnlySpan key, int i) => (ulong)key[i] | ((ulong)key[i + 1] << 16) | ((ulong)key[i + 2] << 32) diff --git a/src/Celerity.Hashing/StringSipHash24Hasher.cs b/src/Celerity.Hashing/StringSipHash24Hasher.cs index 51370f45..a7d1d525 100644 --- a/src/Celerity.Hashing/StringSipHash24Hasher.cs +++ b/src/Celerity.Hashing/StringSipHash24Hasher.cs @@ -75,11 +75,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringSipHash24Hasher : IHashProvider, IHashProvider64 +public struct StringSipHash24Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { // Canonical SipHash reference key (RFC-draft test-vector key, bytes 00..0f), // read as two little-endian 64-bit halves. Fixed because collections build the @@ -109,7 +109,23 @@ public struct StringSipHash24Hasher : IHashProvider, IHashProvider64 + /// Computes the SipHash-2-4 hash of the specified character span (using this type's + /// fixed built-in key), xor-folded to a signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded SipHash-2-4 hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -130,8 +146,16 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } - int charLen = key.Length; // count of UTF-16 code units (chars) + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { + int charLen = key.Length; // count of UTF-16 code units (chars) ulong byteLen = (ulong)charLen * 2UL; // SipHash operates on the byte length ulong v0 = Init0 ^ K0; @@ -186,7 +210,7 @@ public ulong Hash64(string key) /// would read over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Word(string key, int i) => + private static ulong Word(ReadOnlySpan key, int i) => (ulong)key[i] | ((ulong)key[i + 1] << 16) | ((ulong)key[i + 2] << 32) diff --git a/src/Celerity.Hashing/StringXxHash32Hasher.cs b/src/Celerity.Hashing/StringXxHash32Hasher.cs index 497f7b67..1903b5ca 100644 --- a/src/Celerity.Hashing/StringXxHash32Hasher.cs +++ b/src/Celerity.Hashing/StringXxHash32Hasher.cs @@ -47,7 +47,7 @@ namespace Celerity.Hashing; /// the hasher, so this does not collide with the empty-slot sentinel. /// /// -public struct StringXxHash32Hasher : IHashProvider +public struct StringXxHash32Hasher : IHashProvider, ISpanHashProvider { private const uint Prime1 = 2654435761u; private const uint Prime2 = 2246822519u; @@ -71,7 +71,21 @@ public struct StringXxHash32Hasher : IHashProvider public int Hash(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash(key.AsSpan()); + } + /// + /// Computes the xxHash32 hash (seed 0) of the specified character span over + /// its native little-endian UTF-16 byte stream. + /// + /// The characters to hash. + /// + /// The signed 32-bit xxHash32 hash of — the same value + /// returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { int length = key.Length; // count of UTF-16 code units (chars) uint byteLength = (uint)length * 2u; // XXH32 mixes in the byte length @@ -147,7 +161,7 @@ public int Hash(string key) /// byte-oriented XXH32 would read over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static uint Block(string key, int i) => + private static uint Block(ReadOnlySpan key, int i) => (uint)key[i] | ((uint)key[i + 1] << 16); /// The XXH32 accumulator round: rotl(acc + lane * PRIME32_2, 13) * PRIME32_1. diff --git a/src/Celerity.Hashing/StringXxHash3Hasher.cs b/src/Celerity.Hashing/StringXxHash3Hasher.cs index 82e26850..3351357c 100644 --- a/src/Celerity.Hashing/StringXxHash3Hasher.cs +++ b/src/Celerity.Hashing/StringXxHash3Hasher.cs @@ -63,11 +63,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringXxHash3Hasher : IHashProvider, IHashProvider64 +public struct StringXxHash3Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { private const ulong Prime64_1 = 0x9E3779B185EBCA87UL; private const ulong Prime64_2 = 0xC2B2AE3D27D4EB4FUL; @@ -124,7 +124,23 @@ public struct StringXxHash3Hasher : IHashProvider, IHashProvider64 + /// Computes the XXH3-64 hash of the specified character span, xor-folded to a + /// signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded XXH3-64 hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -145,7 +161,15 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { // n = count of UTF-16 code units (chars); the conceptual byte length is 2 * n, // which is always even — so every XXH3 read in the 4-to-240-byte and long // paths lands on a char boundary. The single odd-aligned read in XXH3 (the @@ -197,7 +221,7 @@ public ulong Hash64(string key) // ── Short-length code paths (seed 0) ────────────────────────────────────── [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Len4to8(string s, int n) + private static ulong Len4to8(ReadOnlySpan s, int n) { ulong byteLength = (ulong)(2 * n); uint input1 = Key32(s, 0); // bytes [0, 4) @@ -208,7 +232,7 @@ private static ulong Len4to8(string s, int n) } [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Len9to16(string s, int n) + private static ulong Len9to16(ReadOnlySpan s, int n) { ulong byteLength = (ulong)(2 * n); ulong bitflip1 = Sec64(24) ^ Sec64(32); @@ -222,7 +246,7 @@ private static ulong Len9to16(string s, int n) return Xxh3Avalanche(acc); } - private static ulong Len17to128(string s, int n) + private static ulong Len17to128(ReadOnlySpan s, int n) { int len = 2 * n; // byte length, in (16, 128] ulong acc = (ulong)len * Prime64_1; @@ -250,7 +274,7 @@ private static ulong Len17to128(string s, int n) return Xxh3Avalanche(acc); } - private static ulong Len129to240(string s, int n) + private static ulong Len129to240(ReadOnlySpan s, int n) { int len = 2 * n; // byte length, in (128, 240] ulong acc = (ulong)len * Prime64_1; @@ -276,7 +300,7 @@ private static ulong Len129to240(string s, int n) // ── Long code path (> 240 bytes), default secret / seed 0 ───────────────── - private static ulong HashLong(string s, int n) + private static ulong HashLong(ReadOnlySpan s, int n) { long byteLength = 2L * n; @@ -311,7 +335,7 @@ private static ulong HashLong(string s, int n) } [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static void Accumulate(Span acc, string s, int startChar, int nbStripes) + private static void Accumulate(Span acc, ReadOnlySpan s, int startChar, int nbStripes) { for (int i = 0; i < nbStripes; i++) { @@ -321,7 +345,7 @@ private static void Accumulate(Span acc, string s, int startChar, int nbS } [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static void AccumulateStripe(Span acc, string s, int stripeChar, int secretOffset) + private static void AccumulateStripe(Span acc, ReadOnlySpan s, int stripeChar, int secretOffset) { for (int lane = 0; lane < 8; lane++) { @@ -367,7 +391,7 @@ private static ulong MergeAccs(Span acc, int secretOffset, ulong start) /// through a 128-bit multiply. Seed is 0. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Mix16B(string s, int ci, int so) + private static ulong Mix16B(ReadOnlySpan s, int ci, int so) { ulong inputLo = Key64(s, ci); ulong inputHi = Key64(s, ci + 4); @@ -420,7 +444,7 @@ private static ulong Rrmxmx(ulong h, ulong length) /// byte-oriented XXH3 would read over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Key64(string s, int ci) => + private static ulong Key64(ReadOnlySpan s, int ci) => (ulong)s[ci] | ((ulong)s[ci + 1] << 16) | ((ulong)s[ci + 2] << 32) @@ -431,7 +455,7 @@ private static ulong Key64(string s, int ci) => /// starting at char index . /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static uint Key32(string s, int ci) => + private static uint Key32(ReadOnlySpan s, int ci) => (uint)s[ci] | ((uint)s[ci + 1] << 16); [MethodImpl(MethodImplOptions.AggressiveInlining)] diff --git a/src/Celerity.Hashing/StringXxHash64Hasher.cs b/src/Celerity.Hashing/StringXxHash64Hasher.cs index 5dfde73e..999979a9 100644 --- a/src/Celerity.Hashing/StringXxHash64Hasher.cs +++ b/src/Celerity.Hashing/StringXxHash64Hasher.cs @@ -58,11 +58,11 @@ namespace Celerity.Hashing; /// The algorithm carries 64 bits of state internally, so the type also implements /// : returns that state un-folded, which is /// what the probabilistic sketches want (see for why the extra -/// 32 bits matter there and not in a hash table). is unchanged — it is +/// 32 bits matter there and not in a hash table). is unchanged — it is /// exactly h ^ (h >> 32) of the 64-bit result. /// /// -public struct StringXxHash64Hasher : IHashProvider, IHashProvider64 +public struct StringXxHash64Hasher : IHashProvider, IHashProvider64, ISpanHashProvider { private const ulong Prime1 = 11400714785074694791UL; private const ulong Prime2 = 14029467366897019727UL; @@ -86,7 +86,23 @@ public struct StringXxHash64Hasher : IHashProvider, IHashProvider64 + /// Computes the xxHash64 (XXH64, seed 0) hash of the specified character span, + /// xor-folded to a signed 32-bit result. + /// + /// The characters to hash. + /// + /// The signed 32-bit, xor-folded xxHash64 hash of — the same + /// value returns for a string with the same contents. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public int Hash(ReadOnlySpan key) + { + ulong h64 = Hash64Core(key); return unchecked((int)(h64 ^ (h64 >> 32))); } @@ -107,8 +123,16 @@ public int Hash(string key) public ulong Hash64(string key) { ArgumentNullException.ThrowIfNull(key); + return Hash64Core(key.AsSpan()); + } - int length = key.Length; // count of UTF-16 code units (chars) + // The single 64-bit body. Both Hash64(string) and the span-based Hash(ReadOnlySpan) + // route through it, so the string and span overloads cannot drift apart — the parity the + // ISpanHashProvider contract requires. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ulong Hash64Core(ReadOnlySpan key) + { + int length = key.Length; // count of UTF-16 code units (chars) ulong byteLength = (ulong)length * 2UL; // XXH64 mixes in the byte length ulong h64; @@ -200,7 +224,7 @@ public ulong Hash64(string key) /// over the native little-endian UTF-16 stream. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static ulong Lane(string key, int i) => + private static ulong Lane(ReadOnlySpan key, int i) => (ulong)key[i] | ((ulong)key[i + 1] << 16) | ((ulong)key[i + 2] << 32) @@ -212,7 +236,7 @@ private static ulong Lane(string key, int i) => /// 16 bits, the next char the high 16 bits. /// [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static uint Block(string key, int i) => + private static uint Block(ReadOnlySpan key, int i) => (uint)key[i] | ((uint)key[i + 1] << 16); /// The XXH64 accumulator round: rotl(acc + lane * PRIME64_2, 31) * PRIME64_1. diff --git a/src/Celerity.Tests/Collections/SpanLookupTests.cs b/src/Celerity.Tests/Collections/SpanLookupTests.cs new file mode 100644 index 00000000..4ae77abc --- /dev/null +++ b/src/Celerity.Tests/Collections/SpanLookupTests.cs @@ -0,0 +1,311 @@ +using Celerity.Collections; +using Celerity.Hashing; + +namespace Celerity.Tests.Collections; + +/// +/// Cross-collection tests for the span-keyed lookup surface: the same check run across every +/// string-keyed type that gained it — , +/// , , +/// , and . +/// +/// +/// The contract under test is that the span path and the string path are indistinguishable: +/// same hits, same misses, same values. Every row therefore asserts the span result +/// against the type's own string overload rather than against a hard-coded +/// expectation, so a divergence shows up wherever it is introduced. +/// +public class SpanLookupTests +{ + // A key set that exercises the corners: the empty string, keys that are prefixes of one + // another, keys differing only in a high byte (which the default low-byte hasher collides), + // and a surrogate pair. + private static readonly string[] Keys = + [ + "", + "a", + "ab", + "abc", + "alpha", + "alphabet", + "A", + "Ł", // U+0141 — collides with "A" under the low-byte StringFnV1AHasher + "日本語", + "emoji-\U0001F600", // surrogate pair + "x\0y", // embedded NUL + new string('q', 300), + ]; + + private static readonly string[] Misses = + [ + "z", + "alph", + "alphabets", + "Ń", + "emoji-\U0001F601", + new string('q', 299), + ]; + + private static KeyValuePair[] Pairs() => + Keys.Select((k, i) => new KeyValuePair(k, i)).ToArray(); + + // ── FrozenCelerityDictionary ────────────────────────────────────────────── + + [Fact] + public void TryGetValue_ShouldAgreeWithStringOverload_WhenFrozenCelerityDictionary() + { + var dict = new FrozenCelerityDictionary(Pairs()); + + foreach (string key in Keys.Concat(Misses)) + { + bool expected = dict.TryGetValue(key, out int expectedValue); + bool actual = dict.TryGetValue(key.AsSpan(), out int actualValue); + + Assert.Equal(expected, actual); + Assert.Equal(expectedValue, actualValue); + Assert.Equal(dict.ContainsKey(key), dict.ContainsKey(key.AsSpan())); + } + } + + [Fact] + public void TryGetValue_ShouldAgreeWithStringOverload_WhenFrozenCelerityDictionaryFallsBackToProbing() + { + // "A" and "Ł" share a raw code under the low-byte hasher, so no seed can separate them + // and the build takes the linear-probing fallback. The span probe must mirror it. + var dict = new FrozenCelerityDictionary(Pairs()); + Assert.False(dict.IsPerfectlyHashed); + + foreach (string key in Keys.Concat(Misses)) + { + bool expected = dict.TryGetValue(key, out int expectedValue); + bool actual = dict.TryGetValue(key.AsSpan(), out int actualValue); + + Assert.Equal(expected, actual); + Assert.Equal(expectedValue, actualValue); + } + } + + [Fact] + public void TryGetValue_ShouldNotMatchTheNullKey_WhenSpanIsEmpty() + { + // A span has no null state, so an empty span is the key "" and never the out-of-band + // null key. Here "" is absent and null is present: the empty span must miss. + var dict = new FrozenCelerityDictionary( + [new KeyValuePair(null!, 7), new KeyValuePair("a", 1)]); + + Assert.True(dict.TryGetValue((string)null!, out int viaNull)); + Assert.Equal(7, viaNull); + Assert.False(dict.TryGetValue(ReadOnlySpan.Empty, out _)); + Assert.False(dict.ContainsKey(ReadOnlySpan.Empty)); + } + + [Fact] + public void TryGetValue_ShouldThrowArgumentNullException_WhenFrozenCelerityDictionaryIsNull() + { + FrozenCelerityDictionary dict = null!; + Assert.Throws(() => dict.TryGetValue("a".AsSpan(), out int _)); + Assert.Throws(() => dict.ContainsKey("a".AsSpan())); + } + + // ── FrozenCeleritySet ───────────────────────────────────────────────────── + + [Fact] + public void Contains_ShouldAgreeWithStringOverload_WhenFrozenCeleritySet() + { + var set = new FrozenCeleritySet(Keys); + + foreach (string key in Keys.Concat(Misses)) + Assert.Equal(set.Contains(key), set.Contains(key.AsSpan())); + } + + [Fact] + public void Contains_ShouldAgreeWithStringOverload_WhenFrozenCeleritySetFallsBackToProbing() + { + var set = new FrozenCeleritySet(Keys); + Assert.False(set.IsPerfectlyHashed); + + foreach (string key in Keys.Concat(Misses)) + Assert.Equal(set.Contains(key), set.Contains(key.AsSpan())); + } + + [Fact] + public void Contains_ShouldThrowArgumentNullException_WhenFrozenCeleritySetIsNull() + { + FrozenCeleritySet set = null!; + Assert.Throws(() => set.Contains("a".AsSpan())); + } + + // ── CelerityDictionary ──────────────────────────────────────────────────── + + [Fact] + public void TryGetValue_ShouldAgreeWithStringOverload_WhenCelerityDictionary() + { + var dict = new CelerityDictionary(); + foreach (KeyValuePair pair in Pairs()) + dict.Add(pair.Key, pair.Value); + + foreach (string key in Keys.Concat(Misses)) + { + bool expected = dict.TryGetValue(key, out int expectedValue); + bool actual = dict.TryGetValue(key.AsSpan(), out int actualValue); + + Assert.Equal(expected, actual); + Assert.Equal(expectedValue, actualValue); + Assert.Equal(dict.ContainsKey(key), dict.ContainsKey(key.AsSpan())); + } + } + + [Fact] + public void TryGetValue_ShouldSeeSubsequentMutations_WhenCelerityDictionary() + { + var dict = new CelerityDictionary(); + Assert.False(dict.ContainsKey("late".AsSpan())); + + dict.Add("late", 42); + Assert.True(dict.TryGetValue("late".AsSpan(), out int value)); + Assert.Equal(42, value); + + dict.Remove("late"); + Assert.False(dict.TryGetValue("late".AsSpan(), out _)); + } + + [Fact] + public void TryGetValue_ShouldSurviveAResize_WhenCelerityDictionary() + { + // Grow well past the initial threshold so the span probe runs against a rehashed table. + var dict = new CelerityDictionary(capacity: 4); + for (int i = 0; i < 500; i++) + dict.Add($"key-{i}", i); + + for (int i = 0; i < 500; i++) + { + Assert.True(dict.TryGetValue($"key-{i}".AsSpan(), out int value)); + Assert.Equal(i, value); + } + + Assert.False(dict.TryGetValue("key-500".AsSpan(), out _)); + } + + [Fact] + public void TryGetValue_ShouldNotMatchTheNullKey_WhenCelerityDictionarySpanIsEmpty() + { + var dict = new CelerityDictionary(); + dict.Add(null!, 7); + + Assert.True(dict.TryGetValue((string)null!, out int viaNull)); + Assert.Equal(7, viaNull); + Assert.False(dict.TryGetValue(ReadOnlySpan.Empty, out _)); + + dict.Add(string.Empty, 9); + Assert.True(dict.TryGetValue(ReadOnlySpan.Empty, out int viaSpan)); + Assert.Equal(9, viaSpan); + } + + [Fact] + public void TryGetValue_ShouldThrowArgumentNullException_WhenCelerityDictionaryIsNull() + { + CelerityDictionary dict = null!; + Assert.Throws(() => dict.TryGetValue("a".AsSpan(), out int _)); + Assert.Throws(() => dict.ContainsKey("a".AsSpan())); + } + + // ── CeleritySet ─────────────────────────────────────────────────────────── + + [Fact] + public void Contains_ShouldAgreeWithStringOverload_WhenCeleritySet() + { + var set = new CeleritySet(); + foreach (string key in Keys) + set.Add(key); + + foreach (string key in Keys.Concat(Misses)) + Assert.Equal(set.Contains(key), set.Contains(key.AsSpan())); + } + + [Fact] + public void Contains_ShouldSurviveAResize_WhenCeleritySet() + { + var set = new CeleritySet(capacity: 4); + for (int i = 0; i < 500; i++) + set.Add($"item-{i}"); + + for (int i = 0; i < 500; i++) + Assert.True(set.Contains($"item-{i}".AsSpan())); + + Assert.False(set.Contains("item-500".AsSpan())); + } + + [Fact] + public void Contains_ShouldThrowArgumentNullException_WhenCeleritySetIsNull() + { + CeleritySet set = null!; + Assert.Throws(() => set.Contains("a".AsSpan())); + } + + // ── Trie ────────────────────────────────────────────────────────────────── + + [Fact] + public void TryGetValue_ShouldAgreeWithStringOverload_WhenTrie() + { + var trie = new Trie(); + foreach (KeyValuePair pair in Pairs()) + trie.Add(pair.Key, pair.Value); + + foreach (string key in Keys.Concat(Misses)) + { + bool expected = trie.TryGetValue(key, out int expectedValue); + bool actual = trie.TryGetValue(key.AsSpan(), out int actualValue); + + Assert.Equal(expected, actual); + Assert.Equal(expectedValue, actualValue); + Assert.Equal(trie.ContainsKey(key), trie.ContainsKey(key.AsSpan())); + Assert.Equal(trie.ContainsPrefix(key), trie.ContainsPrefix(key.AsSpan())); + } + } + + [Fact] + public void ContainsPrefix_ShouldAgreeWithStringOverload_WhenTriePrefixIsPartial() + { + var trie = new Trie { ["alphabet"] = 1 }; + + Assert.True(trie.ContainsPrefix("alph".AsSpan())); + Assert.False(trie.ContainsKey("alph".AsSpan())); + Assert.True(trie.ContainsPrefix(ReadOnlySpan.Empty)); + Assert.False(trie.ContainsPrefix("beta".AsSpan())); + } + + // ── The span may be a slice of a caller-owned buffer, not a whole string ── + + [Fact] + public void SpanLookups_ShouldMatchOnASliceOfALargerBuffer_AcrossEveryType() + { + // The shape a real parser hands in: the key sits inside a bigger buffer with + // neighbouring characters on both sides. + const string Buffer = ">>>alphabet<<<"; + ReadOnlySpan slice = Buffer.AsSpan(3, "alphabet".Length); + + var frozenDict = new FrozenCelerityDictionary(Pairs()); + var frozenSet = new FrozenCeleritySet(Keys); + + var dict = new CelerityDictionary(); + var set = new CeleritySet(); + var trie = new Trie(); + foreach (KeyValuePair pair in Pairs()) + { + dict.Add(pair.Key, pair.Value); + set.Add(pair.Key); + trie.Add(pair.Key, pair.Value); + } + + int expected = Array.IndexOf(Keys, "alphabet"); + + Assert.True(frozenDict.TryGetValue(slice, out int a)); + Assert.Equal(expected, a); + Assert.True(frozenSet.Contains(slice)); + Assert.True(dict.TryGetValue(slice, out int b)); + Assert.Equal(expected, b); + Assert.True(set.Contains(slice)); + Assert.True(trie.TryGetValue(slice, out int c)); + Assert.Equal(expected, c); + } +} diff --git a/src/Celerity.Tests/Collections/StringInternTableDifferentialTests.cs b/src/Celerity.Tests/Collections/StringInternTableDifferentialTests.cs new file mode 100644 index 00000000..532ea6ed --- /dev/null +++ b/src/Celerity.Tests/Collections/StringInternTableDifferentialTests.cs @@ -0,0 +1,127 @@ +using Celerity.Collections; +using Celerity.Hashing; + +namespace Celerity.Tests.Collections; + +/// +/// Deterministic, seeded differential coverage for . Each seed +/// drives the same random stream of span interns, string interns, lookups, and clears into the table and +/// into an independent oracle keyed by +/// , then asserts after every operation that the two agree on count, +/// per-token membership, and — the property that makes the type worth having — that the table hands back +/// the same reference for every repeat of a token it has already seen. +/// +/// +/// Tokens are drawn from a tiny alphabet at short lengths so hash collisions and probe chains are dense, +/// and the run interleaves the span and string entry points so a divergence between them surfaces. +/// +public class StringInternTableDifferentialTests +{ + [Theory] + [InlineData(1)] + [InlineData(2)] + [InlineData(7)] + [InlineData(42)] + [InlineData(1234)] + public void RandomizedOperations_MatchDictionaryOracle_WithWeakHasher(int seed) => + RunCase(seed); + + [Theory] + [InlineData(1)] + [InlineData(42)] + [InlineData(1234)] + public void RandomizedOperations_MatchDictionaryOracle_WithStrongHasher(int seed) => + RunCase(seed); + + private static void RunCase(int seed) + where THasher : struct, IHashProvider, ISpanHashProvider + { + var rand = new Random(seed); + var table = new StringInternTable(capacity: 2); + + // The oracle maps a token's contents to the canonical instance the table returned first. + var oracle = new Dictionary(StringComparer.Ordinal); + + const int Steps = 4000; + for (int step = 0; step < Steps; step++) + { + string token = RandomToken(rand); + int op = rand.Next(100); + + if (op < 45) + { + // Intern from a span carved out of a larger buffer — the parser shape. + string padded = "<<" + token + ">>"; + string interned = table.GetOrAdd(padded.AsSpan(2, token.Length)); + + Assert.Equal(token, interned); + if (oracle.TryGetValue(token, out string? canonical)) + Assert.Same(canonical, interned); + else + oracle[token] = interned; + } + else if (op < 70) + { + // Intern from a freshly allocated string: on a miss the supplied instance itself + // becomes canonical; on a hit the already-held instance comes back. + string supplied = new string(token.ToCharArray()); + string interned = table.GetOrAdd(supplied); + + if (oracle.TryGetValue(token, out string? canonical)) + { + Assert.Same(canonical, interned); + } + else + { + Assert.Same(supplied, interned); + oracle[token] = interned; + } + } + else if (op < 88) + { + // Pure lookup: never mutates, and agrees with the oracle on both entry points. + bool expected = oracle.TryGetValue(token, out string? canonical); + Assert.Equal(expected, table.TryGet(token.AsSpan(), out string? actual)); + Assert.Equal(expected, table.Contains(token.AsSpan())); + Assert.Equal(expected, table.Contains(token)); + if (expected) + Assert.Same(canonical, actual); + else + Assert.Null(actual); + } + else if (op < 97) + { + // Enumeration yields exactly the canonical instances, once each. + var seen = new List(); + foreach (string s in table) + seen.Add(s); + + Assert.Equal(oracle.Count, seen.Count); + foreach (string s in seen) + { + Assert.True(oracle.TryGetValue(s, out string? canonical)); + Assert.Same(canonical, s); + } + } + else + { + table.Clear(); + oracle.Clear(); + } + + Assert.Equal(oracle.Count, table.Count); + } + } + + // A tiny alphabet at short lengths, so tokens repeat constantly and probe chains stay dense. + private static string RandomToken(Random rand) + { + const string Alphabet = "abcŁ"; + int length = rand.Next(0, 5); + return string.Create(length, rand, static (span, rng) => + { + for (int i = 0; i < span.Length; i++) + span[i] = Alphabet[rng.Next(Alphabet.Length)]; + }); + } +} diff --git a/src/Celerity.Tests/Collections/StringInternTableTests.cs b/src/Celerity.Tests/Collections/StringInternTableTests.cs new file mode 100644 index 00000000..6e983096 --- /dev/null +++ b/src/Celerity.Tests/Collections/StringInternTableTests.cs @@ -0,0 +1,326 @@ +using Celerity.Collections; +using Celerity.Hashing; + +namespace Celerity.Tests.Collections; + +/// +/// Dedicated tests for / . +/// +public class StringInternTableTests +{ + // ── Construction / validation ───────────────────────────────────────────── + + [Fact] + public void Constructor_ShouldStartEmpty_WhenDefault() + { + var table = new StringInternTable(); + Assert.Equal(0, table.Count); + Assert.Empty(table); + } + + [Theory] + [InlineData(-1)] + [InlineData(int.MinValue)] + public void Constructor_ShouldThrowArgumentOutOfRangeException_WhenCapacityIsNegative(int capacity) + { + Assert.Throws(() => new StringInternTable(capacity)); + Assert.Throws(() => new StringInternTable(capacity)); + } + + [Theory] + [InlineData(0f)] + [InlineData(1f)] + [InlineData(-0.5f)] + [InlineData(1.5f)] + public void Constructor_ShouldThrowArgumentOutOfRangeException_WhenLoadFactorIsOutOfRange(float loadFactor) + { + Assert.Throws(() => new StringInternTable(16, loadFactor)); + Assert.Throws(() => new StringInternTable(16, loadFactor)); + } + + [Fact] + public void Constructor_ShouldAcceptZeroCapacity_WhenRoundedUp() + { + var table = new StringInternTable(0); + Assert.Same(table.GetOrAdd("a".AsSpan()), table.GetOrAdd("a".AsSpan())); + } + + // ── The headline behaviour: allocate once, return the same reference ────── + + [Fact] + public void GetOrAdd_ShouldReturnTheSameReference_WhenTheSameCharactersAreSeenAgain() + { + var table = new StringInternTable(); + + string first = table.GetOrAdd("token".AsSpan()); + string second = table.GetOrAdd("token".AsSpan()); + + Assert.Equal("token", first); + Assert.Same(first, second); + Assert.Equal(1, table.Count); + } + + [Fact] + public void GetOrAdd_ShouldReturnTheSameReference_WhenSpansComeFromDifferentBuffers() + { + var table = new StringInternTable(); + + string first = table.GetOrAdd("xx-token-yy".AsSpan(3, 5)); + char[] buffer = "..token..".ToCharArray(); + string second = table.GetOrAdd(buffer.AsSpan(2, 5)); + + Assert.Equal("token", first); + Assert.Same(first, second); + Assert.Equal(1, table.Count); + } + + [Fact] + public void GetOrAdd_ShouldNotAllocateANewString_WhenGivenAStringOnAMiss() + { + var table = new StringInternTable(); + string supplied = new string('k', 5); + + Assert.Same(supplied, table.GetOrAdd(supplied)); + } + + [Fact] + public void GetOrAdd_ShouldReturnTheAlreadyInternedInstance_WhenGivenAnEqualString() + { + var table = new StringInternTable(); + string canonical = table.GetOrAdd("token".AsSpan()); + string duplicate = new string("token".ToCharArray()); + + Assert.NotSame(canonical, duplicate); + Assert.Same(canonical, table.GetOrAdd(duplicate)); + Assert.Equal(1, table.Count); + } + + [Fact] + public void GetOrAdd_ShouldThrowArgumentNullException_WhenStringIsNull() + { + var table = new StringInternTable(); + Assert.Throws(() => table.GetOrAdd((string)null!)); + } + + [Fact] + public void GetOrAdd_ShouldTreatTheEmptySpanAsTheEmptyString() + { + var table = new StringInternTable(); + + string interned = table.GetOrAdd(ReadOnlySpan.Empty); + + Assert.Equal(string.Empty, interned); + Assert.Equal(1, table.Count); + Assert.True(table.Contains(string.Empty)); + Assert.Same(interned, table.GetOrAdd(string.Empty)); + } + + [Fact] + public void GetOrAdd_ShouldKeepEveryDistinctToken_WhenTheTableResizes() + { + var table = new StringInternTable(capacity: 2); + var canonical = new string[400]; + + for (int i = 0; i < canonical.Length; i++) + canonical[i] = table.GetOrAdd($"token-{i}".AsSpan()); + + Assert.Equal(canonical.Length, table.Count); + + // Every token still resolves to the instance handed out before the growth. + for (int i = 0; i < canonical.Length; i++) + Assert.Same(canonical[i], table.GetOrAdd($"token-{i}".AsSpan())); + } + + // ── TryGet / Contains ───────────────────────────────────────────────────── + + [Fact] + public void TryGet_ShouldNotIntern_WhenTheCharactersAreAbsent() + { + var table = new StringInternTable(); + + Assert.False(table.TryGet("absent".AsSpan(), out string? value)); + Assert.Null(value); + Assert.Equal(0, table.Count); + } + + [Fact] + public void TryGet_ShouldReturnTheInternedInstance_WhenPresent() + { + var table = new StringInternTable(); + string canonical = table.GetOrAdd("present".AsSpan()); + + Assert.True(table.TryGet("present".AsSpan(), out string? value)); + Assert.Same(canonical, value); + } + + [Fact] + public void Contains_ShouldAgreeAcrossTheSpanAndStringOverloads() + { + var table = new StringInternTable(); + table.GetOrAdd("present".AsSpan()); + + Assert.True(table.Contains("present")); + Assert.True(table.Contains("present".AsSpan())); + Assert.False(table.Contains("absent")); + Assert.False(table.Contains("absent".AsSpan())); + } + + [Fact] + public void Contains_ShouldThrowArgumentNullException_WhenStringIsNull() + { + var table = new StringInternTable(); + Assert.Throws(() => table.Contains((string)null!)); + } + + // ── Clear ───────────────────────────────────────────────────────────────── + + [Fact] + public void Clear_ShouldDropEveryInternedString() + { + var table = new StringInternTable(); + string before = table.GetOrAdd("token".AsSpan()); + table.GetOrAdd("other".AsSpan()); + + table.Clear(); + + Assert.Equal(0, table.Count); + Assert.False(table.Contains("token".AsSpan())); + + // A fresh instance is minted after a Clear — the old one is no longer canonical. + Assert.NotSame(before, table.GetOrAdd("token".AsSpan())); + } + + [Fact] + public void Clear_ShouldBeANoOp_WhenAlreadyEmpty() + { + var table = new StringInternTable(); + table.Clear(); + Assert.Equal(0, table.Count); + } + + // ── Enumeration ─────────────────────────────────────────────────────────── + + [Fact] + public void GetEnumerator_ShouldYieldEveryInternedString() + { + var table = new StringInternTable(); + string[] tokens = ["a", "bb", "ccc", string.Empty]; + foreach (string token in tokens) + table.GetOrAdd(token.AsSpan()); + + var seen = new List(); + foreach (string s in table) + seen.Add(s); + + Assert.Equal(tokens.OrderBy(s => s, StringComparer.Ordinal), seen.OrderBy(s => s, StringComparer.Ordinal)); + } + + [Fact] + public void GetEnumerator_ShouldYieldTheCanonicalInstances() + { + var table = new StringInternTable(); + string canonical = table.GetOrAdd("token".AsSpan()); + + Assert.Same(canonical, Assert.Single(table)); + } + + [Fact] + public void MoveNext_ShouldThrowInvalidOperationException_WhenTheTableIsModifiedDuringEnumeration() + { + var table = new StringInternTable(); + table.GetOrAdd("a".AsSpan()); + + StringInternTable.Enumerator enumerator = table.GetEnumerator(); + table.GetOrAdd("b".AsSpan()); + + Assert.Throws(() => enumerator.MoveNext()); + } + + [Fact] + public void Reset_ShouldRestartTheEnumeration() + { + var table = new StringInternTable(); + table.GetOrAdd("a".AsSpan()); + + StringInternTable.Enumerator enumerator = table.GetEnumerator(); + Assert.True(enumerator.MoveNext()); + Assert.False(enumerator.MoveNext()); + + enumerator.Reset(); + Assert.True(enumerator.MoveNext()); + enumerator.Dispose(); + } + + [Fact] + public void Reset_ShouldThrowInvalidOperationException_WhenTheTableWasModified() + { + var table = new StringInternTable(); + table.GetOrAdd("a".AsSpan()); + + StringInternTable.Enumerator enumerator = table.GetEnumerator(); + table.GetOrAdd("b".AsSpan()); + + Assert.Throws(enumerator.Reset); + } + + [Fact] + public void GetEnumerator_ShouldWorkThroughTheGenericInterface() + { + var table = new StringInternTable(); + table.GetOrAdd("a".AsSpan()); + + IEnumerable asEnumerable = table; + Assert.Single(asEnumerable); + + System.Collections.IEnumerable nonGeneric = table; + var seen = new List(); + foreach (object? item in nonGeneric) + seen.Add(item); + Assert.Single(seen); + } + + [Fact] + public void Current_ShouldBeNullBeforeTheFirstMoveNext() + { + var table = new StringInternTable(); + table.GetOrAdd("a".AsSpan()); + + System.Collections.IEnumerator enumerator = ((IEnumerable)table).GetEnumerator(); + Assert.Null(enumerator.Current); + } + + // ── Hasher parameterization ─────────────────────────────────────────────── + + [Fact] + public void GetOrAdd_ShouldBehaveIdentically_AcrossHashers() + { + var weak = new StringInternTable(); + var strong = new StringInternTable(); + + // "A" and "Ł" share a raw code under the low-byte hasher; the table must still keep + // them distinct, because the resolution is the ordinal span compare, not the hash. + string[] tokens = ["A", "Ł", "A", "Ł", "日本語"]; + foreach (string token in tokens) + { + weak.GetOrAdd(token.AsSpan()); + strong.GetOrAdd(token.AsSpan()); + } + + Assert.Equal(3, weak.Count); + Assert.Equal(3, strong.Count); + Assert.NotSame(weak.GetOrAdd("A".AsSpan()), weak.GetOrAdd("Ł".AsSpan())); + } + + [Fact] + public void GetOrAdd_ShouldKeepSurrogatePairsDistinct() + { + var table = new StringInternTable(); + + string a = table.GetOrAdd("\U0001F600".AsSpan()); + string b = table.GetOrAdd("\U0001F601".AsSpan()); + + Assert.NotSame(a, b); + Assert.Equal(2, table.Count); + Assert.Equal("\U0001F600", a); + } +} diff --git a/src/Celerity.Tests/Hashing/SpanHashParityTests.cs b/src/Celerity.Tests/Hashing/SpanHashParityTests.cs new file mode 100644 index 00000000..7ad0fff7 --- /dev/null +++ b/src/Celerity.Tests/Hashing/SpanHashParityTests.cs @@ -0,0 +1,209 @@ +using Celerity.Hashing; + +namespace Celerity.Tests.Hashing; + +/// +/// Family-wide contract tests for : which built-in hashers +/// implement it, and the one invariant every implementation must hold — +/// Hash(s) == Hash(s.AsSpan()). +/// +/// +/// +/// This is not a nice-to-have. The span lookups on the string-keyed collections hash a span +/// and then compare the result against keys that were placed using the string +/// overload. If the two overloads ever disagreed for some input, the lookup would not be +/// slow — it would report a stored key as absent. That failure is silent, data-dependent, +/// and would survive every existing test, so it is pinned here directly. +/// +/// +/// The parity assertions run over , which already sweeps every +/// length class the block-oriented hashers branch on (empty, sub-word tails, exact word and +/// stripe boundaries, several bulk-loop iterations) plus non-ASCII characters whose high byte +/// is set. Each string is checked both as a whole-string span and as a slice of a larger +/// buffer — the shape a real parser hands in, and the one that would catch an +/// implementation that read past its span or keyed off the buffer rather than the slice. +/// +/// +public class SpanHashParityTests +{ + /// + /// The built-in hashers that hash a character span. Every String* hasher qualifies: + /// each already walks the characters, so the span overload is the same body. The integer + /// and hashers do not — a character span is not their key shape. + /// + public static readonly string[] ExpectedSpanHashers = + [ + nameof(StringAdler32Hasher), + nameof(StringCityHash64Hasher), + nameof(StringCrc32Hasher), + nameof(StringDjb2AHasher), + nameof(StringDjb2Hasher), + nameof(StringElfHasher), + nameof(StringFnV164Hasher), + nameof(StringFnV1A64Hasher), + nameof(StringFnV1AFullHasher), + nameof(StringFnV1AHasher), + nameof(StringFnV1Hasher), + nameof(StringHalfSipHash24Hasher), + nameof(StringHighwayHash64Hasher), + nameof(StringJenkinsOaatHasher), + nameof(StringMetroHash64Hasher), + nameof(StringMurmur2Hasher), + nameof(StringMurmur3Hasher), + nameof(StringSdbmHasher), + nameof(StringSipHash13Hasher), + nameof(StringSipHash24Hasher), + nameof(StringXxHash32Hasher), + nameof(StringXxHash3Hasher), + nameof(StringXxHash64Hasher), + ]; + + private static IEnumerable AllHasherTypes() => + typeof(IHashProvider<>).Assembly + .GetExportedTypes() + .Where(t => t.IsValueType && Array.Exists(t.GetInterfaces(), IsHashProvider32)); + + private static bool IsHashProvider32(Type i) => + i.IsGenericType && i.GetGenericTypeDefinition() == typeof(IHashProvider<>); + + private static bool IsSpanHashProvider(Type t) => + Array.Exists(t.GetInterfaces(), i => i == typeof(ISpanHashProvider)); + + // ── Roster ──────────────────────────────────────────────────────────────── + + [Fact] + public void ISpanHashProvider_ShouldBeImplementedByExactlyTheExpectedHashers() + { + string[] actual = AllHasherTypes() + .Where(IsSpanHashProvider) + .Select(t => t.Name) + .OrderBy(n => n, StringComparer.Ordinal) + .ToArray(); + + Assert.Equal( + ExpectedSpanHashers.OrderBy(n => n, StringComparer.Ordinal).ToArray(), + actual); + } + + [Fact] + public void ISpanHashProvider_ShouldNotBeImplementedByTheNonStringHashers() + { + string[] nonStringSpanHashers = AllHasherTypes() + .Where(t => IsSpanHashProvider(t) && !t.Name.StartsWith("String", StringComparison.Ordinal)) + .Select(t => t.Name) + .ToArray(); + + Assert.Empty(nonStringSpanHashers); + } + + // ── The Hash(s) == Hash(s.AsSpan()) contract, per hasher ────────────────── + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenAdler32() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenCityHash64() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenCrc32() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenDjb2() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenDjb2A() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenElf() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenFnV1() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenFnV164() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenFnV1A() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenFnV1A64() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenFnV1AFull() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenHalfSipHash24() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenHighwayHash64() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenJenkinsOaat() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenMetroHash64() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenMurmur2() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenMurmur3() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenSdbm() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenSipHash13() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenSipHash24() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenXxHash32() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenXxHash3() => AssertParity(); + + [Fact] + public void Hash_ShouldMatchStringOverload_WhenXxHash64() => AssertParity(); + + // ── Null handling on the string overload is unchanged ───────────────────── + + [Fact] + public void Hash_ShouldThrowArgumentNullException_WhenStringKeyIsNull() + { + var hasher = new StringFnV1AHasher(); + Assert.Throws(() => hasher.Hash((string)null!)); + } + + [Fact] + public void Hash_ShouldReturnTheEmptyStringCode_WhenSpanIsDefault() + { + // A span has no null state: default(ReadOnlySpan) is empty, and empty means "". + var hasher = new StringFnV1AHasher(); + Assert.Equal(hasher.Hash(string.Empty), hasher.Hash(default(ReadOnlySpan))); + } + + private static void AssertParity() + where THasher : struct, IHashProvider, ISpanHashProvider + { + var hasher = default(THasher); + + foreach (string s in HasherStringCorpus.Strings) + { + int fromString = hasher.Hash(s); + + Assert.Equal(fromString, hasher.Hash(s.AsSpan())); + + // The same characters as a slice of a larger buffer, so the hasher cannot be + // reading past the span or keying off anything but the slice's contents. + string padded = "ÿÿ" + s + "ÿÿ"; + Assert.Equal(fromString, hasher.Hash(padded.AsSpan(2, s.Length))); + + // And as a slice of a char[] the caller owns, which is not a string at all. + char[] buffer = new char[s.Length + 3]; + s.AsSpan().CopyTo(buffer.AsSpan(1)); + Assert.Equal(fromString, hasher.Hash(buffer.AsSpan(1, s.Length))); + } + } +} diff --git a/src/Celerity/Collections/CelerityDictionary.cs b/src/Celerity/Collections/CelerityDictionary.cs index 63689dd3..845e7244 100644 --- a/src/Celerity/Collections/CelerityDictionary.cs +++ b/src/Celerity/Collections/CelerityDictionary.cs @@ -803,6 +803,47 @@ private int ProbeForKey(TKey key) } } + // The hasher instance the span-lookup extension methods hand back to the probe below. + // The field is private and the extension methods live outside the type, so they need a + // way to reach it without the class itself carrying an ISpanHashProvider constraint — + // which could not be added to THasher without breaking every existing instantiation. + internal THasher Hasher => _hasher; + + // Reads the value parked alongside a slot the span probe already located. + internal TValue? ValueAt(int index) => _values[index]; + + // The span twin of ProbeForKey. Generic in its own hasher type parameter (rather than + // reusing THasher) so the ISpanHashProvider constraint lives on the method: the class + // constraint stays exactly what it has always been, and the JIT still devirtualizes the + // hash call because TSpanHasher is a struct type parameter. + // + // Only reachable from SpanLookupExtensions, whose signatures pin TKey to string — which + // is what makes the reinterpretation of the slot below sound. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + internal int ProbeForKey(ReadOnlySpan key, TSpanHasher hasher) + where TSpanHasher : struct, ISpanHashProvider + { + TKey?[] keys = _keys; + ref TKey? keysRef = ref MemoryMarshal.GetArrayDataReference(keys); + int mask = keys.Length - 1; + int index = hasher.Hash(key) & mask; + + while (true) + { + ref TKey? slot = ref Unsafe.Add(ref keysRef, (nint)(uint)index); + if (EmptySlot.Is(slot)) + return -1; + + // TKey is string at every call site (see above), so this is a no-op + // reinterpretation rather than a cast. Comparing the spans is ordinal, matching + // the EqualityComparer.Default the string-keyed probe uses. + if (key.SequenceEqual(Unsafe.As(ref slot).AsSpan())) + return index; + + index = (index + 1) & mask; + } + } + private void Resize() => Resize(FastUtils.DoubleCapacity(_keys.Length)); // Rehashes every live entry into a freshly allocated table of the given power-of-two size. diff --git a/src/Celerity/Collections/CeleritySet.cs b/src/Celerity/Collections/CeleritySet.cs index 87453f9f..2b2c79ad 100644 --- a/src/Celerity/Collections/CeleritySet.cs +++ b/src/Celerity/Collections/CeleritySet.cs @@ -569,6 +569,44 @@ public void Dispose() { } private static bool IsDefaultValue(T item) => EmptySlot.Is(item); + // The hasher instance the span-lookup extension methods hand back to the probe below. + // The field is private and the extension methods live outside the type, so they need a + // way to reach it without the class itself carrying an ISpanHashProvider constraint — + // which could not be added to THasher without breaking every existing instantiation. + internal THasher Hasher => _hasher; + + // The span twin of ProbeForItem. Generic in its own hasher type parameter (rather than + // reusing THasher) so the ISpanHashProvider constraint lives on the method: the class + // constraint stays exactly what it has always been, and the JIT still devirtualizes the + // hash call because TSpanHasher is a struct type parameter. + // + // Only reachable from SpanLookupExtensions, whose signatures pin T to string — which is + // what makes the reinterpretation of the slot below sound. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + internal int ProbeForItem(ReadOnlySpan item, TSpanHasher hasher) + where TSpanHasher : struct, ISpanHashProvider + { + T?[] slots = _slots; + ref T? slotsRef = ref MemoryMarshal.GetArrayDataReference(slots); + int mask = slots.Length - 1; + int index = hasher.Hash(item) & mask; + + while (true) + { + ref T? slot = ref Unsafe.Add(ref slotsRef, (nint)(uint)index); + if (EmptySlot.Is(slot)) + return -1; + + // T is string at every call site (see above), so this is a no-op + // reinterpretation rather than a cast. Comparing the spans is ordinal, matching + // the EqualityComparer.Default the string-keyed probe uses. + if (item.SequenceEqual(Unsafe.As(ref slot).AsSpan())) + return index; + + index = (index + 1) & mask; + } + } + [MethodImpl(MethodImplOptions.AggressiveInlining)] private int ProbeForItem(T item) { diff --git a/src/Celerity/Collections/FrozenCelerityDictionary.cs b/src/Celerity/Collections/FrozenCelerityDictionary.cs index ebe23720..3ec1c41a 100644 --- a/src/Celerity/Collections/FrozenCelerityDictionary.cs +++ b/src/Celerity/Collections/FrozenCelerityDictionary.cs @@ -468,6 +468,47 @@ private static void BuildFallback( // Returns the slot holding , or -1 if absent. In perfect // mode this is a single index + equality check; in fallback mode it linear-probes // until it finds the key or hits an empty slot. The caller guarantees key != null. + // The hasher instance the span-lookup extension methods hand back to the probe below. + // The field is private and the extension methods live outside the type, so they need a + // way to reach it without the class itself carrying an ISpanHashProvider constraint — + // which could not be added to THasher without breaking every existing instantiation. + internal THasher Hasher => _hasher; + + // Reads the value parked alongside a slot the span probe already located. + internal TValue? ValueAt(int index) => _values[index]; + + // The span twin of FindSlot. Generic in its own hasher type parameter (rather than + // reusing THasher) so the ISpanHashProvider constraint lives on the method: the class + // constraint stays exactly what it has always been, and the JIT still devirtualizes the + // hash call because TSpanHasher is a struct type parameter. Both the perfect and the + // linear-probing fallback shapes are mirrored exactly, including the same Mix/seed, so + // the span path and the string path resolve to the same slot. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + internal int FindSlot(ReadOnlySpan key, TSpanHasher hasher) + where TSpanHasher : struct, ISpanHashProvider + { + string?[] keys = _keys; + ref string? keysRef = ref MemoryMarshal.GetArrayDataReference(keys); + int mask = _mask; + int slot = (int)(Mix(unchecked((uint)hasher.Hash(key)), _seed) & (uint)mask); + + if (_isPerfect) + { + string? candidate = Unsafe.Add(ref keysRef, (nint)(uint)slot); + return candidate is not null && key.SequenceEqual(candidate.AsSpan()) ? slot : -1; + } + + while (true) + { + string? candidate = Unsafe.Add(ref keysRef, (nint)(uint)slot); + if (candidate is null) + return -1; + if (key.SequenceEqual(candidate.AsSpan())) + return slot; + slot = (slot + 1) & mask; + } + } + [MethodImpl(MethodImplOptions.AggressiveInlining)] private int FindSlot(string key) { diff --git a/src/Celerity/Collections/FrozenCeleritySet.cs b/src/Celerity/Collections/FrozenCeleritySet.cs index f3d24ddb..8d521d7a 100644 --- a/src/Celerity/Collections/FrozenCeleritySet.cs +++ b/src/Celerity/Collections/FrozenCeleritySet.cs @@ -481,6 +481,44 @@ private static void BuildFallback( // mode this is a single index + equality check; in fallback mode it linear-probes // until it finds the element or hits an empty slot. The caller guarantees // item != null. + // The hasher instance the span-lookup extension methods hand back to the probe below. + // The field is private and the extension methods live outside the type, so they need a + // way to reach it without the class itself carrying an ISpanHashProvider constraint — + // which could not be added to THasher without breaking every existing instantiation. + internal THasher Hasher => _hasher; + + // The span twin of FindSlot. Generic in its own hasher type parameter (rather than + // reusing THasher) so the ISpanHashProvider constraint lives on the method: the class + // constraint stays exactly what it has always been, and the JIT still devirtualizes the + // hash call because TSpanHasher is a struct type parameter. Both the perfect and the + // linear-probing fallback shapes are mirrored exactly, including the same Mix/seed, so + // the span path and the string path resolve to the same slot. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + internal int FindSlot(ReadOnlySpan item, TSpanHasher hasher) + where TSpanHasher : struct, ISpanHashProvider + { + string?[] items = _items; + ref string? itemsRef = ref MemoryMarshal.GetArrayDataReference(items); + int mask = _mask; + int slot = (int)(Mix(unchecked((uint)hasher.Hash(item)), _seed) & (uint)mask); + + if (_isPerfect) + { + string? candidate = Unsafe.Add(ref itemsRef, (nint)(uint)slot); + return candidate is not null && item.SequenceEqual(candidate.AsSpan()) ? slot : -1; + } + + while (true) + { + string? candidate = Unsafe.Add(ref itemsRef, (nint)(uint)slot); + if (candidate is null) + return -1; + if (item.SequenceEqual(candidate.AsSpan())) + return slot; + slot = (slot + 1) & mask; + } + } + [MethodImpl(MethodImplOptions.AggressiveInlining)] private int FindSlot(string item) { diff --git a/src/Celerity/Collections/SpanLookupExtensions.cs b/src/Celerity/Collections/SpanLookupExtensions.cs new file mode 100644 index 00000000..f3852380 --- /dev/null +++ b/src/Celerity/Collections/SpanLookupExtensions.cs @@ -0,0 +1,185 @@ +using Celerity.Hashing; + +namespace Celerity.Collections; + +/// +/// Allocation-free lookups on the string-keyed Celerity collections from a +/// of — a slice of a buffer a parser +/// already holds — without first materializing a . +/// +/// +/// +/// A tokenizer, CSV/log reader, or route dispatcher that has a +/// ReadOnlySpan<char> over its input buffer would otherwise have to call +/// new string(span) to probe a string-keyed collection: one allocation plus a copy +/// per lookup, on the hot path of exactly the workloads these types exist for. +/// These overloads delete both. The stored keys are compared ordinally against the span, +/// which is what does for , +/// so a span lookup and the equivalent string lookup always agree. +/// +/// +/// Why extension methods. The collections are generic in their hasher +/// (where THasher : struct, IHashProvider<string>) and the span probe needs +/// as well. Adding that to the class constraint would break +/// every existing instantiation, so the extra constraint lives on these methods instead: +/// they bind only when the hasher supplies both, are resolved statically (no boxing, and the +/// JIT still devirtualizes the hash call through the struct type parameter), and leave the +/// collections' own signatures untouched. +/// +/// +/// The empty span. A span has no null state, so an empty span means +/// the empty string "" — an ordinary key — and never the out-of-band null key. +/// Look the null key up through the string overload. +/// +/// +/// takes no hasher and carries its span overloads as ordinary +/// instance methods; see . +/// To go the other way — turning a span into a that is allocated only +/// the first time it is seen — use . +/// +/// +/// +/// +/// var routes = new FrozenCelerityDictionary<int, StringXxHash3Hasher>(pairs); +/// +/// ReadOnlySpan<char> path = requestLine.AsSpan(4, length); +/// if (routes.TryGetValue(path, out int handler)) +/// Dispatch(handler); // no string was allocated +/// +/// +public static class SpanLookupExtensions +{ + /// + /// Attempts to get the value associated with the characters in . + /// + /// The type of the stored values. + /// The dictionary's hasher; must also hash spans. + /// The dictionary to probe. + /// The characters to look up. An empty span means the key "". + /// + /// When this method returns, contains the value associated with + /// if found; otherwise the default value of . + /// + /// true if the key was found; otherwise, false. + /// is null. + public static bool TryGetValue( + this FrozenCelerityDictionary dictionary, + ReadOnlySpan key, + out TValue? value) + where THasher : struct, IHashProvider, ISpanHashProvider + { + ArgumentNullException.ThrowIfNull(dictionary); + + int index = dictionary.FindSlot(key, dictionary.Hasher); + if (index < 0) + { + value = default; + return false; + } + + value = dictionary.ValueAt(index); + return true; + } + + /// + /// Determines whether the characters in are present as a key. + /// + /// The type of the stored values. + /// The dictionary's hasher; must also hash spans. + /// The dictionary to probe. + /// The characters to look up. An empty span means the key "". + /// true if the key is found; otherwise, false. + /// is null. + public static bool ContainsKey( + this FrozenCelerityDictionary dictionary, + ReadOnlySpan key) + where THasher : struct, IHashProvider, ISpanHashProvider + { + ArgumentNullException.ThrowIfNull(dictionary); + return dictionary.FindSlot(key, dictionary.Hasher) >= 0; + } + + /// + /// Determines whether the characters in are present in the set. + /// + /// The set's hasher; must also hash spans. + /// The set to probe. + /// The characters to look up. An empty span means the element "". + /// true if the element is found; otherwise, false. + /// is null. + public static bool Contains( + this FrozenCeleritySet set, + ReadOnlySpan item) + where THasher : struct, IHashProvider, ISpanHashProvider + { + ArgumentNullException.ThrowIfNull(set); + return set.FindSlot(item, set.Hasher) >= 0; + } + + /// + /// Attempts to get the value associated with the characters in . + /// + /// The type of the stored values. + /// The dictionary's hasher; must also hash spans. + /// The dictionary to probe. + /// The characters to look up. An empty span means the key "". + /// + /// When this method returns, contains the value associated with + /// if found; otherwise the default value of . + /// + /// true if the key was found; otherwise, false. + /// is null. + public static bool TryGetValue( + this CelerityDictionary dictionary, + ReadOnlySpan key, + out TValue? value) + where THasher : struct, IHashProvider, ISpanHashProvider + { + ArgumentNullException.ThrowIfNull(dictionary); + + int index = dictionary.ProbeForKey(key, dictionary.Hasher); + if (index < 0) + { + value = default; + return false; + } + + value = dictionary.ValueAt(index); + return true; + } + + /// + /// Determines whether the characters in are present as a key. + /// + /// The type of the stored values. + /// The dictionary's hasher; must also hash spans. + /// The dictionary to probe. + /// The characters to look up. An empty span means the key "". + /// true if the key is found; otherwise, false. + /// is null. + public static bool ContainsKey( + this CelerityDictionary dictionary, + ReadOnlySpan key) + where THasher : struct, IHashProvider, ISpanHashProvider + { + ArgumentNullException.ThrowIfNull(dictionary); + return dictionary.ProbeForKey(key, dictionary.Hasher) >= 0; + } + + /// + /// Determines whether the characters in are present in the set. + /// + /// The set's hasher; must also hash spans. + /// The set to probe. + /// The characters to look up. An empty span means the element "". + /// true if the element is found; otherwise, false. + /// is null. + public static bool Contains( + this CeleritySet set, + ReadOnlySpan item) + where THasher : struct, IHashProvider, ISpanHashProvider + { + ArgumentNullException.ThrowIfNull(set); + return set.ProbeForItem(item, set.Hasher) >= 0; + } +} diff --git a/src/Celerity/Collections/StringInternTable.cs b/src/Celerity/Collections/StringInternTable.cs new file mode 100644 index 00000000..346e314a --- /dev/null +++ b/src/Celerity/Collections/StringInternTable.cs @@ -0,0 +1,404 @@ +using System.Collections; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using Celerity.Hashing; +using Celerity.Primitives; + +namespace Celerity.Collections; + +/// +/// A canonicalizing table of s that is probed with a +/// of and allocates a +/// only on a miss, using +/// . Supply a different string hasher via the +/// generic overload. +/// +public sealed class StringInternTable : StringInternTable +{ + /// + /// Initializes a new with the specified capacity + /// and load factor. + /// + /// The initial capacity, rounded up to the next power of two. + /// + /// The fraction of the table that can be filled before it grows. + /// + /// + /// is negative, or is not + /// in the open interval (0, 1). + /// + public StringInternTable( + int capacity = DEFAULT_CAPACITY, + float loadFactor = DEFAULT_LOAD_FACTOR) + : base(capacity, loadFactor) + { + } +} + +/// +/// A canonicalizing table of s that is probed with a +/// of and allocates a +/// only on a miss. +/// +/// +/// The hasher used to compute key hashes. Must be a value type implementing both +/// over and +/// , so the JIT can devirtualize and inline it and so the +/// span and string probes agree. +/// +/// +/// +/// The workload. A parser walking a 10M-cell CSV or a log stream holds each +/// token as a slice of its input buffer. If the token set is small — say a hundred distinct +/// column values — it wants a hundred s, not ten million. Feeding each +/// slice to returns the one canonical instance and +/// materializes a only the first time a token is seen. Downstream +/// reference equality then works, and the GC never sees the other 9,999,900 copies. +/// +/// +/// Why the BCL cannot do this before .NET 9. +/// HashSet<string>.TryGetValue takes a , so you must +/// allocate the string before you can discover you already had it — the allocation +/// this type exists to avoid. .NET 9's +/// Dictionary<string,V>.GetAlternateLookup<ReadOnlySpan<char>>() +/// closes that gap on .NET 9+; this type works the same way on net8.0, which is +/// Celerity's floor, and stays available on all three target frameworks. +/// +/// +/// Not . The runtime intern pool is +/// process-wide, never collected for the life of the process, and — decisively — still +/// requires a to hand it. A is +/// an ordinary object: its scope is yours, releases everything it holds, +/// and dropping the table drops the interned strings with it. +/// +/// +/// Keys are compared ordinally, matching for +/// . The empty string is an ordinary entry (an empty span means ""); +/// null is not storable, and the string overloads reject it. The type is +/// single-threaded and does not guarantee enumeration order. +/// +/// +/// +/// +/// var interned = new StringInternTable(); +/// +/// foreach (ReadOnlySpan<char> cell in Cells(line)) +/// { +/// string token = interned.GetOrAdd(cell); // allocates only the first time +/// Consume(token); +/// } +/// +/// +public class StringInternTable : IReadOnlyCollection + where THasher : struct, IHashProvider, ISpanHashProvider +{ + /// + /// The default initial capacity of the table if no capacity is specified. + /// + protected const int DEFAULT_CAPACITY = 16; + + /// + /// The default load factor of the table if no load factor is specified. + /// + protected const float DEFAULT_LOAD_FACTOR = 0.75f; + + private string?[] _slots; + private int _count; + private readonly float _loadFactor; + private int _threshold; + private readonly THasher _hasher; + + // Incremented on every structural mutation so active enumerators can detect + // concurrent modification and throw, matching BCL semantics. + private int _version; + + /// + /// Initializes a new with the specified + /// capacity and load factor. + /// + /// The initial capacity, rounded up to the next power of two. + /// + /// The fraction of the table that can be filled before it grows. + /// + /// + /// is negative, or is not + /// in the open interval (0, 1). + /// + public StringInternTable( + int capacity = DEFAULT_CAPACITY, + float loadFactor = DEFAULT_LOAD_FACTOR) + { + if (capacity < 0) + throw new ArgumentOutOfRangeException(nameof(capacity), capacity, "Capacity must be non-negative."); + if (loadFactor <= 0f || loadFactor >= 1f) + throw new ArgumentOutOfRangeException(nameof(loadFactor), loadFactor, "Load factor must be between 0 (exclusive) and 1 (exclusive)."); + + int size = FastUtils.NextPowerOfTwo(capacity); + + _slots = new string?[size]; + _loadFactor = loadFactor; + _threshold = (int)(size * _loadFactor); + _hasher = default; + } + + /// + /// Gets the number of distinct strings the table has interned. + /// + public int Count => _count; + + /// + /// Returns the canonical for the characters in + /// , allocating one only if those characters are not + /// already interned. + /// + /// The characters to canonicalize. An empty span means "". + /// + /// The interned instance. Two calls with equal contents return the same reference, so + /// callers may compare the results with . + /// + public string GetOrAdd(ReadOnlySpan key) + { + int index = Probe(key, out bool wasEmpty); + if (!wasEmpty) + return _slots[index]!; + + // The miss path is the only one that materializes a string. + string materialized = key.ToString(); + + if (_count >= _threshold) + { + Resize(); + index = Probe(key, out _); + } + + _slots[index] = materialized; + _count++; + _version++; + return materialized; + } + + /// + /// Returns the canonical for , interning + /// itself if its contents are not already present. + /// + /// The string to canonicalize. + /// + /// The interned instance — itself if it was the first with those + /// contents, otherwise the instance already held. + /// + /// is null. + /// + /// This overload never allocates: on a miss the supplied instance becomes the canonical + /// one. It is the shape to use when you already hold a and want to + /// collapse duplicates. + /// + public string GetOrAdd(string key) + { + ArgumentNullException.ThrowIfNull(key); + + int index = Probe(key.AsSpan(), out bool wasEmpty); + if (!wasEmpty) + return _slots[index]!; + + if (_count >= _threshold) + { + Resize(); + index = Probe(key.AsSpan(), out _); + } + + _slots[index] = key; + _count++; + _version++; + return key; + } + + /// + /// Looks up the characters in without interning them. + /// + /// The characters to look up. An empty span means "". + /// + /// When this method returns, the interned instance if the contents were already present; + /// otherwise null. The table is not modified either way. + /// + /// true if the contents were already interned; otherwise false. + public bool TryGet(ReadOnlySpan key, out string? value) + { + int index = Probe(key, out bool wasEmpty); + if (wasEmpty) + { + value = null; + return false; + } + + value = _slots[index]; + return true; + } + + /// + /// Determines whether the characters in are already interned. + /// + /// The characters to look up. An empty span means "". + /// true if the contents are present; otherwise false. + public bool Contains(ReadOnlySpan key) + { + Probe(key, out bool wasEmpty); + return !wasEmpty; + } + + /// + /// Determines whether 's contents are already interned. + /// + /// The string to look up. + /// true if the contents are present; otherwise false. + /// is null. + public bool Contains(string key) + { + ArgumentNullException.ThrowIfNull(key); + return Contains(key.AsSpan()); + } + + /// + /// Drops every interned string. The backing capacity is preserved. + /// + public void Clear() + { + if (_count == 0) + return; + + Array.Clear(_slots, 0, _slots.Length); + _count = 0; + _version++; + } + + /// + /// Returns an allocation-free enumerator over the interned strings. The order is + /// unspecified and may change across versions. If the table is modified during + /// enumeration, throws + /// . + /// + /// A struct enumerator over this table. + public Enumerator GetEnumerator() => new Enumerator(this); + + IEnumerator IEnumerable.GetEnumerator() => new Enumerator(this); + + IEnumerator IEnumerable.GetEnumerator() => new Enumerator(this); + + /// + /// A struct enumerator over a . Because it is a + /// struct, iterating it via foreach avoids the allocation a compiler-generated + /// IEnumerator<T> would incur. + /// + public struct Enumerator : IEnumerator + { + private readonly StringInternTable _table; + private readonly int _version; + private int _index; + private string _current; + + internal Enumerator(StringInternTable table) + { + _table = table; + _version = table._version; + _index = -1; + _current = null!; + } + + /// Gets the string at the current position of the enumerator. + public string Current => _current; + + object IEnumerator.Current => _current; + + /// Advances the enumerator to the next interned string. + /// + /// true if the enumerator advanced to a new entry; false if it has + /// passed the end of the table. + /// + /// + /// The table was modified since the enumerator was created. + /// + public bool MoveNext() + { + if (_version != _table._version) + throw new InvalidOperationException("Collection was modified; enumeration operation may not execute."); + + string?[] slots = _table._slots; + while (++_index < slots.Length) + { + string? slot = slots[_index]; + if (slot is not null) + { + _current = slot; + return true; + } + } + + _current = null!; + return false; + } + + /// Resets the enumerator to its initial position, before the first entry. + /// + /// The table was modified since the enumerator was created. + /// + public void Reset() + { + if (_version != _table._version) + throw new InvalidOperationException("Collection was modified; enumeration operation may not execute."); + + _index = -1; + _current = null!; + } + + /// Releases any resources held by the enumerator. No-op for this type. + public void Dispose() { } + } + + // Returns the slot the characters belong in. wasEmpty is true when the slot is vacant + // (a miss — the caller may write there) and false when it already holds a string with + // these contents. A vacant slot always exists because the load factor is < 1. + // + // The probe walks _slots via Unsafe.Add against a base reference taken at the top, so + // per-iteration bounds checks disappear; the bound is structural (mask = length - 1). + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private int Probe(ReadOnlySpan key, out bool wasEmpty) + { + string?[] slots = _slots; + ref string? slotsRef = ref MemoryMarshal.GetArrayDataReference(slots); + int mask = slots.Length - 1; + int index = _hasher.Hash(key) & mask; + + while (true) + { + string? slot = Unsafe.Add(ref slotsRef, (nint)(uint)index); + if (slot is null) { wasEmpty = true; return index; } + if (key.SequenceEqual(slot.AsSpan())) { wasEmpty = false; return index; } + index = (index + 1) & mask; + } + } + + // Rehashes every interned string into a table of twice the size. Entries are known to be + // distinct, so the reinsert loop only has to find the first vacant slot. + private void Resize() + { + int newSize = FastUtils.DoubleCapacity(_slots.Length); + int mask = newSize - 1; + string?[] oldSlots = _slots; + string?[] newSlots = new string?[newSize]; + + for (int i = 0; i < oldSlots.Length; i++) + { + string? slot = oldSlots[i]; + if (slot is null) + continue; + + int index = _hasher.Hash(slot) & mask; + while (newSlots[index] is not null) + index = (index + 1) & mask; + + newSlots[index] = slot; + } + + _slots = newSlots; + _threshold = (int)(newSize * _loadFactor); + } +} diff --git a/src/Celerity/Collections/Trie.cs b/src/Celerity/Collections/Trie.cs index a63fd547..90efd42a 100644 --- a/src/Celerity/Collections/Trie.cs +++ b/src/Celerity/Collections/Trie.cs @@ -259,6 +259,45 @@ public bool TryGetValue(string key, out TValue? value) return false; } + /// + /// Determines whether the trie contains the characters in , without + /// materializing a from the span. + /// + /// The characters to locate. An empty span means the key "". + /// true if the key is present; otherwise false. + /// + /// A descends its keys character by character, so a span key costs + /// exactly what a key costs — minus the new string(span) a caller + /// holding a slice of a parse buffer would otherwise have to allocate per lookup. + /// + public bool ContainsKey(ReadOnlySpan key) + { + Node? node = FindNode(key); + return node is not null && node.HasValue; + } + + /// + /// Attempts to get the value associated with the characters in , without + /// materializing a from the span. + /// + /// The characters to locate. An empty span means the key "". + /// + /// When this method returns, the associated value if the key was found; otherwise the default + /// value of . + /// + /// true if the key was found; otherwise false. + public bool TryGetValue(ReadOnlySpan key, out TValue? value) + { + Node? node = FindNode(key); + if (node is not null && node.HasValue) + { + value = node.Value; + return true; + } + value = default; + return false; + } + /// Removes from the trie. /// The key to remove. /// true if the key was found and removed; otherwise false. @@ -372,6 +411,19 @@ public bool ContainsPrefix(string prefix) return node is not null && (node.HasValue || node.ChildCount != 0); } + /// + /// Determines whether any stored key starts with the characters in + /// (a key equal to the prefix counts), without materializing a from the + /// span. The empty span matches whenever the trie is non-empty. + /// + /// The characters to test as a prefix. + /// true if at least one key has as a prefix; otherwise false. + public bool ContainsPrefix(ReadOnlySpan prefix) + { + Node? node = FindNode(prefix); + return node is not null && (node.HasValue || node.ChildCount != 0); + } + /// /// Enumerates every entry whose key starts with (an entry whose key equals the /// prefix is included), in ascending ordinal key order. The empty prefix enumerates the whole trie. @@ -490,7 +542,9 @@ public bool TryGetLongestPrefix(string query, out string? key, out TValue? value // ---- internal machinery ---------------------------------------------------------------------- // Walks the key from the root and returns the node it ends on, or null if the path breaks. - private Node? FindNode(string key) + // Takes a span so the string and span lookup surfaces share one descent; a string key is + // handed in as key.AsSpan(), which is free. + private Node? FindNode(ReadOnlySpan key) { Node node = _root; for (int i = 0; i < key.Length; i++) diff --git a/web/dev/bench/detail.html b/web/dev/bench/detail.html index 683182c2..c4485aa0 100644 --- a/web/dev/bench/detail.html +++ b/web/dev/bench/detail.html @@ -384,6 +384,7 @@ { key: 'DisjointSet', title: 'DisjointSet', vs: 'Dictionary> merge' }, { key: 'IndexedPriorityQueue', title: 'IndexedPriorityQueue', vs: 'PriorityQueue' }, { key: 'Trie', title: 'Trie', vs: 'Dictionary' }, + { key: 'StringInternTable', title: 'StringInternTable', vs: 'HashSet / Dictionary' }, { key: 'FenwickTree', title: 'FenwickTree', vs: 'long[] (naive prefix sum)' }, { key: 'BTreeDictionary', title: 'BTreeDictionary', vs: 'SortedDictionary' }, { key: 'BTreeSet', title: 'BTreeSet', vs: 'SortedSet' }, diff --git a/web/dev/bench/index.html b/web/dev/bench/index.html index 9ebf6bd9..054114af 100644 --- a/web/dev/bench/index.html +++ b/web/dev/bench/index.html @@ -419,7 +419,7 @@

Hash function throughput

{ key: 'SwissDictionary', title: 'SwissDictionary', vs: 'Dictionary', ops: ['Insert', 'Lookup', 'Remove'] }, { key: 'HashCachingDictionary', title: 'HashCachingDictionary', vs: 'Dictionary', ops: ['Insert', 'Lookup', 'Remove'] }, { key: 'PooledCelerityDictionary', title: 'PooledCelerityDictionary', vs: 'Dictionary', ops: ['Insert', 'Lookup', 'Remove'] }, - { key: 'FrozenCelerityDictionary', title: 'FrozenCelerityDictionary', vs: 'FrozenDictionary', ops: ['Build', 'Lookup'] }, + { key: 'FrozenCelerityDictionary', title: 'FrozenCelerityDictionary', vs: 'FrozenDictionary', ops: ['Build', 'Lookup', 'SpanLookup'] }, { key: 'CelerityMultiMap', title: 'CelerityMultiMap', vs: 'Dictionary>', ops: ['Insert', 'Lookup', 'Remove'] }, { key: 'CelerityMultiSet', title: 'CelerityMultiSet', vs: 'Dictionary', ops: ['Count', 'Lookup', 'Remove'] }, // SmallDictionary is a small-n collection: it is benchmarked at 8 / 64 items @@ -459,7 +459,11 @@

Hash function throughput

// IndexedPriorityQueue is an addressable heap benchmarked against the BCL PriorityQueue (lazy-deletion decrease-key). { key: 'IndexedPriorityQueue', title: 'IndexedPriorityQueue', vs: 'PriorityQueue', ops: ['Enqueue', 'DecreaseKey'] }, // Trie is an ordered prefix tree benchmarked against Dictionary; PrefixMatch is the win. - { key: 'Trie', title: 'Trie', vs: 'Dictionary', ops: ['Add', 'Lookup', 'PrefixMatch'] }, + // SpanLookup probes from a ReadOnlySpan slice, which the baseline must first turn into a string. + { key: 'Trie', title: 'Trie', vs: 'Dictionary', ops: ['Add', 'Lookup', 'PrefixMatch', 'SpanLookup'] }, + // StringInternTable canonicalizes tokens probed as spans: it allocates one string per distinct token, + // where the BCL arms must allocate one per occurrence before they can dedupe. Allocation is the headline. + { key: 'StringInternTable', title: 'StringInternTable', vs: 'HashSet / Dictionary', ops: ['Dedupe', 'Lookup'] }, // FenwickTree is a Binary Indexed Tree benchmarked against a plain long[] holding the raw values: point // updates are O(1) there, but every prefix / range sum re-adds the slice (O(n)). The tree keeps both O(log n). { key: 'FenwickTree', title: 'FenwickTree', vs: 'long[] (naive prefix sum)', ops: ['Mixed', 'RangeSum'] }, diff --git a/web/index.html b/web/index.html index 2295e39a..db43dbe2 100644 --- a/web/index.html +++ b/web/index.html @@ -305,6 +305,7 @@

What ships in the box

DisjointSet<T>
Union-find over arbitrary elements: partitions them into disjoint sets with near-O(1) amortized Union/Find/Connected via union-by-size and path halving. The union-find the BCL lacks — incremental connectivity, connected components, and Kruskal MST in near-linear time, where a Dictionary+HashSet set-merge is quadratic.
IndexedPriorityQueue<E, P, H>
Addressable binary min-heap: unlike the BCL PriorityQueue it can change a queued element's priority (Update / decrease-key) and remove an arbitrary element in O(log n), and answer Contains/TryGetPriority in O(1). The heap the priority-relaxation loop of Dijkstra / Prim / A* needs — no lazy-deletion heap growth.
Trie<TValue>
Ordered prefix tree mapping string keys to values: GetByPrefix lists every entry under a prefix in O(prefix + matches) and TryGetLongestPrefix finds the longest stored prefix of a query in O(query). The trie the BCL lacks — autocomplete, routing, and ordered iteration, where a Dictionary must scan every key and run StartsWith.
+
StringInternTable
Canonicalizing token table probed with a ReadOnlySpan<char>: GetOrAdd returns the one shared string for those characters and allocates only on a miss, so a 10M-cell parse over 100 distinct tokens creates 100 strings, not 10,000,000. The collection you cannot build on the pre-.NET-9 BCL — HashSet<string> makes you allocate the string before you can discover you already had it. The same span-keyed lookups also ship on FrozenCelerityDictionary, FrozenCeleritySet, CelerityDictionary, CeleritySet, and Trie.
FenwickTree<T>
Binary Indexed Tree over a fixed-length numeric sequence: point update and prefix / range sum both in O(log n), in one array with no per-node overhead. The prefix-sum structure the BCL lacks — running aggregates, rank counters and cumulative-frequency tables, where a plain array is O(n) per query or O(n) per update.
BTreeDictionary<K, V, C>
Sorted map backed by a B-tree with up to 31 keys per node in flat arrays, so a lookup visits ~log32(n) nodes instead of chasing ~log2(n) pointers. The B-tree the BCL lacks — Min/Max, lower/upper bound and O(log n + k) range scans, where SortedDictionary is a red-black tree with an object per entry and SortedList memmoves on every insert.
BTreeSet<T, C>
The set counterpart: ordered elements packed 31 to a node, with the same ordered surface and an in-order walk over contiguous arrays instead of successor pointers. Beats SortedSet on the interleaved insert + membership + range-scan workload, and stores no values, so the memory saving is larger still.
From 40d49e1091e3122eee1978ae5357d7b2a00919fc Mon Sep 17 00:00:00 2001 From: Marius Bughiu Date: Sun, 26 Jul 2026 16:37:36 +0300 Subject: [PATCH 2/3] fix(collections): assert TKey/T is string in the span probes; single BCL baseline per benchmark category MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the Copilot review on #323. The span probes reinterpret each slot as a string via Unsafe.As, which is a no-op only while the collection is closed over string. The extension methods' signatures are what guarantee that, and nothing in the type system restates it inside the probe — so a future in-assembly caller could silently get undefined behaviour. Debug.Assert is [Conditional("DEBUG")], so the guardrail costs nothing in a release build and the hot path is unchanged. Separately, self-review of the dashboard wiring: StringInternTableBenchmark had two BCL arms in its Dedupe category (HashSet and Dictionary). The dashboard indexes one bcl value per collection|op|itemCount, so the two baselines would have overwritten each other and the card would have compared against whichever landed last. Dropped the HashSet arm — Dictionary is also the truer analogue, since it hands back the canonical instance exactly as GetOrAdd does. The Lookup category keeps HashSet as its only BCL arm. Co-Authored-By: Claude Opus 5 (1M context) --- .../StringInternTableBenchmark.cs | 22 +++++-------------- .../Collections/CelerityDictionary.cs | 8 +++++++ src/Celerity/Collections/CeleritySet.cs | 8 +++++++ 3 files changed, 21 insertions(+), 17 deletions(-) diff --git a/src/Celerity.Benchmarks/StringInternTableBenchmark.cs b/src/Celerity.Benchmarks/StringInternTableBenchmark.cs index 239992c4..f280a5f1 100644 --- a/src/Celerity.Benchmarks/StringInternTableBenchmark.cs +++ b/src/Celerity.Benchmarks/StringInternTableBenchmark.cs @@ -81,25 +81,13 @@ public void Setup() // ---- Dedupe: walk the stream and end up holding one instance per distinct token ------------------- - // The pre-.NET-9 BCL answer: materialize every occurrence, then let the set throw the duplicates away. - // Correct, and it allocates one string per occurrence. + // The pre-.NET-9 BCL answer, and the true functional analogue: a dictionary keyed by the token + // hands the canonical instance back, exactly as GetOrAdd does. It still allocates one string per + // *occurrence*, because the key has to exist before it can be looked up — which is the whole gap. + // (A HashSet fill is the same shape minus the canonical instance, so it is not benchmarked + // separately here; the Lookup category below uses HashSet as its baseline.) [Benchmark(Baseline = true)] [BenchmarkCategory("Dedupe")] - public int HashSet_Dedupe() - { - var set = new HashSet(StringComparer.Ordinal); - for (int i = 0; i < tokens.Length; i++) - { - (int offset, int length) = tokens[i]; - set.Add(new string(buffer, offset, length)); - } - return set.Count; - } - - // The same shape with a dictionary, which at least hands the canonical instance back — still one - // allocation per occurrence, because the key must exist before it can be looked up. - [Benchmark] - [BenchmarkCategory("Dedupe")] public int Dictionary_Dedupe() { var map = new Dictionary(StringComparer.Ordinal); diff --git a/src/Celerity/Collections/CelerityDictionary.cs b/src/Celerity/Collections/CelerityDictionary.cs index 845e7244..86cd896b 100644 --- a/src/Celerity/Collections/CelerityDictionary.cs +++ b/src/Celerity/Collections/CelerityDictionary.cs @@ -1,4 +1,5 @@ using System.Collections; +using System.Diagnostics; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using Celerity.Hashing; @@ -823,6 +824,13 @@ private int ProbeForKey(TKey key) internal int ProbeForKey(ReadOnlySpan key, TSpanHasher hasher) where TSpanHasher : struct, ISpanHashProvider { + // The slot reinterpretation below is a no-op only while TKey really is string. Nothing in + // the type system says so — the extension methods' signatures do — so assert it. Debug.Assert + // is [Conditional("DEBUG")], so this costs nothing in a release build and the hot path is + // unchanged; it exists to catch a future in-assembly caller that forgets the precondition. + Debug.Assert(typeof(TKey) == typeof(string), + "The span probe reinterprets each slot as a string; it is only valid for TKey == string."); + TKey?[] keys = _keys; ref TKey? keysRef = ref MemoryMarshal.GetArrayDataReference(keys); int mask = keys.Length - 1; diff --git a/src/Celerity/Collections/CeleritySet.cs b/src/Celerity/Collections/CeleritySet.cs index 2b2c79ad..eba8a189 100644 --- a/src/Celerity/Collections/CeleritySet.cs +++ b/src/Celerity/Collections/CeleritySet.cs @@ -1,4 +1,5 @@ using System.Collections; +using System.Diagnostics; using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using Celerity.Hashing; @@ -586,6 +587,13 @@ private static bool IsDefaultValue(T item) => internal int ProbeForItem(ReadOnlySpan item, TSpanHasher hasher) where TSpanHasher : struct, ISpanHashProvider { + // The slot reinterpretation below is a no-op only while T really is string. Nothing in the + // type system says so — the extension methods' signatures do — so assert it. Debug.Assert is + // [Conditional("DEBUG")], so this costs nothing in a release build and the hot path is + // unchanged; it exists to catch a future in-assembly caller that forgets the precondition. + Debug.Assert(typeof(T) == typeof(string), + "The span probe reinterprets each slot as a string; it is only valid for T == string."); + T?[] slots = _slots; ref T? slotsRef = ref MemoryMarshal.GetArrayDataReference(slots); int mask = slots.Length - 1; From 15c5753dd11719080009202b702d039431480a60 Mon Sep 17 00:00:00 2001 From: Marius Bughiu Date: Sun, 26 Jul 2026 16:44:26 +0300 Subject: [PATCH 3/3] docs(changelog): condense the #311 entries to what a caller observes Addresses the Copilot review on #323. The bullets were PR-body prose in a changelog: CONTRIBUTING.md asks for a few sentences at most per entry, and the release workflow lifts the whole version section verbatim into the GitHub Release body, where an over-long section has actually broken a release here before. Kept one bullet per rollout facet so the change is still reconstructable from the changelog alone, but cut each to the observable change. The [Unreleased] section drops from 6009 to 4439 characters. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1c1df54a..8884356e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,13 +6,12 @@ All notable changes to Celerity are documented here. This project follows [Keep ### Added -- **`ISpanHashProvider`** in `Celerity.Hashing` — a `Hash(ReadOnlySpan)` contract, implemented by all 23 built-in `String*Hasher` types, that returns exactly what `Hash(string)` returns for the same characters. It is a sibling interface rather than an overload on `IHashProvider` because a `ref struct` cannot be a generic type argument on `net8.0`, this library's floor. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). -- **Span-keyed lookups on the string-keyed collections** — `TryGetValue(ReadOnlySpan, out TValue?)` / `ContainsKey(ReadOnlySpan)` / `Contains(ReadOnlySpan)` on `FrozenCelerityDictionary`, `FrozenCeleritySet`, `CelerityDictionary`, `CeleritySet`, and `Trie`. A tokenizer, CSV/log reader, or route dispatcher holding a slice of its input buffer no longer pays a `new string(span)` allocation and copy per probe. Results match the `string` overloads exactly (ordinal comparison); an empty span means `""`, never the `null` key. Available on all three target frameworks, including the `net8.0` floor where the BCL has no equivalent. On the four hashed collections these are extension methods, so no existing type's constraints change. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). -- **`StringInternTable` / `StringInternTable`** in `Celerity.Collections` — a canonicalizing token table probed with a `ReadOnlySpan`: `GetOrAdd` returns the one shared `string` for those characters and allocates **only on a miss**, so a 10M-cell parse over 100 distinct tokens creates 100 strings instead of 10,000,000. The pre-.NET-9 BCL cannot express this — `HashSet.TryGetValue` takes a `string`, so you must allocate before you can discover you already had it — and `string.Intern` is process-wide, never collected, and still needs a `string`. Implements `IReadOnlyCollection`; not thread-safe. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). -- **`SpanHashParityTests`** — a family-wide roster test plus a per-hasher `Hash(s) == Hash(s.AsSpan())` assertion over every length class, checked both as a whole string and as a slice of a larger buffer, since a divergence there would silently report a stored key as absent. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). -- **`SpanLookupTests`, `StringInternTableTests`, `StringInternTableDifferentialTests`**, and a `StringInternTable` target in the `Celerity.Fuzz` differential harness — cross-collection coverage asserting each span path against its own type's string overload, plus seeded differential runs against a `Dictionary` oracle. `Celerity.AotSmokeTest` exercises every new span instantiation and the intern table under a Native AOT publish. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). -- **`StringInternTableBenchmark`** (registered in the CI-tracked suite) plus `SpanLookup` rows on `FrozenCelerityDictionaryBenchmark` and `TrieBenchmark`, with the matching dashboard cards. Allocation is the headline: the BCL arms allocate one string per occurrence, the Celerity arms one per distinct token. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). -- API reference sections for `ISpanHashProvider`, span-keyed lookups, and `StringInternTable`, plus README entries in the collections list, the "Choosing a collection" decision table, and the hasher-picking notes. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- **`ISpanHashProvider`** in `Celerity.Hashing` — a `Hash(ReadOnlySpan)` contract returning exactly what `Hash(string)` returns for the same characters. All 23 built-in `String*Hasher` types implement it. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- **Span-keyed lookups** — `TryGetValue` / `ContainsKey` / `Contains` now take a `ReadOnlySpan` on `FrozenCelerityDictionary`, `FrozenCeleritySet`, `CelerityDictionary`, `CeleritySet` and `Trie`, so a caller holding a slice of a buffer no longer allocates a `string` per probe. Results match the `string` overloads; an empty span means `""`, never the `null` key. Additive on all three target frameworks. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- **`StringInternTable` / `StringInternTable`** in `Celerity.Collections` — canonicalizes tokens probed as spans: `GetOrAdd` returns the shared `string` and allocates only on a miss, so a parse over few distinct tokens allocates per token rather than per occurrence. Implements `IReadOnlyCollection`; not thread-safe. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- Test coverage for the above: `SpanHashParityTests` (the `Hash(s) == Hash(s.AsSpan())` contract, per hasher), `SpanLookupTests`, `StringInternTableTests`, `StringInternTableDifferentialTests`, a `Celerity.Fuzz` target, and Native AOT smoke-test coverage. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- `StringInternTableBenchmark` in the CI-tracked suite, plus `SpanLookup` rows on the frozen-dictionary and trie benchmarks, with the matching dashboard cards. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). +- Docs for the new surface: `ISpanHashProvider`, span-keyed lookups and `StringInternTable` in the API reference, and matching README entries. Closes [#311](https://github.com/marius-bughiu/Celerity/issues/311). - **`StringKeyProbeBenchmark`** — the tracked suite's first `string`-keyed dictionary and set rows (lookup hit, lookup miss, set `Contains`), shown as the **String-keyed probe** card on the benchmark dashboard. Closes [#308](https://github.com/marius-bughiu/Celerity/issues/308). - **`ReferenceKeyProbeTests`** — cross-collection coverage for vacant-slot detection on the open-addressed collections with reference-type keys. Closes [#308](https://github.com/marius-bughiu/Celerity/issues/308). - **`BTreeDictionary` and `BTreeSet`** (with `BTreeDictionary` / `BTreeSet` aliases and the `DefaultComparer` struct comparer) in `Celerity.Collections` — the library's first sorted map and set, and the B-tree the BCL lacks. Up to 31 keys per node keep a lookup `log₃₂(n)` node visits deep instead of chasing the `log₂(n)` pointers a red-black tree costs, and both add the ordered surface a hash table cannot answer: `Min` / `Max`, lower / upper bound, `EnumerateRange` in `O(log n + k)`, and in-order enumeration. They win on the interleaved insert + lookup + range-scan workload and on memory, and lose slightly on a delete-dominated one. Not thread-safe. Closes [#305](https://github.com/marius-bughiu/Celerity/issues/305).