Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ All notable changes to Celerity are documented here. This project follows [Keep

### Added

- **`IDictionary<TKey, TValue?>` on the mutable dictionary family** — `CelerityDictionary`, `SwissDictionary`, `RobinHoodDictionary`, `HashCachingDictionary`, `PooledCelerityDictionary`, `SmallDictionary`, `IntDictionary`, `LongDictionary` and `EnumMap` now implement the mutable BCL interface alongside `IReadOnlyDictionary<,>`, so passing one to an existing API taking `IDictionary<,>` compiles. `Keys` / `Values` widen to read-only `ICollection<T>` views whose mutators throw `NotSupportedException`, and `Contains` / `Remove` over a `KeyValuePair<,>` match on the pair rather than the key alone — both matching `Dictionary<,>`. Additive: no existing public signature changed, and `EnumMap`'s bounded key universe still rejects an out-of-range cast on the write surface. Closes [#307](https://github.com/marius-bughiu/Celerity/issues/307).
- A public `CopyTo(KeyValuePair<TKey, TValue?>[], int)` on each of those nine dictionaries, and `Contains` / `CopyTo` / `IsReadOnly` on their `KeyCollection` / `ValueCollection` views. Closes [#307](https://github.com/marius-bughiu/Celerity/issues/307).
- `DictionaryInterfaceTests` — a cross-collection suite driving every `IDictionary<,>` member through the interface against a `Dictionary<,>` oracle, one row per dictionary (`BTreeDictionary` included, so the family contract lives in one place), plus the bind-to-an-`IDictionary`-parameter case, `EnumMap`'s out-of-range key, and the disposed-`PooledCelerityDictionary` corner. Closes [#307](https://github.com/marius-bughiu/Celerity/issues/307).
- Docs for the new interface: an `IDictionary<TKey, TValue?>` section in the API reference covering the semantics table and the read-only views, `EnumMap`'s bounded-universe caveat, and the updated README interface notes. Closes [#307](https://github.com/marius-bughiu/Celerity/issues/307).
- **`CompressedIntSet`** in `Celerity.Collections` — an exact, compressed set of 32-bit integers for the huge-and-sparse shape `BitSet`, `SparseSet` and `IntSet` do not serve. Each 65,536-value chunk is stored as a sorted array or a bitmap by density (plus an opt-in run-length form for clustered data that `Optimize()` and `AddRange` produce), so set algebra works chunk-at-a-time instead of one hash probe per element: at 1M values over a 100M universe it intersects ~9x faster and unions ~11x faster than `HashSet<int>`, in ~9x less memory. Implements `ISet<int>` and `IReadOnlySet<int>`, plus `AddRange`, `Optimize`, `IntersectCount`, `Cardinality` and `MemoryUsageInBytes`; enumeration is in ascending order. There is **no portable Roaring format** — Celerity ships no serializers — so this is an in-process structure, not Lucene / Druid / Spark interop. Closes [#310](https://github.com/marius-bughiu/Celerity/issues/310).
- `CompressedIntSetBenchmark` in the CI-tracked suite and the matching **CompressedIntSet** dashboard card, plus API-reference and README docs, dedicated and cross-collection tests, a `Celerity.Fuzz` target, and Native AOT smoke coverage. Closes [#310](https://github.com/marius-bughiu/Celerity/issues/310).
- **`RankSelectBitVector`** in `Celerity.Collections` — an immutable succinct index over a dense bit vector that answers `Rank(i)` (set bits below a position) in `O(1)` and `Select(k)` (position of the `k`-th set bit) in `O(log n)`, filling a BCL gap: .NET ships no rank or select anywhere, so the alternative is a hand-rolled `O(i/64)` popcount loop. Builds from a `BitSet`, packed `ulong[]`, or a list of set positions; `Rank0`, `TrySelect`, `IndexSizeInBytes`, and `ToBitSet` round out the surface. The index costs 25% over the bits and is **build-once** — any mutation requires an `O(n/64)` rebuild, so a vector that keeps changing should stay a `BitSet`. Closes [#312](https://github.com/marius-bughiu/Celerity/issues/312).
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ Both take their ordering as a **struct** `IComparer<T>` type parameter (`Default
- `CountMinSketch<T, THasher>` — **probabilistic** frequency estimator: estimates per-element counts from a fixed grid, **never underestimating** (overestimate bounded by `epsilon · TotalCount`). Mergeable.
- `TopKSketch<T, THasher>` — **probabilistic** top-k / heavy-hitters sketch (Space-Saving): reports a stream's most frequent elements from a fixed `k` monitors in `O(k)` memory, **never underestimating** and never missing a hitter above `TotalCount / k`.

All dictionaries implement `IReadOnlyDictionary<TKey, TValue?>` and ship allocation-free struct enumerators, `Keys` / `Values` views, and an `IEnumerable<KeyValuePair<TKey, TValue>>` constructor. The hash-table collections store `default(TKey)` (zero / `null`) out-of-band so it never collides with the empty-slot sentinel; `SmallDictionary` stores it inline.
The mutable dictionaries implement **both** `IDictionary<TKey, TValue?>` and `IReadOnlyDictionary<TKey, TValue?>`, so they drop into an existing API taking either BCL interface; the immutable `FrozenCelerityDictionary` and the prefix-tree `Trie<TValue>` implement the read-only one only. All of them ship allocation-free struct enumerators, `Keys` / `Values` views, and an `IEnumerable<KeyValuePair<TKey, TValue>>` constructor. The hash-table collections store `default(TKey)` (zero / `null`) out-of-band so it never collides with the empty-slot sentinel; `SmallDictionary` stores it inline.

## Quick start

Expand Down Expand Up @@ -621,7 +621,7 @@ Each type buys a different tradeoff. Find your workload below; if it isn't here,
| **Prefix / range sums over a sequence you keep mutating** — running aggregates, rank / order-statistics counters (inversions, "how many ≤ x seen"), cumulative-frequency tables | `FenwickTree<T>` | Binary Indexed Tree (`T : INumber<T>`): **point update** and **prefix / range sum** both `O(log n)`, in one array with no per-node overhead. The BCL has no prefix-sum structure; a plain array forces `O(n)` per query (recompute the slice) *or* `O(n)` per update (fix the suffix). Wins precisely when updates and partial-sum queries interleave. If the data is immutable after build, a one-shot precomputed prefix-sum array answers in `O(1)` with less code; if you only update and never query a partial sum, a raw array is simpler. |
| Need a stable iteration order or multi-threaded access | `BTreeDictionary<,>` / `BTreeSet<>` for sorted order, `Trie<TValue>` for ordered string keys; BCL `ConcurrentDictionary<,>` for concurrency | Celerity is single-threaded, and the **hash-based** collections leave iteration order unspecified. The ordered collections do promise order by contract: the B-trees iterate in comparer order, `Trie<TValue>` in ascending ordinal key order. |

**Celerity is not the right answer when** you need concurrent access (use `ConcurrentDictionary<,>` or your own lock — Celerity is single-threaded), or a guaranteed iteration order from the **hash-based** collections (those dictionaries expose `IReadOnlyDictionary<,>` and those sets `ISet<>`, and neither promises an order across versions). When you do need ordered iteration, reach for the ordered collections instead: `BTreeDictionary<,>` / `BTreeSet<>` iterate in comparer order and support bounds and range scans, and `Trie<TValue>` gives ascending ordinal order over string keys. `BTreeDictionary<,>` also implements the **mutable** `IDictionary<,>` interface, and `BTreeSet<>` implements `ISet<>`.
**Celerity is not the right answer when** you need concurrent access (use `ConcurrentDictionary<,>` or your own lock — Celerity is single-threaded), or a guaranteed iteration order from the **hash-based** collections (they implement `IDictionary<,>` / `IReadOnlyDictionary<,>` and `ISet<>`, but none promises an order across versions). When you do need ordered iteration, reach for the ordered collections instead: `BTreeDictionary<,>` / `BTreeSet<>` iterate in comparer order and support bounds and range scans, and `Trie<TValue>` gives ascending ordinal order over string keys. Interface support is no longer a reason to choose one over another — every mutable dictionary implements `IDictionary<,>` and every mutable set implements `ISet<>`.

## Choosing a hasher

Expand Down Expand Up @@ -770,7 +770,7 @@ Celerity is **Native AOT and trimming compatible** — no reflection, runtime co

## API at a glance

The dictionaries mirror the parts of `Dictionary<TKey, TValue>` most callers reach for: indexer get/set, `ContainsKey`, `TryGetValue`, `Add`, `TryAdd`, `Remove` (both overloads), `Clear`, `EnsureCapacity` / `TrimExcess`, `Count`, `Keys`, `Values`, `GetEnumerator()`. The string-keyed types additionally take a `ReadOnlySpan<char>` on `TryGetValue` / `ContainsKey` / `Contains`, so a caller holding a slice of a buffer never allocates a `string` to probe. They implement `IReadOnlyDictionary<TKey, TValue?>` and accept an `IEnumerable<KeyValuePair<TKey, TValue>>` at construction. The sets expose `Add`, `TryAdd`, `Contains`, `Remove`, `Clear`, `EnsureCapacity` / `TrimExcess`, `Count`, and a struct enumerator. `EnsureCapacity(n)` pre-grows the table once for a known-size bulk insert (no incremental rehashes); `TrimExcess()` rehashes back down to fit `Count`. The zero / `default(TKey)` key (or element) is stored out-of-band so it never collides with the empty-slot sentinel.
The dictionaries mirror the parts of `Dictionary<TKey, TValue>` most callers reach for: indexer get/set, `ContainsKey`, `TryGetValue`, `Add`, `TryAdd`, `Remove` (both overloads), `Clear`, `EnsureCapacity` / `TrimExcess`, `Count`, `Keys`, `Values`, `GetEnumerator()`. The string-keyed types additionally take a `ReadOnlySpan<char>` on `TryGetValue` / `ContainsKey` / `Contains`, so a caller holding a slice of a buffer never allocates a `string` to probe. They implement `IDictionary<TKey, TValue?>` and `IReadOnlyDictionary<TKey, TValue?>` — the `Keys` / `Values` views widen to read-only `ICollection<T>`s through the mutable interface, whose mutators throw `NotSupportedException` exactly as `Dictionary<,>.KeyCollection` does — and accept an `IEnumerable<KeyValuePair<TKey, TValue>>` at construction. The sets expose `Add`, `TryAdd`, `Contains`, `Remove`, `Clear`, `EnsureCapacity` / `TrimExcess`, `Count`, and a struct enumerator. `EnsureCapacity(n)` pre-grows the table once for a known-size bulk insert (no incremental rehashes); `TrimExcess()` rehashes back down to fit `Count`. The zero / `default(TKey)` key (or element) is stored out-of-band so it never collides with the empty-slot sentinel.

Full constructors, signatures, exceptions, and per-type examples: **[API reference](docs/README.md)**.

Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ The 2026-Q3 review surveyed the shipped surface against the current .NET 8/9/10
**Drop-in parity and correctness in the shipped surface.** Work on code already on NuGet, and the highest-confidence group.

- Fix `HyperLogLog`'s hash-entropy floor. `Hash64` widened a 32-bit `IHashProvider<T>` result, so the reachable hash space was 2^32 — while the type's own docs asserted a 64-bit space and skipped the classical large-range correction on that basis. The bias exceeded the advertised 0.81% standard error from ~1e8 distinct elements, in exactly the regime the type is sold for. Status: `done` — `IHashProvider64<T>` (`ulong Hash64(T key)`) ships as a standalone sibling interface in `Celerity.Hashing`, deliberately *not* deriving from `IHashProvider<T>` so the two contracts stay independent and a 64-bit hasher is never forced to publish a lossy 32-bit fold. Fourteen built-in hashers implement it — `Int64WangHasher`, `Int64Murmur3Hasher`, `UInt64WangHasher`, `UInt64Hasher`, `GuidHasher`, and the nine 64-bit `string` hashers — each of which already computed 64 bits internally and folded them away, so `Hash64` is the same mixer minus the narrowing. The 32-bit-only hashers (`Int32*` / `UInt32*`, the naive folds, `DefaultHasher<T>`) deliberately do not, since a key type narrower than 64 bits has no entropy to publish; a roster test pins that judgement. All five sketches route through it when the hasher provides it, via a compile-time type test the JIT folds away (so neither path allocates or branches) and with existing constructors and type parameters unchanged; on a 32-bit hasher `HyperLogLog` now applies the classical Flajolet large-range correction it previously skipped. `HashQualityEvaluator.Evaluate64` reports distribution over the 64-bit surface. Tracked in [#304](https://github.com/marius-bughiu/Celerity/issues/304).
- Implement `IReadOnlySet<T>` on the mutable sets and `IDictionary<TKey, TValue>` on the dictionaries. The sets implement `ISet<T>` and the dictionaries `IReadOnlyDictionary<,>`, but `ISet<T>` does not derive from `IReadOnlySet<T>` — so an ordinary BCL-shaped API taking either interface is a compile error against a Celerity type today. This is the same Guiding Principle #3 gap the 2.2.0 set-algebra work closed, one level up. Status: `planned`.
- Implement `IReadOnlySet<T>` on the mutable sets and `IDictionary<TKey, TValue>` on the dictionaries. The sets implement `ISet<T>` and the dictionaries `IReadOnlyDictionary<,>`, but `ISet<T>` does not derive from `IReadOnlySet<T>` — so an ordinary BCL-shaped API taking either interface is a compile error against a Celerity type today. This is the same Guiding Principle #3 gap the 2.2.0 set-algebra work closed, one level up. Status: the dictionary half is `done`; the set half is `in-progress` in a community PR ([#306](https://github.com/marius-bughiu/Celerity/issues/306)). Nine dictionaries now declare `IDictionary<TKey, TValue?>` alongside the read-only interface — explicit-interface forwarders only, so no existing public signature moved and the concrete indexer still returns the non-nullable `TValue`. Two calls were worth recording. First, the `KeyCollection` / `ValueCollection` struct views were widened from `IEnumerable<T>` to `ICollection<T>` rather than boxing into a fresh adapter type, which is what keeps `dict.Keys` allocation-free on the direct path while `IDictionary<,>.Keys` still hands back a read-only `ICollection<TKey>` whose mutators throw, exactly as `Dictionary<,>.KeyCollection` does. Second, `EnumMap` was kept in rather than left out for its bounded key universe: an out-of-range enum cast is rejected with `ArgumentOutOfRangeException`, which *is* an `ArgumentException` — the failure `IDictionary<,>.Add` already documents for a key it cannot accept — so the implementation is honest rather than a member that throws where the contract says it should not; it is documented on both surfaces and pinned by a test. `Trie<TValue>` is the one mutable one-value-per-key dictionary deliberately left out: its `Keys` / `Values` are lazy `IEnumerable<T>` traversals, not counted views, so widening them is a design change rather than a forwarder. Tracked in [#307](https://github.com/marius-bughiu/Celerity/issues/307).
- Delete the per-probe virtual call. The probe loops test for an empty slot with `EqualityComparer<TKey>.Default.Equals(slot, default(TKey))`, which the JIT devirtualizes for value-type keys but not under `__Canon`-shared reference-type instantiations — one `callvirt` per probe iteration to perform what is a null check. Guiding Principle #2 exists to remove exactly this. Status: `done` — the twelve open-addressed collections now route every vacant-slot test through an internal `EmptySlot.Is<T>` helper whose `typeof(T).IsValueType` guard the JIT folds, so a reference-type instantiation compiles to a plain null test and a value-type one keeps the existing intrinsic comparison unchanged. Behaviour is identical by construction and the whole existing suite passes untouched; `ReferenceKeyProbeTests` pins the substitution against a key type whose `Equals` claims equality with `null`, and the new `StringKeyProbeBenchmark` gives the dashboard its first reference-type-key rows. The follow-up `IEqualityProvider<T>` idea was **not** opened: a `HashCachingDictionary` control arm showed the residual reference-type-key deficit is dominated by re-hashing the key on every probe, not by the remaining equality dispatch — the actionable guidance is to use the hash-caching variants, now documented in [`docs/performance.md`](docs/performance.md#reference-type-keys-cache-the-hash). Tracked in [#308](https://github.com/marius-bughiu/Celerity/issues/308).
- Restore the family-wide no-op-`Clear()` contract. The library is otherwise strict that an operation which changes nothing observable does not invalidate enumerators — `FenwickTree` documents it for a zero delta, `BTreeDictionary` for a rejected duplicate `TryAdd`, `LruCache` for a hit on the already-MRU entry — but `Deque<T>` bumped its version outside the guard that skips the array clearing, so clearing an already-empty deque tore down every live enumerator, contradicting `Deque`'s own documented contract. Status: `done` — the bump moved inside the guard (Option A of the issue: match Celerity's own family, since the BCL points both ways — `Dictionary<K,V>.Clear()` bumps only when non-empty while `Queue<T>` / `Stack<T>` bump unconditionally). The rule is now pinned once per collection by the new family-wide `ClearNoOpVersionTests`, which also pins the two deliberate exceptions: `BitSet` and `FenwickTree` are fixed-length, so establishing "already empty" costs the same scan as the unconditional clear it would skip, and they agree with each other. Tracked in [#333](https://github.com/marius-bughiu/Celerity/issues/333).
- Span-keyed lookups on the string-keyed collections. .NET 9's `GetAlternateLookup<ReadOnlySpan<char>>` lets the BCL `Dictionary` probe with a span key and no allocation; Celerity's string-keyed types require a materialized `string`, so the BCL is now *ahead* on the axis this library has invested most in. Status: `done` — `ISpanHashProvider` (`int Hash(ReadOnlySpan<char> key)`) ships as a standalone sibling interface in `Celerity.Hashing`, deliberately *not* deriving from `IHashProvider<T>`: that interface is generic in its key type, and a `ref struct` could not be a generic type argument before `allows ref struct` (C# 13 / .NET 9) while `net8.0` remains the floor — expressing the span overload as a non-generic sibling sidesteps that, because the span is a method parameter rather than a type argument. All 23 built-in `String*Hasher` types implement it, each sharing one body between the two overloads so they cannot drift; `SpanHashParityTests` pins `Hash(s) == Hash(s.AsSpan())` per hasher across every length class and as a slice of a larger buffer, since a divergence would silently report a stored key as absent rather than merely being slow. `FrozenCelerityDictionary`, `FrozenCeleritySet`, `CelerityDictionary<string, …>`, `CeleritySet<string, …>` and `Trie<TValue>` gained span `TryGetValue` / `ContainsKey` / `Contains`; on the four hashed types they are extension methods carrying the extra `ISpanHashProvider` constraint on the *method*, so no shipped type's own constraints changed (which would have broken every existing instantiation) and the JIT still devirtualizes through the struct type parameter. `StringInternTable` ships alongside them as the type the pattern makes possible: `GetOrAdd(ReadOnlySpan<char>)` allocates only on a miss, so a 10M-cell parse over 100 distinct tokens creates 100 strings — the one collection the pre-.NET-9 BCL cannot express, since `HashSet<string>.TryGetValue` makes you allocate the string before you can discover you already had it. The optional `ReadOnlySpan<byte>` UTF-8 axis and the `#if NET9_0_OR_GREATER` `IAlternateEqualityComparer` plumbing were both left out as the issue's own scoping allowed — neither is needed for the workload win, and each would widen a brand-new public abstraction before it is load-bearing. Tracked in [#311](https://github.com/marius-bughiu/Celerity/issues/311).
Expand Down
Loading
Loading