feat(collections): add SparseSet — bounded-universe sparse integer set with O(1) clear & dense iteration - #288
Merged
Merged
Conversation
Coverage
Files below 100% line coverage
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds SparseSet to Celerity’s specialized collections: a bounded-universe [0, Universe) integer set using the classic dense+sparse representation to deliver O(1) Clear() (no memory touched) and dense iteration, plus full parity wiring (tests, benchmarks, docs, dashboard, AOT smoke test, changelog).
Changes:
- Introduces
SparseSet : ISet<int>with allocation-free struct enumerator and bounded-universe semantics. - Adds dedicated unit + enumeration + randomized differential tests, and a benchmark (incl.
ClearRefill) registered in core benchmarks. - Updates docs/README + web benchmark dashboard wiring + AOT smoke test + changelog entry.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| web/index.html | Adds SparseSet to “What ships in the box” list. |
| web/dev/bench/index.html | Registers SparseSet in benchmark dashboard collection list (incl. ClearRefill). |
| web/dev/bench/detail.html | Adds SparseSet to per-collection benchmark detail list. |
| src/Celerity/Collections/SparseSet.cs | New bounded-universe sparse set implementation + struct enumerator. |
| src/Celerity.Tests/Collections/SparseSetTests.cs | Core behavior tests (range rules, add/remove/clear, capacity helpers, conformance). |
| src/Celerity.Tests/Collections/SparseSetEnumerationTests.cs | Enumerator behavior + mutation detection coverage. |
| src/Celerity.Tests/Collections/SparseSetDifferentialTests.cs | Randomized differential testing vs HashSet<int> over non-negative bounded universe. |
| src/Celerity.Tests/Collections/SetAlgebraTests.cs | Adds SparseSet conformance + bounded-universe set-algebra caveat test. |
| src/Celerity.Benchmarks/SparseSetBenchmark.cs | New benchmark comparing SparseSet vs HashSet<int> (Add/Contains/ClearRefill/Remove). |
| src/Celerity.Benchmarks/Program.cs | Registers SparseSetBenchmark in core benchmark suite. |
| src/Celerity.AotSmokeTest/Program.cs | Adds runtime smoke coverage for SparseSet (incl. clear-then-reuse). |
| README.md | Documents SparseSet in sets list, narrative, example, and decision table. |
| docs/api/collections.md | Adds full API documentation section for SparseSet. |
| CHANGELOG.md | Adds [Unreleased] entries for SparseSet and parity facets. |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
CHANGELOG.md:15
- The new
[Unreleased]entries are far more detailed than the repo's changelog guidance: CONTRIBUTING.md asks for entries to be "a few sentences at most" and user-facing, and to avoid implementation walkthroughs / internal wiring. Consider collapsing these into a single concise, caller-focused bullet for the new public API (and leaving test/benchmark/dashboard wiring details to the PR description).
- **`SparseSet`** in `Celerity.Collections` — a set of non-negative integers over a bounded universe `[0, Universe)`, backed by the Briggs–Torczon sparse-set representation (a dense value array + a sparse index array), filling a BCL gap (.NET ships no sparse set). Its two wins over `HashSet<int>`: `Clear()` is `O(1)` and touches no memory (`HashSet` zeroes its whole table), and iteration is a dense, cache-friendly scan over exactly the present elements — the winning shape for clear-and-rebuild "visited" sets in graph traversal, ECS, and sweep-line code. `Add` / `Contains` / `Remove` are `O(1)` with no hashing. Costs `O(Universe)` memory and stores only values in `[0, Universe)` (out-of-range throws on `Add`, reads as absent on `Contains` / `Remove`). Implements `ISet<int>`. An opt-in specialized type, not a `HashSet<int>` replacement. Closes [#287](https://github.com/marius-bughiu/Celerity/issues/287).
- `SparseSetTests`, `SparseSetEnumerationTests`, and `SparseSetDifferentialTests` (`Celerity.Tests/Collections`) — dedicated coverage mirroring the `IntSet*` / `SmallSet*` files: the core surface (add / try-add / duplicate / out-of-range / contains / remove-swap / dense-array growth / the O(1) `Clear`-then-reuse path that must reject stale sparse entries), the enumeration surface (every-element-once, growth, post-remove/-clear, mutation detection, `Reset`, non-generic path), and a 6,000-step randomized differential reconciling every mutating / query / clear operation against a `HashSet<int>` oracle over a bounded non-negative universe.
- Cross-collection set-algebra rows for `SparseSet` in `SetAlgebraTests` — `ISet<int>` conformance, the basic algebra within the universe, and the bounded-universe caveat that a mutating op which must add an out-of-universe value throws `ArgumentOutOfRangeException`. (`SparseSet` carries its own `SparseSetDifferentialTests` rather than joining the shared `SetAlgebraDifferentialTests`, whose universe spans negatives it cannot store — exactly as `EnumSet` does.)
- `SparseSetBenchmark` in `Celerity.Benchmarks`, registered in `Program.cs`'s `CoreBenchmarks` array (so it joins the per-PR core run and the gh-pages dashboard, mirroring `IntSetBenchmark`) — `SparseSet` vs `HashSet<int>` across `Add` / `Contains` / **`ClearRefill`** (the O(1)-clear headline win) / `Remove` at `[Params(1000, 100_000)]` over a universe 4× the item count.
- Dashboard wiring for `SparseSet`: the "What ships in the box" ship card in [`web/index.html`](web/index.html) and the `COLLECTIONS` arrays in [`web/dev/bench/index.html`](web/dev/bench/index.html) (key / title / vs / ops incl. `ClearRefill`) and [`web/dev/bench/detail.html`](web/dev/bench/detail.html) (key / title / vs).
- A `SparseSet` differential target in the `Celerity.Fuzz` harness (`SparseSetCase`) — reconciling `TryAdd` / `Remove` / `Clear` churn and enumeration against a `HashSet<int>` oracle over the type's bounded non-negative universe (it draws from `[0, 32)` rather than the shared `[-8, 24]` key domain it cannot store).
- Documentation for `SparseSet`: a full API section in [`docs/api/collections.md`](docs/api/collections.md#sparseset) (the Briggs–Torczon mechanism, the O(1)-clear and dense-iteration wins, the `O(Universe)`-memory / non-negative-only tradeoffs, constructors, the method table, the set-algebra caveat, and a runnable BFS-visited-set example), plus README entries — the Sets list, the `ISet<T>` note, a `SparseSet` paragraph with a runnable example, and a new "cleared-and-rebuilt bounded-int set" row in the "Choosing a collection" decision table.
Benchmarks26 regressions Highlights
Collections (412)
Hashers (100)
Same-runner A/B (sharded 6-way): main ( |
marius-bughiu
added a commit
that referenced
this pull request
Jul 23, 2026
…for a fair comparison SparseSet's universe is mandatory, so `new SparseSet(universe)` inherently pre-allocates its sparse array and pays no sparse resize on Add, whereas the `new HashSet<int>()` baseline paid full incremental-rehash growth — an unfair per-add comparison. Give the baseline the matching capacity hint (`new HashSet<int>(ItemCount)`, as the Setup / Remove paths already do) so the Add category measures per-add cost rather than the baseline's rehash overhead. Addresses a Copilot review comment on PR #288. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
CHANGELOG.md:10
- The new
SparseSetchangelog entries are much longer than the repo’s documented guidance (“a few sentences at most”) and include implementation-level detail. This is a release-safety rule because the release workflow uses the full section as the GitHub Release body, so please tighten these to short, user-facing bullets (and consider folding the rollout bullet into a brief parity note).
- **`SparseSet`** in `Celerity.Collections` — a bounded-universe `[0, Universe)` integer set (the Briggs–Torczon sparse set), filling a BCL gap. It wins over `HashSet<int>` where the set is cleared and rebuilt often: `Clear()` is `O(1)` (it leaves the backing arrays untouched, versus `HashSet` zeroing its table) and iteration is a dense scan over just the present elements — the "visited"-set shape in graph traversal, ECS, and sweep-line code. Costs `O(Universe)` memory and stores only values in `[0, Universe)`. Implements `ISet<int>`; an opt-in specialized type, not a `HashSet<int>` replacement. Closes [#287](https://github.com/marius-bughiu/Celerity/issues/287).
- Full parity rollout for `SparseSet`: dedicated tests (`SparseSetTests` / `SparseSetEnumerationTests` / `SparseSetDifferentialTests`), a `SetAlgebraTests` conformance row, a `Celerity.Fuzz` `SparseSetCase`, `SparseSetBenchmark` (registered in `Program.cs`, with a `ClearRefill` category), dashboard wiring (`web/index.html`, `web/dev/bench/{index,detail}.html`), and docs (`docs/api/collections.md` + README).
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
CHANGELOG.md:9
- This changelog entry is much longer than the contributor guidance (“a few sentences at most”). Consider tightening it to focus on the observable user-facing wins and constraints, and keep implementation-level detail minimal so release notes stay compact.
- **`SparseSet`** in `Celerity.Collections` — a bounded-universe `[0, Universe)` integer set (the Briggs–Torczon sparse set), filling a BCL gap. It wins over `HashSet<int>` where the set is cleared and rebuilt often: `Clear()` is `O(1)` (it leaves the backing arrays untouched, versus `HashSet` zeroing its table) and iteration is a dense scan over just the present elements — the "visited"-set shape in graph traversal, ECS, and sweep-line code. Costs `O(Universe)` memory and stores only values in `[0, Universe)`. Implements `ISet<int>`; an opt-in specialized type, not a `HashSet<int>` replacement. Closes [#287](https://github.com/marius-bughiu/Celerity/issues/287).
marius-bughiu
added a commit
that referenced
this pull request
Jul 23, 2026
…eSet Round-3 tightened the Clear() wording (it resets the count; the version bump only happens on the non-empty path) but missed two spots: the README SparseSet paragraph and the SparseSetBenchmark header comment. Both now say "resets the count" to match the observable behavior and the wording used elsewhere. Addresses Copilot review comments on PR #288. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
CHANGELOG.md:9
- The new
[Unreleased]changelog bullet is much longer than the repo’s documented guidance (“a few sentences at most”). Consider tightening it to a short, user-facing summary so the release body stays within GitHub’s size limit.
- **`SparseSet`** in `Celerity.Collections` — a bounded-universe `[0, Universe)` integer set (the Briggs–Torczon sparse set), filling a BCL gap. It wins over `HashSet<int>` where the set is cleared and rebuilt often: `Clear()` is `O(1)` (it leaves the backing arrays untouched, versus `HashSet` zeroing its table) and iteration is a dense scan over just the present elements — the "visited"-set shape in graph traversal, ECS, and sweep-line code. Costs `O(Universe)` memory and stores only values in `[0, Universe)`. Implements `ISet<int>`; an opt-in specialized type, not a `HashSet<int>` replacement. Closes [#287](https://github.com/marius-bughiu/Celerity/issues/287).
marius-bughiu
added a commit
that referenced
this pull request
Jul 23, 2026
Fold the membership round-trip into Remove so _sparse[item] is read a single time (previously once inside ContainsUnchecked and again to compute the swap index) on this hot path. Same semantics — validated by the 69 dedicated tests and a 30,000-case fuzz run against the HashSet<int> oracle. Addresses a Copilot review comment on PR #288. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t with O(1) clear & dense iteration (#287) SparseSet is a set of non-negative integers over a bounded universe [0, Universe), backed by the Briggs-Torczon sparse-set representation (a dense value array + a sparse index array). It fills a BCL gap (.NET ships no sparse set) and beats HashSet<int> on the two things that representation is built for: an O(1) Clear that touches no memory (HashSet zeroes its whole table) and dense, cache-friendly iteration over exactly the present elements — the winning shape for clear-and-rebuild "visited" sets in graph traversal, ECS, and sweep-line code. Add/Contains/Remove are O(1) with no hashing. Tradeoffs: O(Universe) memory and non-negative-values-only (out of range throws on Add, reads as absent on Contains/Remove). Implements ISet<int>. Full parity rollout in one PR: - Collection: src/Celerity/Collections/SparseSet.cs (ISet<int>, struct enumerator). - Dedicated tests: SparseSetTests, SparseSetEnumerationTests, and a 6,000-step SparseSetDifferentialTests vs a HashSet<int> oracle over a bounded non-negative universe (it cannot join the shared SetAlgebraDifferentialTests, whose universe spans negatives it cannot store — exactly like EnumSet). - Cross-collection tests: SparseSet rows in SetAlgebraTests (conformance + the bounded-universe out-of-range-add caveat). - Benchmark: SparseSetBenchmark (Add/Contains/ClearRefill/Remove vs HashSet<int>), registered in Program.cs CoreBenchmarks. - Dashboard: ship card (web/index.html) + COLLECTIONS arrays (web/dev/bench/index.html, detail.html). - Docs: full API section in docs/api/collections.md + README (Sets list, ISet note, paragraph + example, decision-table row). - AOT smoke test: SparseSet coverage block. ROADMAP: no status to flip — the specialized-collection work is done; this is a post-roadmap tier-(c) BCL-gap enhancement (like EnumSet/SwissSet/TopKSketch). Closes #287. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add SparseSetCase to the Celerity.Fuzz harness, reconciling TryAdd / Remove / Clear churn and enumeration against a HashSet<int> oracle over SparseSet's bounded non-negative universe [0, 32) (it cannot draw from the shared [-8, 24] key domain, which spans negatives it rejects by design). 20,000 cases pass. Also adds the matching CHANGELOG bullet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r wording + terser changelog - Reword the "uninitialized garbage" / "touches no memory" claims in SparseSet's XML docs, docs/api/collections.md, README, ship card, and benchmark comment: .NET arrays are zero-initialized, so the real property is tolerating *stale* sparse entries across Clear/mutations, and Clear() resets count/version rather than touching no memory (it just leaves the backing arrays untouched). - Condense the CHANGELOG [Unreleased] SparseSet entries to the repo's brevity standard (a user-facing lead bullet + one parity-rollout bullet), so the section stays within the release-body size the release workflow extracts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…for a fair comparison SparseSet's universe is mandatory, so `new SparseSet(universe)` inherently pre-allocates its sparse array and pays no sparse resize on Add, whereas the `new HashSet<int>()` baseline paid full incremental-rehash growth — an unfair per-add comparison. Give the baseline the matching capacity hint (`new HashSet<int>(ItemCount)`, as the Setup / Remove paths already do) so the Add category measures per-add cost rather than the baseline's rehash overhead. Addresses a Copilot review comment on PR #288. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- README: qualify the "mutable sets drop in wherever a HashSet<T> is used" claim — the bounded-domain sets (EnumSet, SparseSet) are the exception: a mutating op that must add an out-of-domain value throws. Adding SparseSet to that list had made the blanket claim inaccurate. - SparseSet XML docs / docs/api/collections.md: tighten the Clear() wording — it resets the count (the version bump only happens on the non-empty path, which the method early-returns past when already empty), so drop the imprecise "and version" from the summary claims. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ullet Fold the separate "full parity rollout" bullet (tests / fuzz / benchmark / dashboard / docs list) away, leaving a single user-facing entry that matches the shipped 2.3.0 convention (one bullet per collection, ending with Closes #NNN). Per CONTRIBUTING.md / CLAUDE.md, implementation-level detail belongs in the PR body, not the changelog (a release-safety rule — the release workflow extracts the whole version section as the GitHub Release body). The full rollout remains documented in the PR description table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold the membership round-trip into Remove so _sparse[item] is read a single time (previously once inside ContainsUnchecked and again to compute the swap index) on this hot path. Same semantics — validated by the 69 dedicated tests and a 30,000-case fuzz run against the HashSet<int> oracle. Addresses a Copilot review comment on PR #288. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eSet Round-3 tightened the Clear() wording (it resets the count; the version bump only happens on the non-empty path) but missed two spots: the README SparseSet paragraph and the SparseSetBenchmark header comment. Both now say "resets the count" to match the observable behavior and the wording used elsewhere. Addresses Copilot review comments on PR #288. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Trim the single [Unreleased] bullet to the leaner end of the shipped entry style (cf. Deque / LruCache), keeping the what / why-it-wins / cost / positioning while matching CONTRIBUTING.md's "a few sentences at most" release-safety rule. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
marius-bughiu
force-pushed
the
feat/sparse-set
branch
from
July 24, 2026 20:07
450329f to
856ae17
Compare
marius-bughiu
added a commit
that referenced
this pull request
Jul 24, 2026
Resolves conflicts with the Trie (#286) and SparseSet (#288) merges, which touched the same shared parity files. Every conflict was a "keep both, in order" resolution — no content from either side was dropped: - Program.cs — CoreBenchmarks keeps Trie + Fenwick (SparseSet auto-merged) - web/index.html — ship cards for both Trie and FenwickTree - web/dev/bench/*.html — COLLECTIONS entries for both - README.md — both the "Prefix trees" and "Prefix sums" groups, both details blocks, and both decision-table rows (keeping main's updated iteration-order row that mentions Trie) - docs/api/collections.md — main's Trie section plus the FenwickTree section - CHANGELOG.md — all three Added entries plus main's new Fixed section Also condenses the FenwickTree changelog entry to a single user-facing bullet, matching the convention main just applied to the Trie and SparseSet entries. Full suite green after the merge: 4413 passed, 0 failed.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
SparseSet— a set of non-negative integers over a bounded universe[0, Universe), backed by the classic Briggs–Torczon sparse-set representation (a dense value array + a sparse index array). Closes #287..NET ships no sparse set, and
HashSet<int>cannot match its two defining wins:Clear()isO(1)— it resets the count and touches no memory (HashSet<int>.Clear()isO(capacity), zeroing its entire entry table). This is the headline for clear-and-rebuild "visited" sets: graph BFS/DFS, ECS entity membership, sweep-line, register-allocation liveness.[0, Count)of the dense array).Add/Contains/RemoveareO(1)with no hashing, no probe chain, no allocation — a direct array index plus the sparse↔dense round-trip check (sparse[v] < Count && dense[sparse[v]] == v), which is correct even against the stale garbage a Clear leaves behind. ImplementsISet<int>.Tradeoffs (documented honestly):
O(Universe)memory for the sparse index array; stores only non-negative values belowUniverse(out of range throws onAdd, reads as absent onContains/Remove). It is an opt-in specialized type, not aHashSet<int>replacement — for an unbounded or huge-and-sparse key space,IntSet/HashSet<int>remain right.This is the established post-roadmap tier-(c) pattern (
EnumSet/SwissSet/TopKSketch): the roadmap's specialized-collection work isdone; this fills a BCL gap found while reading the source.Parity rollout (all in this PR)
src/Celerity/Collections/SparseSet.cs—ISet<int>, allocation-free struct enumeratorSparseSetTests,SparseSetEnumerationTests,SparseSetDifferentialTests(6,000-step randomized differential vsHashSet<int>over a bounded non-negative universe, with clear-then-reuse stress)SetAlgebraTestsrows —ISet<int>conformance + the bounded-universe out-of-range-add caveat. It carries its own differential rather than joining the sharedSetAlgebraDifferentialTests, whose universe spans negatives it cannot store — exactly likeEnumSet.SparseSetBenchmark(Add/Contains/ClearRefill/RemovevsHashSet<int>), registered inProgram.csCoreBenchmarksweb/index.html) +COLLECTIONSarrays (web/dev/bench/index.html,web/dev/bench/detail.html)docs/api/collections.md+ README (Sets list,ISet<T>note, paragraph + example, decision-table row)[Unreleased]bullet per facetSparseSetcoverage block inCelerity.AotSmokeTestROADMAP: no status to flip —
SparseSetwas not a roadmap line item; the specialized-collection milestone is alreadydone. Noted, per the parity checklist's "items that genuinely don't apply".Parity facets that genuinely don't apply: the hash-oriented shared suites (
SetConstructorValidationTests/TryAddProbeCountTests/EnsureCapacityAndTrimExcessTests/LoadFactorBoundaryTests, etc.) assume a hash table with aloadFactor, a probe count, and power-of-two sizing —SparseSethas auniverse, no hasher, and no probing, so it does not join them (same rationale asEnumSet).Test plan
dotnet build(library + tests + benchmarks + AOT smoke test) — clean, 0 warnings on the library (XML docs complete).main, the benchmark workflow publishesSparseSetBenchmark'sdata.jsto gh-pages; the dashboard wiring here surfaces it (Add / Contains / ClearRefill / Remove).🤖 Generated with Claude Code