You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(dashboard): render the EnumMap and EnumSet cards, and gate blank cards in CI
The dashboard required an `(ItemCount: N)` suffix on every BenchmarkDotNet result
name. `EnumMapBenchmark` and `EnumSetBenchmark` deliberately declare no `[Params]`
sweep — their workload is the bounded enum universe — so their measurements were
published to gh-pages and then dropped by `parseName`, and both cards had rendered
empty since they shipped.
The `ItemCount` group is now optional in both `index.html` and `detail.html`. An
unparameterized class parses with `itemCount === NO_SWEEP` and charts as a single
bucket: the grid card omits the "@ N items" detail, the detail page drops its
item-count toggle, and such benchmarks are excluded from the headline speedup
stats rather than bucketed at zero items. The suffix is still matched strictly as
`ItemCount`, so a params property named anything else is rejected rather than
charted under an "items" label it does not mean.
That failure mode was silent, and it had bitten twice: `DisjointSet` also blanked
for five runs when its params property was briefly named `ElementCount`. So
`scripts/check_dashboard_coverage.js` now fails CI on it. It lifts the
`COLLECTIONS` tables and both name parsers straight out of the dashboard HTML
rather than reimplementing them, so it validates the parser that actually ships.
Structural checks (the two `COLLECTIONS` arrays agree; every charted collection
has a `{Key}Benchmark` in `CoreBenchmarks`) run on every PR in `ci.yml`, since
`benchmarks.yml` only fires on `src/**` and would never see a dashboard-only
change. The full check runs against the merged report in the aggregate job, last,
so a wiring mistake reddens the job without costing us the measurement.
Closes#301
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: CHANGELOG.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,6 +17,8 @@ All notable changes to Celerity are documented here. This project follows [Keep
17
17
18
18
### Fixed
19
19
20
+
-**The `EnumMap` and `EnumSet` cards on the benchmark dashboard rendered empty.** The page required an `(ItemCount: N)` suffix on every result name, and those two benchmarks deliberately declare no item-count sweep — the enum universe is bounded — so their measurements were published to `gh-pages` and then discarded at render time. Both cards now chart their real numbers, on the grid and on their detail pages, and unparameterized benchmarks are excluded from the headline speedup stats rather than bucketed at zero items. Closes [#301](https://github.com/marius-bughiu/Celerity/issues/301).
21
+
- A blank dashboard card is now a red CI check instead of a silent gap. `scripts/check_dashboard_coverage.js` asserts that the two `COLLECTIONS` arrays agree, that every charted collection is registered in the CI benchmark suite, and — against the merged benchmark report — that every published result name parses and every card resolves to both a BCL and a Celerity measurement. It runs on every PR in `ci.yml` and, in full, in the aggregate job of `benchmarks.yml`. Closes [#301](https://github.com/marius-bughiu/Celerity/issues/301).
20
22
-**The coverage gate measured only one of the six shipped packages.** Coverlet's assembly filter is exact-match, so `Celerity.Hashing`, `Celerity.Primitives`, and the three showcase packages had been outside the gate since the 2.0.0 package split — any of them could have dropped to 0% with CI green. All six are now measured, the gaps that exposed are backfilled to **100% line and branch** coverage, and the floor is raised from 95%/90% to match. Closes [#314](https://github.com/marius-bughiu/Celerity/issues/314).
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+23-3Lines changed: 23 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -76,15 +76,35 @@ dotnet run -c Release -- --filter '*' # run everything with the default (slow, h
76
76
77
77
### CI
78
78
79
-
The `benchmarks` job in [`.github/workflows/ci.yml`](.github/workflows/ci.yml) runs the full suite on `ubuntu-latest` after `build-and-test` succeeds. It uses a faster `CiConfig` (3 warmup × 5 measurement iterations) so the whole suite completes in ~5 min.
79
+
[`.github/workflows/benchmarks.yml`](.github/workflows/benchmarks.yml) runs the CI-tracked core suite (the `CoreBenchmarks` array in `Program.cs`) at full BenchmarkDotNet accuracy, sharded across a parallel matrix. On a PR each shard measures its slice of both the PR head and the `main` tip back-to-back on the same runner, so hardware variance cancels; an aggregate job stitches the shard reports back together.
80
80
81
81
Results are parsed by [`benchmark-action/github-action-benchmark`](https://github.com/benchmark-action/github-action-benchmark) and:
82
82
83
-
-**On a PR**: a comment is posted with the comparison vs the last `main` baseline. If any benchmark regresses by more than **200%** (i.e. is 2× slower or worse), the job fails red. The threshold is deliberately loose because GitHub-hosted runners are noisy — we'll tighten it once we have history to calibrate against.
84
-
-**On a push to `main`**: the new measurement is appended to the `gh-pages`-stored history powering the dashboard at `https://marius-bughiu.github.io/Celerity/dev/bench/` (enable Pages on the `gh-pages` branch once the first run creates it).
83
+
-**On a PR**: a comment is posted with the same-runner A/B comparison vs `main`. Rows that move by more than ±10% *and* beyond the combined standard deviation of both measurements are flagged; the flags are advisory, so a noisy row does not fail the job.
84
+
-**On a push to `main`**: the new measurement is appended to the `gh-pages`-stored history powering the dashboard at <https://marius-bughiu.github.io/Celerity/dev/bench/>.
85
85
86
86
If a change is motivated by performance, include before/after numbers from a local Release run in the PR description — the CI job is a guardrail, not a precision instrument. Numbers without `-c Release` are not useful — BenchmarkDotNet refuses to run in Debug.
87
87
88
+
### The dashboard
89
+
90
+
The dashboard reads BenchmarkDotNet result *names*, so a benchmark's naming is part of its contract with the site. A name that the page's parser does not recognise is dropped silently — the data publishes to `gh-pages` correctly and the card just renders blank, with nothing red anywhere.
91
+
92
+
Two rules keep a benchmark chartable:
93
+
94
+
- Methods are named `{TypeName}_{Op}` — `EnumSet_Contains`, `Dictionary_Lookup`. The type name decides whether the row is the Celerity arm or the BCL baseline (`BCL_TYPES` in the dashboard source), and `{Op}` is what the card is titled.
95
+
- A `[Params]` sweep property must be called **`ItemCount`**. A class may declare no sweep at all — `EnumMap` / `EnumSet` are bounded by the enum universe, so a synthetic item count would chart a dimension that does not exist — in which case the dashboard renders a single bucket. Any *other* property name is rejected rather than charted under an "items" label it does not mean.
96
+
97
+
Adding a collection to the site means updating three lists by hand, since the published data alone does not tell the page what to draw: the ship card in `web/index.html`, and the `COLLECTIONS` array in **both**`web/dev/bench/index.html` and `web/dev/bench/detail.html` (`items: [NO_SWEEP]` for an unparameterized class).
98
+
99
+
[`scripts/check_dashboard_coverage.js`](scripts/check_dashboard_coverage.js) enforces all of this so the failure mode is a red check rather than a blank card. It lifts the `COLLECTIONS` tables and the name parsers out of the dashboard HTML rather than reimplementing them, so it validates the code that actually ships. Run it any time you touch a benchmark or the dashboard:
node scripts/check_dashboard_coverage.js path/to/joined-report-full.json # + verify the data
104
+
```
105
+
106
+
The structural half — the two `COLLECTIONS` arrays agree, and every charted collection has a `{Key}Benchmark` registered in `CoreBenchmarks` — runs on every PR in `ci.yml`. The full check runs in the aggregate job of `benchmarks.yml`, against the merged report, and additionally asserts that every published name parses and that every card resolves to both a BCL and a Celerity measurement.
107
+
88
108
## Versioning
89
109
90
110
Celerity uses [MinVer](https://github.com/adamralph/minver) to derive NuGet package versions exclusively from **git tags**. There is no `<Version>` or `<PackageVersion>` property in any `.csproj` file — the single source of truth is the `v`-prefixed annotated tag on the commit that represents a release.
Copy file name to clipboardExpand all lines: ROADMAP.md
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -225,11 +225,12 @@ The 2026-Q3 review surveyed the shipped surface against the current .NET 8/9/10
225
225
226
226
**A fourth core package: `Celerity.Sorting`.**`Array.Sort` / `MemoryExtensions.Sort` are scalar comparison introsort with no radix, counting, or selection path for primitive keys — and the BCL structurally cannot close it, because `Array.Sort` is contractually in-place while radix needs `O(n)` scratch. That is precisely the flexibility-for-speed trade this project's Vision licenses, against a named BCL counterpart. Would layer on `Celerity.Primitives`, mirroring how `Hashing` and `Collections` layer today. Status: `planned` — see the package-scoping caveat below.
227
227
228
-
**Build- and release-pipeline integrity.**Three guards the repo advertises but does not have.
228
+
**Build- and release-pipeline integrity.**Guards the repo advertises but does not have.
229
229
230
230
- The coverage gate measures one of the six shipping assemblies. `src/coverage.runsettings` filters to `[Celerity]*` with the comment "Measure only the shipping library assembly" — written when there was one. `Celerity.Hashing`, `Celerity.Primitives` and the three showcase packages are unmeasured, while `CONTRIBUTING.md` and `CLAUDE.md` describe the 95%/90% gate as library-wide. Status: `planned`.
231
231
- Nothing can fail after the NuGet push. `release.yml` pushes six packages irreversibly, *then* extracts the release notes and creates the GitHub Release — so an over-long release body (a failure this repo has actually hit) leaves a half-published release. The notes check should be hoisted ahead of the push. Status: `planned`.
232
232
- No API-compatibility gate. Six packages publish on a tag with no `ApiCompat` / `PackageValidation` / public-API-baseline check anywhere in the repo — in a project that already needed a hand-written `TypeForwarders.cs` to survive one assembly split. Status: `planned`.
233
+
- No guard on the benchmark dashboard. The site parses BenchmarkDotNet result *names*, so a benchmark it cannot parse is dropped at render time — the data publishes correctly and the card just goes blank, with no CI signal. `EnumMap` and `EnumSet` had rendered empty since they shipped (they declare no `[Params]` sweep, by design), and `DisjointSet` blanked for five runs when its params property was briefly named `ElementCount`. Status: `done` — the parser now treats the `ItemCount` suffix as optional and renders an unparameterized class as a single bucket, excluded from the headline stats; `scripts/check_dashboard_coverage.js` fails CI on an unparseable name, a card with no measurements behind it, a collection missing from either `COLLECTIONS` array, or one not registered in the CI benchmark suite. It lifts those tables and the parsers out of the dashboard HTML rather than reimplementing them, so the check cannot drift from the page it guards. Tracked in [#301](https://github.com/marius-bughiu/Celerity/issues/301).
233
234
234
235
Two areas were judged real but deliberately deferred rather than rostered: a `Celerity.Statistics` package (DDSketch / reservoir sampling / running moments — a coherent fourth axis, but two new packages in one cycle is too much at once), and a batch of fuzz-target and AOT-smoke-coverage gaps (real, but low expected defect yield; better folded into whichever collection PR lands next than pursued on their own).
0 commit comments