Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ jobs:
exit 0
fi
echo "have_head=true" >> "$GITHUB_OUTPUT"
echo "head_shards=${#head_files[@]}" >> "$GITHUB_OUTPUT"

# Stitch every shard's joined report back into one: keep shard 0's metadata,
# concatenate all shards' Benchmarks arrays.
Expand Down Expand Up @@ -418,3 +419,11 @@ jobs:
git -c user.name="github-actions" -c user.email="github-actions@github.com" \
commit -m "Sync custom dashboard from ${GITHUB_SHA:0:7}"
git push origin gh-pages

# Deliberately the LAST step: a dashboard-wiring mistake must not cost us the
# measurement, so the comment and the gh-pages publish above happen first and this
# only reddens the job. Skipped unless every shard reported, since a partial merge
# is legitimately missing whole benchmark classes (the job runs with if: always()).
- name: Check dashboard coverage of the merged report
if: steps.merge.outputs.have_head == 'true' && steps.merge.outputs.head_shards == env.SHARD_TOTAL
run: node scripts/check_dashboard_coverage.js /tmp/pr-report-full.json
15 changes: 15 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,21 @@ jobs:
if-no-files-found: warn
retention-days: 7

# The benchmark workflow only fires on `src/**`, so a dashboard-only change never
# reaches the report-backed run of this same script in benchmarks.yml. These are the
# structural checks, which need no measurements and cost seconds.
dashboard-coverage:
name: dashboard-coverage
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
filter: tree:0

- name: Check dashboard wiring
run: node scripts/check_dashboard_coverage.js

aot-publish:
name: aot-publish (linux-x64, ${{ matrix.tfm }})
runs-on: ubuntu-latest
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ All notable changes to Celerity are documented here. This project follows [Keep

### Fixed

- **The `EnumMap` and `EnumSet` cards on the benchmark dashboard rendered empty.** The page required an `(ItemCount: N)` suffix on every result name, and both benchmarks deliberately declare no item-count sweep, so their measurements were published and then discarded at render time. Both cards now chart their real numbers, and unparameterized benchmarks are excluded from the headline speedup stats. Closes [#301](https://github.com/marius-bughiu/Celerity/issues/301).
- A blank dashboard card is now a red CI check rather than a silent gap: `scripts/check_dashboard_coverage.js` fails when a published result name is unparseable, when a card has no measurements behind it, or when a charted collection is missing from either `COLLECTIONS` array or from the CI benchmark suite. Closes [#301](https://github.com/marius-bughiu/Celerity/issues/301).
- **The coverage gate measured only one of the six shipped packages.** Coverlet's assembly filter is exact-match, so `Celerity.Hashing`, `Celerity.Primitives`, and the three showcase packages had been outside the gate since the 2.0.0 package split — any of them could have dropped to 0% with CI green. All six are now measured, the gaps that exposed are backfilled to **100% line and branch** coverage, and the floor is raised from 95%/90% to match. Closes [#314](https://github.com/marius-bughiu/Celerity/issues/314).

## [2.4.0] - 2026-07-26
Expand Down
26 changes: 23 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,15 +76,35 @@ dotnet run -c Release -- --filter '*' # run everything with the default (slow, h

### CI

The `benchmarks` job in [`.github/workflows/ci.yml`](.github/workflows/ci.yml) runs the full suite on `ubuntu-latest` after `build-and-test` succeeds. It uses a faster `CiConfig` (3 warmup × 5 measurement iterations) so the whole suite completes in ~5 min.
[`.github/workflows/benchmarks.yml`](.github/workflows/benchmarks.yml) runs the CI-tracked core suite (the `CoreBenchmarks` array in `Program.cs`) at full BenchmarkDotNet accuracy, sharded across a parallel matrix. On a PR each shard measures its slice of both the PR head and the `main` tip back-to-back on the same runner, so hardware variance cancels; an aggregate job stitches the shard reports back together.

Results are parsed by [`benchmark-action/github-action-benchmark`](https://github.com/benchmark-action/github-action-benchmark) and:

- **On a PR**: a comment is posted with the comparison vs the last `main` baseline. If any benchmark regresses by more than **200%** (i.e. is 2× slower or worse), the job fails red. The threshold is deliberately loose because GitHub-hosted runners are noisy — we'll tighten it once we have history to calibrate against.
- **On a push to `main`**: the new measurement is appended to the `gh-pages`-stored history powering the dashboard at `https://marius-bughiu.github.io/Celerity/dev/bench/` (enable Pages on the `gh-pages` branch once the first run creates it).
- **On a PR**: a comment is posted with the same-runner A/B comparison vs `main`. Rows that move by more than ±10% *and* beyond the combined standard deviation of both measurements are flagged; the flags are advisory, so a noisy row does not fail the job.
- **On a push to `main`**: the new measurement is appended to the `gh-pages`-stored history powering the dashboard at <https://marius-bughiu.github.io/Celerity/dev/bench/>.

If a change is motivated by performance, include before/after numbers from a local Release run in the PR description — the CI job is a guardrail, not a precision instrument. Numbers without `-c Release` are not useful — BenchmarkDotNet refuses to run in Debug.

### The dashboard

The dashboard reads BenchmarkDotNet result *names*, so a benchmark's naming is part of its contract with the site. A name that the page's parser does not recognise is dropped silently — the data publishes to `gh-pages` correctly and the card just renders blank, with nothing red anywhere.

Two rules keep a benchmark chartable:

- Methods are named `{TypeName}_{Op}` — `EnumSet_Contains`, `Dictionary_Lookup`. The type name decides whether the row is the Celerity arm or the BCL baseline (`BCL_TYPES` in the dashboard source), and `{Op}` is what the card is titled.
- A `[Params]` sweep property must be called **`ItemCount`**. A class may declare no sweep at all — `EnumMap` / `EnumSet` are bounded by the enum universe, so a synthetic item count would chart a dimension that does not exist — in which case the dashboard renders a single bucket. Any *other* property name is rejected rather than charted under an "items" label it does not mean.

Adding a collection to the site means updating three lists by hand, since the published data alone does not tell the page what to draw: the ship card in `web/index.html`, and the `COLLECTIONS` array in **both** `web/dev/bench/index.html` and `web/dev/bench/detail.html` (`items: [NO_SWEEP]` for an unparameterized class).

[`scripts/check_dashboard_coverage.js`](scripts/check_dashboard_coverage.js) enforces all of this so the failure mode is a red check rather than a blank card. It lifts the `COLLECTIONS` tables and the name parsers out of the dashboard HTML rather than reimplementing them, so it validates the code that actually ships. Run it any time you touch a benchmark or the dashboard:

```bash
node scripts/check_dashboard_coverage.js # structural checks
node scripts/check_dashboard_coverage.js path/to/joined-report-full.json # + verify the data
```

The structural half — the two `COLLECTIONS` arrays agree, and every charted collection has a `{Key}Benchmark` registered in `CoreBenchmarks` — runs on every PR in `ci.yml`. The full check runs in the aggregate job of `benchmarks.yml`, against the merged report, and additionally asserts that every published name parses and that every card resolves to both a BCL and a Celerity measurement.

## Versioning

Celerity uses [MinVer](https://github.com/adamralph/minver) to derive NuGet package versions exclusively from **git tags**. There is no `<Version>` or `<PackageVersion>` property in any `.csproj` file — the single source of truth is the `v`-prefixed annotated tag on the commit that represents a release.
Expand Down
3 changes: 2 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,11 +225,12 @@ The 2026-Q3 review surveyed the shipped surface against the current .NET 8/9/10

**A fourth core package: `Celerity.Sorting`.** `Array.Sort` / `MemoryExtensions.Sort` are scalar comparison introsort with no radix, counting, or selection path for primitive keys — and the BCL structurally cannot close it, because `Array.Sort` is contractually in-place while radix needs `O(n)` scratch. That is precisely the flexibility-for-speed trade this project's Vision licenses, against a named BCL counterpart. Would layer on `Celerity.Primitives`, mirroring how `Hashing` and `Collections` layer today. Status: `planned` — see the package-scoping caveat below.

**Build- and release-pipeline integrity.** Three guards the repo advertises but does not have.
**Build- and release-pipeline integrity.** Guards the repo advertises but does not have.

- The coverage gate measures one of the six shipping assemblies. `src/coverage.runsettings` filters to `[Celerity]*` with the comment "Measure only the shipping library assembly" — written when there was one. `Celerity.Hashing`, `Celerity.Primitives` and the three showcase packages are unmeasured, while `CONTRIBUTING.md` and `CLAUDE.md` describe the 95%/90% gate as library-wide. Status: `planned`.
- Nothing can fail after the NuGet push. `release.yml` pushes six packages irreversibly, *then* extracts the release notes and creates the GitHub Release — so an over-long release body (a failure this repo has actually hit) leaves a half-published release. The notes check should be hoisted ahead of the push. Status: `planned`.
- No API-compatibility gate. Six packages publish on a tag with no `ApiCompat` / `PackageValidation` / public-API-baseline check anywhere in the repo — in a project that already needed a hand-written `TypeForwarders.cs` to survive one assembly split. Status: `planned`.
- No guard on the benchmark dashboard. The site parses BenchmarkDotNet result *names*, so a benchmark it cannot parse is dropped at render time — the data publishes correctly and the card just goes blank, with no CI signal. `EnumMap` and `EnumSet` had rendered empty since they shipped (they declare no `[Params]` sweep, by design), and `DisjointSet` blanked for five runs when its params property was briefly named `ElementCount`. Status: `done` — the parser now treats the `ItemCount` suffix as optional and renders an unparameterized class as a single bucket, excluded from the headline stats; `scripts/check_dashboard_coverage.js` fails CI on an unparseable name, a card with no measurements behind it, a collection missing from either `COLLECTIONS` array, or one not registered in the CI benchmark suite. It lifts those tables and the parsers out of the dashboard HTML rather than reimplementing them, so the check cannot drift from the page it guards. Tracked in [#301](https://github.com/marius-bughiu/Celerity/issues/301).

Two areas were judged real but deliberately deferred rather than rostered: a `Celerity.Statistics` package (DDSketch / reservoir sampling / running moments — a coherent fourth axis, but two new packages in one cycle is too much at once), and a batch of fuzz-target and AOT-smoke-coverage gaps (real, but low expected defect yield; better folded into whichever collection PR lands next than pursued on their own).

Expand Down
Loading
Loading