Add benchmark suite and per-PR benchmark CI workflow - #185
Merged
Conversation
Adds a bench/ suite (element creation, component mount depth, attribute update at depth, keyed/unkeyed v-for reconciliation, and unkeyed list grow/shrink) using pytest-benchmark with DictRenderer and synchronous event loop, mirroring the setup in the sibling observ repo. The new benchmark workflow runs the suite twice on every PR - once against the master version of collagraph/ and once against the PR version - and fails when mean time regresses more than 5%. The CI test job is scoped to the tests directory so the matrix does not execute benchmarks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 6, 2026
The allocation burst of list reconciliation triggers collection pauses in some rounds, which inflated stddev on the grow benchmarks to ~40% of the mean - far too noisy for the 5% regression gate in CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
3 tasks
Merged
berendkleinhaneveld
added a commit
that referenced
this pull request
Jul 21, 2026
Features: - Add pure-Python view API as alternative to cgx templates (#193) - Support text elements for PySide widgets that display text (#191) Fixes & internals: - Fragment parenting overhaul (#162) - Fix PyInstaller hook for CGX files inside packages (#184) - Write compiled AST to temp file when CGX_DEBUG is set (#175) Performance: - Speed up mount path: cheap arity check, reuse first(), leaner emit (#186) - Cache Fragment._component_parent() lookups (#187) - Avoid redundant anchor lookups in Fragment.anchor() and unkeyed v-for (#188) Documentation: - Add MkDocs documentation with GitHub Pages deployment (#176) - Add internals architecture documentation page (#194) - Add docs badge and links to README (#192) Tooling & CI: - Add benchmark suite and per-PR benchmark CI workflow (#185) - Make benchmark CI guard robust against run-to-run noise (#196) - Update GitHub actions from Node 20 to Node 24 (#190) - Migrate from pre-commit to prek (#195) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First PR of the performance series that supersedes #183 (split so each optimization is proven independently by CI).
pytest-benchmarksuite underbench/, mirroring the siblingobservrepo's setup. Scenarios: plain/bound element creation (10/100/1000), nested component mount (depth 10/50/200), attribute update at depth (10/50/200), keyed and unkeyedv-forreconciliation (append/prepend/reverse/shuffle × 10/100/1000), and a new unkeyed list grow/shrink scenario (100/1000, with and without sibling elements) that exposes the current O(n²) anchor recomputation when appending items..github/workflows/benchmark.yml: on every PR it runs the suite against the master version ofcollagraph/and against the PR version, fails when mean time regresses >5%, and uploads.benchmarks/as an artifact. All benchmarks useDictRenderer+ a synchronous event loop, so no Qt/GL system dependencies are needed.testsso the 5-way Python matrix no longer collects the benchmark suite.pytest-benchmark/pytest-timeoutto the dev group, benchmark display columns, a ruff per-file ignore forbench/*, and gitignores.benchmarks/.Since this PR does not touch
collagraph/source, the benchmark workflow on this PR compares identical code and should report ~0% deltas — validating the pipeline itself. Follow-up PRs with the actual optimizations will then show their gains automatically in this workflow.Test plan
uv run pytest tests -q— 349 passed, 1 skippeduv run ruff check ./uv run ruff format --check .— cleanuv run pytest bench --collect-only— 42 benchmarks collected; new grow/shrink benchmarks run green locallybench/🤖 Generated with Claude Code