Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 8, 2026

Adds automated performance regression detection for pull requests. Benchmarks run on both PR and base branches, with results posted as a comparison comment.

Implementation

  • Workflow: .github/workflows/benchmark.yml triggers on PRs to master
  • Execution: Runs all tests/benchmarks/ tests via pytest-benchmark on both branches
  • Comparison: JavaScript analysis calculates percentage changes, flags regressions >10%
  • Output: Posts formatted table with visual indicators (🚀 faster, ⚠️ slower, ✅ similar, 🆕 new)

Comment Format

## 📊 Performance Benchmark Results

### ⚠️ Significant Performance Changes Detected
- test_benchmark_rbac_model: 15.23% slower

### Detailed Comparison
| Benchmark | Status | PR Mean (s) | Base Mean (s) | Change |
|-----------|--------|-------------|---------------|--------|
| test_benchmark_raw || 3.0000e-7 | 3.0000e-7 | 0.00% |
| test_benchmark_rbac_model | ⚠️ SLOWER | 5.7615e-6 | 5.0000e-6 | 15.23% |

Leverages existing pytest-benchmark infrastructure, no new dependencies.

Original prompt

This section details on the original issue you should resolve

<issue_title>ci: add automatic performance benchmark CI for Pull Requests</issue_title>
<issue_description>An automatic CI workflow for performance benchmarking is proposed:

  • Benchmark tests are executed automatically when a new Pull Request is submitted.

  • Post a clear comparison table in PR comments to show speed differences.

  • Help maintainers catch performance regressions before merging.</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@hsluoyz hsluoyz marked this pull request as ready for review January 8, 2026 06:04
Copilot AI changed the title [WIP] Add automatic performance benchmark CI for Pull Requests ci: add automatic performance benchmark CI for pull requests Jan 8, 2026
Copilot AI requested a review from hsluoyz January 8, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: add automatic performance benchmark CI for Pull Requests

3 participants