Skip to content

Add performance testing workflows and enhance reporting #314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
karthiknadig wants to merge 6 commits into
base: main
Choose a base branch
from
+1,478 −14
Open

Add performance testing workflows and enhance reporting #314

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6deac53
Add performance testing workflow and related configurations
karthiknadig Feb 2, 2026
f5dedac
Add performance baseline workflow and enhance performance test reporting
karthiknadig Feb 2, 2026
8a7bcfa
Fix formatting issues in performance report scripts for consistency
karthiknadig Feb 2, 2026
db2d64d
Add continue-on-error option for performance test steps
karthiknadig Feb 3, 2026
f3ea97e
Enhance executable path resolution for performance tests to support t…
karthiknadig Feb 3, 2026
e2efde1
Add in-progress comments for performance tests on Linux, Windows, and…
karthiknadig Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions .github/workflows/perf-baseline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: Performance Baseline

on:
push:
branches:
- main

permissions:
contents: read

jobs:
performance:
name: Performance Baseline
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
target: x86_64-unknown-linux-musl
- os: windows-latest
target: x86_64-pc-windows-msvc
- os: macos-latest
target: x86_64-apple-darwin
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set Python to PATH
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Add Conda to PATH (Windows)
if: startsWith(matrix.os, 'windows')
run: |
$path = $env:PATH + ";" + $env:CONDA + "\condabin"
echo "PATH=$path" >> $env:GITHUB_ENV

- name: Add Conda to PATH (Linux)
if: startsWith(matrix.os, 'ubuntu')
run: echo "PATH=$PATH:$CONDA/condabin" >> $GITHUB_ENV
shell: bash

- name: Install Conda + add to PATH (macOS)
if: startsWith(matrix.os, 'macos')
run: |
curl -o ~/miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
bash ~/miniconda.sh -b -p ~/miniconda
echo "PATH=$PATH:$HOME/miniconda/bin" >> $GITHUB_ENV
echo "CONDA=$HOME/miniconda" >> $GITHUB_ENV
shell: bash

- name: Create test Conda environment
run: conda create -n perf-test-env python=3.12 -y

- name: Create test venv
run: python -m venv .venv
shell: bash

- name: Rust Tool Chain setup
uses: dtolnay/rust-toolchain@stable
with:
toolchain: stable
targets: ${{ matrix.target }}

- name: Cargo Fetch
run: cargo fetch
shell: bash

- name: Build Release
run: cargo build --release --target ${{ matrix.target }}
shell: bash

- name: Run Performance Tests
continue-on-error: true
run: cargo test --release --features ci-perf --target ${{ matrix.target }} --test e2e_performance test_performance_summary -- --nocapture 2>&1 | tee perf-output.txt
env:
RUST_BACKTRACE: 1
RUST_LOG: warn
shell: bash

- name: Extract Performance Metrics
id: metrics
run: |
# Extract JSON metrics from test output
if grep -q "JSON metrics:" perf-output.txt; then
# Extract lines after "JSON metrics:" until the closing brace
sed -n '/JSON metrics:/,/^}/p' perf-output.txt | tail -n +2 > metrics.json
echo "Metrics extracted:"
cat metrics.json
else
echo '{"server_startup_ms": 0, "full_refresh_ms": 0, "environments_count": 0}' > metrics.json
echo "No metrics found, created empty metrics"
fi
shell: bash

- name: Upload Performance Baseline Artifact
uses: actions/upload-artifact@v4
with:
name: perf-baseline-${{ matrix.os }}
path: metrics.json
retention-days: 90
Loading
Loading