gh-1088: Add realistic benchmark tests - #1098

Open

connoraird wants to merge 66 commits into

mainfrom

connor/1088-realistic-benchmarks

Commits on Jul 30, 2026

WIP: adding first benchmark
connoraird
committed
250518d
Increase benchmark size
connoraird
committed
f2f1bb6
Add instruction for setting up UV on Archer2 to benchmarks
connoraird
committed
352a6c4
Add note about clusters
connoraird
committed
cfad05c
Linting
connoraird
committed
2fcdb7e
Fix linting for benchmark and add assertion
connoraird
committed
c7954d8
lint
connoraird
committed
05b5ba2
Add archer2 submission script and fix typos
connoraird
committed
61ab9bd
Rename archer2.sh to archer2-cpu.sh
connoraird
committed
bddcdcc
Update linear_windows to correctly set zeff to a float
connoraird
committed
ee23d2e
Port shear_from_convergence to the Array API
connoraird
committed
089cb58
Add lensing benchmark which does not use pytest
connoraird
committed
e0260ca
Fix linting
connoraird
committed
9ed8194
Add archer2-gpu dependency group wit jax rocm
connoraird
committed
46e4e4f
Add archer2 submission script and README
connoraird
committed
024bbd3
Use correct command
connoraird
committed
5ee36a2
Restructure benchmsrk scipts dirs and add archer2 gpu script
connoraird
committed
a7620fa
Add correct config for running jax with rocm on Archer2 with docs to explain how
connoraird
committed
a081c9d
Lint and add note about python 3.12
connoraird
committed
4a1c7b9
Correctly source setup-env.sh
connoraird
committed
948df55
Pass RUN_PROFILE into to benchmark from submission script
connoraird
committed
640116d
Rename setup-env.sh to be clearer this is for gpu only
connoraird
committed
af21d48
Setup uv venv within setup env scripts for cpu and gpu
connoraird
committed
df9b0cf
Stop using deprecated sheard_from_convergence in benchmark
connoraird
committed
40f41b0
Do not create venv within submitted job
connoraird
committed
1668ba6
Move call to generator within the function to be benchmarked
connoraird
committed
572ecc0
Allow repeats in benchmark
connoraird
committed
1aa7863
corretly handle passing RUN_PROFILE to command
connoraird
committed
ecc18d2
Improve instructions for running benchmarks
connoraird
committed
7725c46
Temporarily update benchmark to run over various values of nside
connoraird
committed
fb12ff9
Temporarily update cpu benchmark to run over various values of nside
connoraird
committed
5c942e6
Add cupy as option in benchmarks
connoraird
committed
3dd1b54
Do not run nside = 2048
connoraird
committed
c3f9cac
Remove cupy
connoraird
committed
3ec81c6
Remove profiling option
connoraird
committed
889a2b1
Simplify building archer2-gpu group and add performance flags to submit-gpu script
connoraird
committed
a85e850
Add config for cosma
connoraird
committed
3013e6f
Add benchmarks to cosma group
connoraird
committed
5979bb4
Remove sourceing env from cosma script
connoraird
committed
600d626
Remove archer2 env setup from cosma submission scripts
connoraird
committed
e4d7ef6
Add instructions for cosma8 benchmarks
connoraird
committed
b8fa0c9
lint
connoraird
committed
d8d0478
Fix output name from cosma sub scripts and add aac6 script
connoraird
committed
7205807
Finish documenting how to run benchmarks and add aac6-gpu group
connoraird
committed
1f7b5de
Add instructions for writing new benchmarks
connoraird
committed
03e1d18
Remove cprofile from run_benchmark as this would not work for gpu anyway
connoraird
committed
9329122
Fix spelling
paddyroddy
committed
f36a663
Fix readthedocs
paddyroddy
committed
46b0f7f
Fix benchmark doc
paddyroddy
committed
fd94baf
Try removing `glass-ext-camb`
paddyroddy
committed
60df134
Applysome suggestions and fix broken import
connoraird
committed
14562d9

Commits on Jul 31, 2026

Archer2 -> ARCHER2
connoraird
committed
22e402c
Removing JAX opt flags as these seem to slow things down on aac6
connoraird
committed
0842f27
Apply Paddy's formatting suggestions
connoraird
committed
e851670
Do not loop throough HSA_XNACK for cosma because it has no shared memory
connoraird
committed
275a3f4
Save alm value
connoraird
committed
c9b74e7
Remove repeated and redundant code from sub script
connoraird
committed
ab869a0
Apply copilot suggestions
connoraird
committed
23d988b
Merge branch 'main' into connor/1088-realistic-benchmarks
paddyroddy
authored
7e334fd
Remove __init__.py from benchmarks
connoraird
committed
afa4ee2
Ensure asynchronous jax work is included in benchmark times and do not benchmark array_api_strict
connoraird
committed
a7c2acc
Revert "Try removing `glass-ext-camb`"
connoraird
committed
6f6ce96
Revert "Fix benchmark doc"
connoraird
committed
51e04c7
Merge branch 'main' into connor/1088-realistic-benchmarks
connoraird
authored
78c045e
Add `ty` to benchmarks
paddyroddy
committed
69f9c6a

Commits on Aug 12, 2026

Remove 'performance improving' flags as they don't appear to work
connoraird
committed
ac5df4b