Skip to content

Commit f0ac195

Browse files
authored
👷 Track Rust performance benchmarks on CI (#61)
* ⬆️ Bump criterion from 0.5.1 to 0.7.0 Bumps [criterion](https://github.com/bheisler/criterion.rs) from 0.5.1 to 0.7.0. - [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md) - [Commits](bheisler/criterion.rs@0.5.1...0.7.0) No need to use the criterion.rs fork set at f07ae72/#27, because the original error `assertion failed: slice.len() > 1 && slice.iter().all(|x| !x.is_nan())` can been fixed by setting nresamples to 2, yeah I know right! * 👷 Run criterion benchmarks on CI with cargo-codspeed Use codspeed-criterion-compat crate inplace of criterion, and setup CI job to run the Rust benchmarks! * 💚 Run on ghcr.io/osgeo/gdal:ubuntu-small-3.11.5 container And bump to georust/gdal version with GDAL 3.11 bindings. * 🚨 Set git config core.longpaths to true Fix `failed to update submodule `gdal-src/source` ... Caused by: path too long: 'C:/Users/runneradmin/.cargo/git/checkouts/gdal-3b51095beae5b4fc/3df2c28/gdal-src/source/...`. * 💚 Cargo remove gdal-src and gdal-sys in benchmark tests Fix `undefined reference to `GDALGetRasterBand'` and various other linker errors. Just rely on existing GDAL installation within the docker container, rather than the pre-built bindings. * ⚗️ Benchmark with LiberTIFF driver Benchmark reading with GDAL 3.11's LiberTIFF driver, using 4 threads. Also renumbered the benchmark IDs to start from 0. * 🥚 Do host to device copy when 'cuda' flag is enabled Ensure fair comparison between nvTIFF which reads to CUDA memory (even though it's the fastest already), and the GDAL LiberTIFF and image-tiff backend readers.
1 parent f8f4f64 commit f0ac195

File tree

5 files changed

+256
-65
lines changed

5 files changed

+256
-65
lines changed

.github/workflows/benchmarks.yml

Lines changed: 53 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Run performance benchmarks
22
#
3-
# Continuous benchmarking using pytest-codspeed. Measures the execution speed
4-
# of tests marked with @pytest.mark.benchmark decorator.
3+
# Continuous benchmarking with CodSpeed. Measures the execution speed of Rust benchmarks
4+
# via cargo-codspeed, and Python tests marked with @pytest.mark.benchmark decorator.
55

66
name: Benchmarks
77

@@ -16,10 +16,56 @@ on:
1616
# performance analysis in order to generate initial data.
1717
workflow_dispatch:
1818

19+
env:
20+
CARGO_TERM_COLOR: always
21+
1922
permissions: {}
2023

2124
jobs:
22-
benchmarks:
25+
rust-benchmarks:
26+
name: "Run Rust benchmarks"
27+
runs-on: ubuntu-24.04
28+
container:
29+
image: ghcr.io/osgeo/gdal:ubuntu-small-3.11.5
30+
options: --privileged
31+
steps:
32+
- name: Install dev dependencies and setup git
33+
run: |
34+
apt update
35+
apt install -y build-essential cmake git libclang-dev pkg-config
36+
git config --global --add safe.directory $GITHUB_WORKSPACE
37+
38+
- name: Checkout repository
39+
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
40+
with:
41+
persist-credentials: false
42+
43+
- name: Download sample GeoTIFF
44+
run: |
45+
curl --create-dir --remote-name --output-dir benches https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/37/M/BV/2024/10/S2A_37MBV_20241029_0_L2A/TCI.tif
46+
ls -lh benches/
47+
48+
- name: Setup rust toolchain, cache and cargo-codspeed binary
49+
uses: moonrepo/setup-rust@ede6de059f8046a5e236c94046823e2af11ca670 # v1.2.2
50+
with:
51+
channel: 1.85.0 # msrv
52+
cache: false
53+
cache-target: release
54+
bins: cargo-codspeed
55+
56+
- name: Build the benchmark target(s)
57+
run: |
58+
cargo remove --dev gdal-src gdal-sys
59+
cargo codspeed build
60+
61+
- name: Run the benchmarks
62+
uses: CodSpeedHQ/action@6a8e2b874c338bf81cc5e8be715ada75908d3871 # v4.3.4
63+
with:
64+
mode: instrumentation
65+
run: cargo codspeed run
66+
67+
python-benchmarks:
68+
name: "Run Python benchmarks"
2369
runs-on: ubuntu-24.04
2470
defaults:
2571
run:
@@ -28,7 +74,7 @@ jobs:
2874
steps:
2975
# Checkout current git repository
3076
- name: Checkout
31-
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
77+
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
3278
with:
3379
persist-credentials: false
3480

@@ -55,7 +101,7 @@ jobs:
55101
56102
# Run the benchmark tests
57103
- name: Run benchmarks
58-
uses: CodSpeedHQ/action@0010eb0ca6e89b80c88e8edaaa07cfe5f3e6664d # v3.5.0
104+
uses: CodSpeedHQ/action@6a8e2b874c338bf81cc5e8be715ada75908d3871 # v4.3.4
59105
with:
60-
run: |
61-
python -m pytest --verbose --codspeed
106+
mode: instrumentation
107+
run: python -m pytest --verbose --codspeed

.github/workflows/ci.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,9 @@ jobs:
134134
python-version: "3.12"
135135
architecture: ${{ matrix.target }}
136136

137+
- name: Support longpaths
138+
run: git config --system core.longpaths true
139+
137140
- name: Build wheels
138141
uses: PyO3/maturin-action@aef21716ff3dcae8a1c301d23ec3e4446972a6e3 # v1.49.1
139142
with:

0 commit comments

Comments
 (0)