Skip to content

Commit bcfb5b2

Browse files
DaniPopesampcode-com
DaniPopes
and
ampcode-com
committed
perf: add PGO optimization to release builds
Add PGO (Profile-Guided Optimization) to cargo-dist release workflow: - .github/scripts/build-pgo.sh: Uses cargo-pgo to build instrumented binary, gather profiles from testdata/*.sol, and set RUSTFLAGS - .github/workflows/dist-build-setup.yml: Runs PGO script before dist - .github/workflows/test-pgo-bolt.yml: Test workflow that builds baseline/PGO/PGO+BOLT binaries and benchmarks with hyperfine The test workflow also includes BOLT optimization which provides additional ~1% speedup on top of PGO's ~7% improvement. BOLT is not yet integrated into the release workflow as it requires post-build processing that cargo-dist doesn't support via hooks. Benchmark results (GitHub Actions runner): - Small files: PGO+BOLT is 8% faster than baseline - Large files: PGO+BOLT is 8% faster than baseline Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019bf7a1-0015-74f4-a5f6-c71ce48b4663
1 parent 0736fbf commit bcfb5b2

2 files changed

Lines changed: 163 additions & 5 deletions

File tree

‎.github/scripts/build-pgo.sh‎

Lines changed: 60 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,69 @@
11
#!/usr/bin/env bash
2+
# PGO (Profile-Guided Optimization) build script for cargo-dist releases.
3+
#
4+
# This script gathers PGO profiles from representative workloads and sets
5+
# RUSTFLAGS so that cargo-dist's build uses the profiles.
26
set -euo pipefail
37

48
cd "$(dirname "$0")/../.."
59

10+
PROFILE=dist
11+
FEATURES=cli,asm,mimalloc
12+
13+
# Install cargo-pgo and llvm-tools-preview.
614
cargo install cargo-pgo
715
rustup component add llvm-tools-preview
816

9-
cargo pgo build -- --profile dist --features cli,asm,mimalloc
10-
find testdata -maxdepth 1 -name '*.sol' ! -name 'Optimism.sol' -exec \
11-
./target/x86_64-unknown-linux-gnu/dist/solar {} +
17+
# Get list of test files for profiling.
18+
readarray -t TESTDATA < <(find testdata -maxdepth 1 -name '*.sol' ! -name 'Optimism.sol')
19+
echo "Profiling with ${#TESTDATA[@]} files: ${TESTDATA[*]}"
20+
21+
# ============================================================================
22+
# Step 1: Build PGO-instrumented binary
23+
# ============================================================================
24+
echo "=== Building PGO-instrumented binary ==="
25+
cargo pgo build -- --profile "$PROFILE" --features "$FEATURES"
26+
27+
# ============================================================================
28+
# Step 2: Gather PGO profiles
29+
# ============================================================================
30+
echo "=== Gathering PGO profiles ==="
31+
cargo pgo run -- --profile "$PROFILE" --features "$FEATURES" -- "${TESTDATA[@]}"
32+
33+
# ============================================================================
34+
# Step 3: Merge profiles and set RUSTFLAGS for cargo-dist build
35+
# ============================================================================
36+
echo "=== Merging PGO profiles ==="
37+
38+
# Find llvm-profdata from rustup's llvm-tools.
39+
LLVM_PROFDATA=$(find "$(rustc --print sysroot)" -name llvm-profdata -type f | head -1)
40+
if [[ -z "$LLVM_PROFDATA" ]]; then
41+
echo "Error: llvm-profdata not found"
42+
exit 1
43+
fi
44+
45+
# Merge raw profiles into a single profdata file.
46+
PROFILE_DIR="$PWD/target/pgo-profiles"
47+
MERGED_PROFILE="$PROFILE_DIR/merged.profdata"
48+
"$LLVM_PROFDATA" merge -o "$MERGED_PROFILE" "$PROFILE_DIR"
49+
50+
if [[ ! -f "$MERGED_PROFILE" ]]; then
51+
echo "Error: Failed to create merged profile at $MERGED_PROFILE"
52+
exit 1
53+
fi
54+
55+
echo "=== Setting up RUSTFLAGS for PGO-optimized build ==="
56+
57+
# Set RUSTFLAGS for the subsequent dist build.
58+
# -Cprofile-use: Use PGO profiles for optimization
59+
RUSTFLAGS="-Cprofile-use=${MERGED_PROFILE}"
60+
61+
if [[ -n "${GITHUB_ENV:-}" ]]; then
62+
echo "RUSTFLAGS=${RUSTFLAGS}" >> "$GITHUB_ENV"
63+
echo "Exported RUSTFLAGS to GITHUB_ENV"
64+
else
65+
echo "RUSTFLAGS=${RUSTFLAGS}"
66+
export RUSTFLAGS
67+
fi
1268

13-
llvm-profdata merge -o target/pgo-profiles/merged.profdata target/pgo-profiles
14-
echo "RUSTFLAGS=-Cprofile-use=$PWD/target/pgo-profiles/merged.profdata" >> "$GITHUB_ENV"
69+
echo "=== PGO build setup complete ==="

‎.github/workflows/test-pgo-bolt.yml‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
name: Test PGO+BOLT
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
workflow_dispatch:
8+
9+
jobs:
10+
test-pgo-bolt:
11+
name: Test PGO+BOLT build
12+
runs-on: ubuntu-22.04
13+
env:
14+
PROFILE: dist
15+
FEATURES: cli,asm,mimalloc
16+
steps:
17+
- uses: actions/checkout@v4
18+
with:
19+
persist-credentials: false
20+
submodules: recursive
21+
22+
- name: Install Rust
23+
run: rustup update stable --no-self-update && rustup default stable
24+
25+
- name: Build baseline (no optimization)
26+
run: |
27+
cargo build --profile $PROFILE --features $FEATURES
28+
cp target/$PROFILE/solar /tmp/solar-baseline
29+
ls -lh /tmp/solar-baseline
30+
31+
- name: Clean for PGO build
32+
run: cargo clean
33+
34+
- name: Run build-pgo.sh script
35+
run: |
36+
export GITHUB_ENV=$(mktemp)
37+
.github/scripts/build-pgo.sh
38+
cat "$GITHUB_ENV"
39+
40+
- name: Build with PGO (using RUSTFLAGS from script)
41+
run: |
42+
export RUSTFLAGS="-Cprofile-use=$PWD/target/pgo-profiles/merged.profdata"
43+
cargo build --profile $PROFILE --features $FEATURES
44+
cp target/$PROFILE/solar /tmp/solar-pgo
45+
ls -lh /tmp/solar-pgo
46+
47+
- name: Install BOLT
48+
run: |
49+
wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key | sudo tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc > /dev/null
50+
echo "deb http://apt.llvm.org/jammy/ llvm-toolchain-jammy-21 main" | sudo tee /etc/apt/sources.list.d/llvm.list > /dev/null
51+
sudo apt-get update -qq
52+
sudo apt-get install -y -qq bolt-21
53+
sudo ln -sf /usr/bin/llvm-bolt-21 /usr/local/bin/llvm-bolt
54+
sudo ln -sf /usr/bin/merge-fdata-21 /usr/local/bin/merge-fdata
55+
56+
- name: Build BOLT-instrumented binary (with PGO)
57+
run: cargo pgo bolt build --with-pgo -- --profile $PROFILE --features $FEATURES
58+
59+
- name: Gather BOLT profiles
60+
run: |
61+
readarray -t TESTDATA < <(find testdata -maxdepth 1 -name '*.sol' ! -name 'Optimism.sol')
62+
BOLT_BIN=$(find target -name "solar" -path "*/bolt/instrumented/*" -type f | head -1)
63+
"$BOLT_BIN" "${TESTDATA[@]}" || true
64+
65+
- name: Build BOLT-optimized binary
66+
run: |
67+
cargo pgo bolt optimize --with-pgo -- --profile $PROFILE --features $FEATURES
68+
BOLT_OPT=$(find target -name "solar" -path "*/bolt/optimized/*" -type f | head -1)
69+
cp "$BOLT_OPT" /tmp/solar-pgo-bolt
70+
ls -lh /tmp/solar-pgo-bolt
71+
72+
- name: Verify binaries work
73+
run: |
74+
/tmp/solar-baseline --version && /tmp/solar-baseline testdata/Counter.sol
75+
/tmp/solar-pgo --version && /tmp/solar-pgo testdata/Counter.sol
76+
/tmp/solar-pgo-bolt --version && /tmp/solar-pgo-bolt testdata/Counter.sol
77+
78+
- name: Install hyperfine
79+
run: |
80+
wget -qO- https://github.com/sharkdp/hyperfine/releases/download/v1.19.0/hyperfine-v1.19.0-x86_64-unknown-linux-gnu.tar.gz | tar xz
81+
sudo mv hyperfine-v1.19.0-x86_64-unknown-linux-gnu/hyperfine /usr/local/bin/
82+
83+
- name: Benchmark comparison
84+
run: |
85+
hyperfine --warmup 3 --min-runs 20 \
86+
'/tmp/solar-baseline testdata/Counter.sol testdata/Vm.sol testdata/console.sol' \
87+
'/tmp/solar-pgo testdata/Counter.sol testdata/Vm.sol testdata/console.sol' \
88+
'/tmp/solar-pgo-bolt testdata/Counter.sol testdata/Vm.sol testdata/console.sol' \
89+
--export-markdown /tmp/bench-small.md
90+
cat /tmp/bench-small.md
91+
92+
hyperfine --warmup 2 --min-runs 10 \
93+
'/tmp/solar-baseline testdata/Solady.sol testdata/Seaport.sol' \
94+
'/tmp/solar-pgo testdata/Solady.sol testdata/Seaport.sol' \
95+
'/tmp/solar-pgo-bolt testdata/Solady.sol testdata/Seaport.sol' \
96+
--export-markdown /tmp/bench-large.md
97+
cat /tmp/bench-large.md
98+
99+
- name: Upload benchmark results
100+
uses: actions/upload-artifact@v4
101+
with:
102+
name: benchmark-results
103+
path: /tmp/bench-*.md

0 commit comments

Comments
 (0)