Skip to content

Conversation

@roryharr
Copy link
Contributor

@roryharr roryharr commented Nov 4, 2025

Problem Statement:

When upgrading Rust from 1.86 to 1.87, it was found the performance of the PoH hashing degraded. Since then, all rust upgrades have been blocked.

Solution

After analysis it was found that inlining differences during the LLVM compilation were causing the performance difference. By force inlining the performance improved in current nightly to better than 1.87

Results were verified with proof of hash bench:
1.86 without force inline

test bench_poh_hash                   ... bench:   1,397,297.50 ns/iter (+/- 4,870.21)

1.86 with force inline

test bench_poh_hash                   ... bench:   1,391,815.20 ns/iter (+/- 2,922.94)

nightly without force inline

test bench_poh_hash                   ... bench:   1,474,467.60 ns/iter (+/- 6,121.96)

nightly with force inline

test bench_poh_hash                   ... bench:   1,348,648.70 ns/iter (+/- 2,663.92)

And on chain:

Below shows performance with nightly
Before the restart, nightly without the fix is running.
After the restart, nightly with the fix is running.
It is also better than 1.86 (which is not shown on the graph)
Image

anza-xyz/agave#8869

@steviez
Copy link
Collaborator

steviez commented Nov 4, 2025

anza-xyz/agave#8869

I'm trying to reproduce Rory's results in this issue, I think it would be good for us to figure that out before merging this PR. Discussion still happening over there

Copy link
Collaborator

@steviez steviez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to reproduce Rory's results in this issue, I think it would be good for us to figure that out before merging this PR. Discussion still happening over there

Ok, Rory and I figured out the discrepancy between our results (details back in the Agave issue) so I feel comfortable shipping this one now

Copy link
Collaborator

@joncinque joncinque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thanks for digging in and finding a fix!

@joncinque joncinque added the backport maintenance/v3.x Automatically creates a backport PR to `maintenance/v3.x` once the PR is closed label Nov 4, 2025
@joncinque
Copy link
Collaborator

We're in the process of publishing breaking changes on the sdk, so I'll backport to v3 so Agave can get the fix quicker

@roryharr roryharr merged commit f544553 into master Nov 4, 2025
27 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 4, 2025
@steviez steviez mentioned this pull request Nov 4, 2025
6 tasks
joncinque pushed a commit that referenced this pull request Nov 4, 2025
… by rust upgrade (#427)

Fix hasher performance degredation caused by rust upgrade (#426)

(cherry picked from commit f544553)

Co-authored-by: Rory Harris <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport maintenance/v3.x Automatically creates a backport PR to `maintenance/v3.x` once the PR is closed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants