Skip to content

Benchmark Beefy submitFinal by number of signatures #1457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from

Conversation

yrong
Copy link
Contributor

@yrong yrong commented Apr 18, 2025

Resolves: https://linear.app/snowfork/issue/SNO-1483

  • To sync the BEEFY test fixture with the configuration on Polkadot mainnet.
  • Benchmark submitFinal using different numbers of signatures.

Copy link

codecov bot commented Apr 18, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.74%. Comparing base (5e9470a) to head (aeef3a2).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1457   +/-   ##
=======================================
  Coverage   79.74%   79.74%           
=======================================
  Files          20       20           
  Lines         859      859           
  Branches      121      121           
=======================================
  Hits          685      685           
  Misses        151      151           
  Partials       23       23           
Flag Coverage Δ
solidity 79.74% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@yrong
Copy link
Contributor Author

yrong commented Apr 18, 2025

Context

In this PR we aim to benchmark the cost of verifying a higher number of BEEFY signatures.

We set VALIDATOR_SET_SIZE to 600 to match the configuration on Polkadot mainnet, and increased MINIMUM_REQUIRED_SIGNATURES from 17 to 100. For each value, we regenerated the test fixtures and ran forge test with gas-report output enabled.

Benchmark

cd web/packages/test-helpers
./scripts/benchmark-beefy.sh

Report

********** cost of submitFinal by signatures *********
minimum signature: 17, computed signature: 28, gas cost: 797277
minimum signature: 18, computed signature: 29, gas cost: 814124
minimum signature: 19, computed signature: 30, gas cost: 830495
minimum signature: 20, computed signature: 31, gas cost: 845298
minimum signature: 21, computed signature: 32, gas cost: 861728
minimum signature: 22, computed signature: 33, gas cost: 879014
minimum signature: 23, computed signature: 34, gas cost: 895417
minimum signature: 24, computed signature: 35, gas cost: 913743
minimum signature: 25, computed signature: 36, gas cost: 930600
minimum signature: 26, computed signature: 37, gas cost: 948294
minimum signature: 27, computed signature: 38, gas cost: 963144
minimum signature: 28, computed signature: 39, gas cost: 980129
minimum signature: 29, computed signature: 40, gas cost: 997395
minimum signature: 30, computed signature: 41, gas cost: 1014269
minimum signature: 50, computed signature: 61, gas cost: 1346793
minimum signature: 60, computed signature: 71, gas cost: 1510996
minimum signature: 70, computed signature: 81, gas cost: 1659485
minimum signature: 80, computed signature: 91, gas cost: 1829282
minimum signature: 90, computed signature: 101, gas cost: 1998109
minimum signature: 100, computed signature: 111, gas cost: 2164080
image

Conclusion

Basiclly, the cost of submitFinal is linear with the the number of signatures. Adding one more signature costs nearly 16,000 more gas.

@yrong yrong marked this pull request as ready for review April 23, 2025 10:15
@yrong yrong requested a review from alistair-singh April 25, 2025 09:31
@yrong yrong mentioned this pull request May 2, 2025
@vgeddes
Copy link
Collaborator

vgeddes commented May 8, 2025

Since you've implemented the fiat shamir algorithm, surely we can benchmark that instead?

Means we can close this off?

@yrong
Copy link
Contributor Author

yrong commented May 9, 2025

Closed in favor of #1462

@yrong yrong closed this May 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants