-
Notifications
You must be signed in to change notification settings - Fork 141
asset-hub-polkadot: regenerate weights for staking and election pallets #1042
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
asset-hub-polkadot: regenerate weights for staking and election pallets #1042
Conversation
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async |
|
Sorry, only fellows can run commands. |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async" has started 🚀 See logs here |
|
Thanks @PolkadotDom 🚀 |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:No changes found. |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_staking_async" has started 🚀 See logs here |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:No changes found. |
|
Ok I am going to ask again in the fellowship runner room |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_staking_async |
|
Sorry, only fellows can run commands. |
It failed to compile the PAH runtime. |
Both missing solc and the issue we were returning OK even if /cmd failed have been fixed in #1043 so I am rebasing on top of that. |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async" has started 🚀 See logs here |
… --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async'
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:No changes found. |
|
@bkchr , @Ank4n from https://github.com/polkadot-fellows/runtimes/actions/runs/20483026864/job/58862929149#step:12:112 I took the command from #974 (comment) where it worked just fine... (also it hasn't errored out but returned success 😥) Extended logs: -- listing pallets for benchmark for asset-hub-polkadot
Pallets in {'name': 'asset-hub-polkadot', 'package': 'asset-hub-polkadot-runtime', 'path': 'system-parachains/asset-hubs/asset-hub-polkadot', 'uri': 'wss://polkadot-asset-hub-rpc.polkadot.io:443', 'is_relay': False, 'blocktime': 6000, 'extra_args': '--disable-mbm-checks', 'try_runtime_args': '--overwrite-state-version 1', 'build_extra_features': 'polkadot-ahm', 'benchmarks_templates': {'pallet_xcm_benchmarks::generic': 'templates/xcm-bench-template.hbs', 'pallet_xcm_benchmarks::fungible': 'templates/xcm-bench-template.hbs'}}: ['pallet_asset_rate', 'snowbridge_pallet_system_frontend', 'pallet_nfts', 'pallet_utility', 'pallet_child_bounties', 'pallet_xcm', 'pallet_parameters', 'pallet_treasury', 'pallet_timestamp', 'cumulus_pallet_parachain_system', 'pallet_referenda', 'pallet_xcm_benchmarks::generic', 'frame_system_extensions', 'pallet_uniques', 'pallet_multisig', 'pallet_collator_selection', 'pallet_bags_list', 'pallet_scheduler', 'pallet_whitelist', 'pallet_election_provider_multi_block', 'pallet_assets', 'pallet_transaction_payment', 'pallet_balances', 'pallet_election_provider_multi_block::verifier', 'pallet_xcm_benchmarks::fungible', 'pallet_conviction_voting', 'pallet_staking_async', 'pallet_asset_conversion', 'pallet_xcm_bridge_hub_router', 'pallet_proxy', 'pallet_bounties', 'pallet_ah_ops', 'pallet_indices', 'pallet_ah_migrator', 'pallet_message_queue', 'polkadot_runtime_common::claims', 'pallet_preimage', 'pallet_election_provider_multi_block::unsigned', 'pallet_session', 'pallet_election_provider_multi_block::signed', 'cumulus_pallet_xcmp_queue', 'pallet_asset_conversion_tx_payment', 'frame_system', 'pallet_vesting']
Pallet: ['pallet_election_provider_multi_block', 'pallet_election_provider_multi_block_signed', 'pallet_election_provider_multi_block_verifier', 'pallet_election_provider_multi_block_unsigned', 'pallet_staking_async']
Filtered out runtimes & pallets: {}
No pallets [['pallet_election_provider_multi_block', 'pallet_election_provider_multi_block_signed', 'pallet_election_provider_multi_block_verifier', 'pallet_election_provider_multi_block_unsigned', 'pallet_staking_async']] found in ['asset-hub-polkadot'] |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has started 🚀 See logs here |
|
thanks @bkontur , I was DM Basti as we speak to tell him that with |
…ot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async'
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:
|
|
Still running on VM 😔 |
|
so it runs now on tugy/Amorfc runner. @tugytur mentioned that all the ipb runners run on VMs but this one specifically is validator spec one, it's a VM with the docker setup within that, and the VM passes benchmarks. |
|
@bkontur: @seadanda mentioned that you could potentially run benchmark locally on a machine matching ref HW. Would it be possible to do that for you and just push the results so we can have proper staking weights for incoming release on PAH (cut end of the week AFAIK) while we are trying to have things in order on infra side? thanks in advance 🙏 |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has started 🚀 See logs here |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has finished ✅ See logs here |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has started 🚀 See logs here |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has finished ✅ See logs here |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has started 🚀 See logs here |
… --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async'
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:
|
|
The genesis config includes devStakers: Some((2_000, 25_000)) which pre-populates 27,000 stakers (2,000 validators + 25,000 nominators). Many pallet_staking_async benchmarks call clear_validators_and_nominators() in their setup phase, which:
Even though this happens in the "setup" phase (before timing), the cost appears to leak into the measured execution time and this is the bug! Evidence Affected Benchmarks All benchmarks calling clear_validators_and_nominators():
Benchmarks using create_validator_with_nominators helper are also affected (e.g., process_offence_queue). |
|
if we just want to unlock staking very quickly, the easiest seems just adding frame_benchmarking::benchmarking::commit_db();at the end of the infamous Tested and works just fine for e.g. force_apply_min_commission and process_offence_queue. I've created an issue in the SDK: paritytech/polkadot-sdk#10798 |
|
I have merged paritytech/polkadot-sdk#10802 on SDK which will solve the bulk operation leaking into benchmark result. This change is needed in order to be able to remeasure weights decently for staking. |
They were run on VM and not on reference HW in the scope of #974.