Milestone 1 and 2 for Offline Election Prediction Tool

[Ipsa11]

Milestone Delivery Checklist

• The milestone-delivery-template.md has been copied and updated.
• This pull request is being made by the same account as the accepted application.
• I have disclosed any and all sources of reused code in the submitted repositories and have done my due diligence to meet its license requirements.
• In case of acceptance, invoices must be submitted and payments will be transferred to the Polkadot AssetHub and/or fiat account provided in the application.
• The delivery is according to the Guidelines for Milestone Deliverables.

Link to the application pull request: w3f/Grants-Program#2623

[michalisFr]

On a different note, there's a mistype in milestones 2 and 3 in the proposal:

• In Milestone 2, deliverable 1 is the same as deliverable 1 on Milestone 1, when it should be phrased like deliverable 1 in Milestone 3
• In milestone 3, deliverable 1 should be about the API functionality

[Ipsa11]

Hi @michalisFr
Thank you for pointing that out! Could you please clarify what you mean? I just want to make sure I fully understand — are you referring to the phrasing overlap between Milestone 2 and 3 deliverables, and suggesting that Milestone 2’s first deliverable should match the structure of Milestone 3’s, while Milestone 3’s first deliverable should instead focus on API functionality?

[michalisFr]

Thanks for following up @Ipsa11. Basically yes. In more details, what I mean is the following:

In Milestone 2, Deliverable 1 says:

Hypothetical Account Simulation	Updated version of the existing election script with support for accurate simulation of on-chain validator election logic using Phragmén and other supported algorithms.

The title is correct, but the description is the same as Deliverable 1 of Milestone 1: "Core Election Engine"

At the same time, in Milestone 3 you say:

Core Election Engine	Extend simulation engine to support voters or candidates that do not exist on-chain or lack bonded amounts.

Here the title ("Core Election Engine") is incorrect and the description corresponds to the above deliverable of Milestone 2. At the same time, based on the RFP, the 3rd milestone is about the API functionality which isn't mentioned.

So basically the deliverable in Milestone 2 should read:

Hypothetical Account Simulation	Extend simulation engine to support voters or candidates that do not exist on-chain or lack bonded amounts.

and for Milestone 3 something like:

API functionality	Implement API functionality to set parameters and expose results

[Ipsa11]

Thank you @michalisFr for the clarification — you’re absolutely right. Could you please guide me on the next steps to update the PR, since it has already been merged? I want to make sure the corrections are made properly.

[michalisFr]

@semuelle Can you please help? ☝️

[semuelle]

Hey @Ipsa11. You can simply pull the latest master branch, change the affected lines in your original application document and then create a new PR. We will review it asap. Should be quick for such a small change.

[Ipsa11]

Hi @semuelle @michalisFr,
I’ve created a new PR with the requested changes. Could you please take a look and share your review.The link for the same is:
w3f/Grants-Program#2683
Thank you!

[semuelle]

It looks like you did not use a recent fork. It's 40 commits behind the current master branch. I recommend you check out the repo anew and apply the changes there.

[Ipsa11]

Hii @semuelle I have made the changes. Please have a look once.

[Ipsa11]

Hey @semuelle just checking in have you had a chance to go through Milestone 1 for the Offline Election Prediction Tool? Please share any updates or feedback when you get a moment. Thanks!

[semuelle]

Hey @Ipsa11, sorry for the delay. Let me check with @michalisFr.

[Ipsa11]

Thanks @semuelle Waiting for your reply once you’ve had a chance to check with @michalisFr.

[michalisFr]

Hey @Ipsa11. I have left some review comments. Can you please address them?

In the meantime, I'll try to run the tool and see if it works as expected. But I'd also like for a core dev to review the code before approving. I've pinged @sigurpol but he is OOO until Mon and he might be busy next week with the AHM.

[Ipsa11]

Hey @michalisFr thanks for the update. Could you please share the link where you’ve added the review comments so I can check and address them?

[michalisFr]

Hey @michalisFr thanks for the update. Could you please share the link where you’ve added the review comments so I can check and address them?

I've added them inline to the file. They are also visible above in the conversation. Can you see them?

[Ipsa11]

Hey @michalisFr Thanks for letting me know. I’m not able to see the comments on my end — could you please double-check if they were submitted or visible in the file? Attaching the ss for the same.

[michalisFr]

• Missing instructions on the format of the input JSON and examples of its usage
• Inconsistent examples, e.g. --use-cached-data without --cache-dir or --uri instead of --chain-uri
• Instructions for Docker
• desired-validators refers to the size of the active set? It's not clear
• Is there a reason all examples are for 19 validators?

[Ipsa11]

Hey @michalisFr,

• The missing instructions for the input JSON format have been added.
• All inconsistent examples have been corrected.
• Docker usage instructions are now included as well.
• Yes, the desired-validators parameter refers to the size of the active set.
• There’s no fixed limit of 19 validators — you can specify any number. If not provided, the tool will automatically fetch the active validator count from the chain.

[michalisFr]

This links to the README instead of the Dockerfile

[Ipsa11]

Hey @michalisFr
The link to Docker file has been updated.The link is as follows
https://github.com/antiers-solutions/polkadot-staking-miner/blob/feat/offline-election-prediction-tool/Dockerfile.README.md

[michalisFr]

@sigurpol Could you please provide a review?

[sigurpol]

Sure, I will review it, but after Polkadot AHM so realistically not before the end of this week if it's OK for you.
@Ipsa11 I am assuming we want to merge https://github.com/paritytech/polkadot-staking-miner/antiers-solutions:polkadot-staking-miner:feat/offline-election-prediction-tool branch into main, is it correct ? If it's the case, can we have a PR against main branch so it's easier to review for me as well?

[Ipsa11]

Hey @sigurpol
Yes, we want it to be merged into the main branch. The PR for the same is [https://github.com/paritytech/polkadot-staking-miner/pull/1189]

[michalisFr]

Hey @Ipsa11. My sincere apologies! I hadn't submitted the comments 🤦‍♂️ You should be able to see them now

[sigurpol]

I took an initial look at https://github.com/antiers-solutions/polkadot-staking-miner and have some general preliminary questions and comments after briefly reviewing the changes.

1. do we see this tool as a standalone tool or integrated in the miner? The current approach seems the latter where we are introducing a predict command as extra command together with the existing ones (monitor, info). But then, there is quite a lot of code duplication where we could have reused what the miner already provides :

• Connection handling (creates own OnlineClient vs using main miner's Client)
• Fetching Staking::ActiveEra, ValidatorCount, Validators, Nominators, Ledger
• Fetching MultiBlockElection snapshot data
• Batch processing with concurrency
• SCALE decoding
• AccountId32 extraction from storage keys
• JSON caching logic

All of this could theoretically have reused utilities from the main miner. The current design seems instead of having the prediction tools completely independent from the main miner and let the miner import offline-election-prediction-staking as sort of plugin / standalone library and offline-election-prediction-staking redefines whatever it needs.
If we aim to use it as extended miner and eventually merge this branch into main, then I would suggest to avoid code duplication. If we aim to distribute the prediction tool independently from the miner, then I would argue we shouldn't fork from the miner but just reuse what is in offline-election-prediction-staking standalone. It's not clear to me the direction we want to take.
My initial thought was to extend the miner with a predict command (like you did) and reuse the existing code as much as possible

2. Now that also for Polkadot, AHM is completed we should update documentation, and only have AssetHub URIs. And maybe add some chain validation at startup in case we try to connect to RC

I have added a comment here to explain how I would approach the problem. It's just my 2c and the way I envisioned it, but I am more than open to discuss radically different approaches

[michalisFr]

Thank you for the comment @sigurpol! I'll revert on what approach might be best, but I want to echo your approach on how the prediction command should work that you posted in here, regardless if it's an extension of the miner or a standalone tool.

I also echo your second point above.

@Ipsa11 I'd like to provide some feedback on the functionality of the tool itself. I ran it earlier today on AH outside the election window and with no parameters and I'm attaching the results.

Here are some "cosmetic" changes that would be useful:

1. Addresses should be encoded based on the chain ss58 prefix and not using the general format, i.e. Polkadot addresses should begin with 1 and Kusama with a capital letter.
2. Values should be denominated in the chain's token and not in Plancks
3. The output should

include the specific nominators with the stake allocated to each validator (as opposed to just their number).

as previously mentioned and discussed here.

Besides these, I have a few questions:

1. You mention a list of "backup/runners-up" validators. I understand these backup validators to be the "next in line" in order of how "close" they were to make it in the active set. Is that correct? However, these were not included in the output.
2. I don't understand the usefulness of the cache as an option. The cache is built during run. The idea is you can then later reuse the same input? So, if you do back-to-back runs you save time from querying the chain again?

However, the most important thing I'd like to note is that the results don't make sense I'm afraid. The tool predicts that validators will make to the active set with ~600k DOT, which is clearly wrong, while others will be in the active set with 3+ MDOT, which is also extremely unlikely.

Finally, what would be useful is if there where two outputs: a validators_prediction.json like the one produced (with the above addition) and a nominators_prediction.json that displays the results from the nominators' perspective that should include:

1. Nominator's address and stake
2. Active validator(s) with the allocated stake to each, i.e. the nominated validators this nominator will support
3. Inactive validators, i.e. the nominated validators that are active but not supported by this nominator
4. Waiting validators, i.e. the nominated validators that won't be in the active set

I acknowledge that this is the first I'm mentioning this request 😅

election_prediction.json

[Ipsa11]

Thank You @michalisFr and @sigurpol for the review and suggestions! We’re implementing the recommended changes and will update the codebase shortly.

[Ipsa11]

Hey @michalisFr @sigurpol,
All the suggested changes have been implemented. Kindly take a look and share your feedback when convenient.

[filippoweb3]

@Ipsa11 ok now it works for Polkadot, I am able to correctly predict when the snapshot is fetched and when it is built just at the end of the off phase

I am not able to predict for Kusama. When the snapshot is fetched, predictions are off
cargo run -- --uri wss://rpc-asset-hub-kusama.luckyfriday.io predict --do-reduce --block-number 12781004 --balancing-iterations 10

outside the snapshot window the prediction file is empty
cargo run -- --uri wss://rpc-asset-hub-kusama.luckyfriday.io predict --do-reduce --block-number 12780985 --balancing-iterations 10

[filippoweb3]

@Ipsa11 I run some additional tests, predictions for Polkadot are OK

For Kusama, when a snapshot is available, predictions are slightly off, see for block 12792990

When the snapshot is not availble the built one is totally different from the fetched one (tested for block 12792971). The validaotors_prediction.json is empty

{
  "metadata": {
    "timestamp": "1769007939",
    "desired_validators": 1000,
    "round": 424,
    "block_number": 12792971,
    "solution_score": {
      "minimal_stake": 340282366920938463463374607431768211455,
      "sum_stake": 0,
      "sum_stake_squared": 0
    },
    "data_source": "staking"
  },
  "results": []
}

[Ipsa11]

Hey @filippoweb3
Thanks for the feedback. We are looking into the issue.

[filippoweb3]

@Ipsa11 I tested the tool and I am able to predict correctly for both Kusama and Polkadot within and outside the snapshot window.

Regarding the custom data, do I need to add all nominators and candidates there, or is there a way to remove specific accounts, add new ones, or edit nominations for specific accounts? (as requested in the Milestones)

I tried to use the example you have in the README, and I get the following error

The aim is that the custom file provides the data to:

• Add candidates that may not exist on-chain
• Remove specific candidates from the election
• Add or override voters with custom stake amounts (regardless of on-chain bonded amounts)
• Remove specific voters from the election

The current system "sees" only the custom file, but ideally we should:

• fetch the snapshot or build it if not available
• override the snapshot with the custom data

Afaik, you did not provide an API for the prediction feature, correct?

[Ipsa11]

Hey @filippoweb3
Thank you for testing the tool and confirming correct predictions for Kusama and Polkadot — we really appreciate the detailed validation.

To clarify scope and status:

• Milestone 1 delivered the core election engine and custom data support.
• Milestone 2 delivered hypothetical data support.

We have implemented all the requested custom data functionality . The overrides option allows adding candidates that may not exist on-chain, removing specific candidates from the election, adding or overriding voters with custom stake amounts regardless of on-chain bonded amounts, and removing specific voters from the election. Now, the system first fetches or builds the snapshot and then overrides it with provided custom data as expected.

Could you please confirm whether the current custom data support meets the Milestone 2 expectations?
If so, we kindly request you to merge the Milestone 1 & 2 code and initiate the payment process as this is the milestone based delivery.

Regarding the API, you are correct — it is part of Milestone 3, and we will push that code in the meanwhile.

Looking forward to your feedback.

[michalisFr]

Hi @Ipsa11. First of all thank you for the work you've done so far on this tool!

At this point and after our testing, I consider the core election engine part of the first milestone delivered. However, the custom input either with existing accounts (milestone 1) or with non-existent or not bonded accounts (milestone 2) is not delivered I'm afraid.

As you mention in your comment the custom file should allow adding and removing candidates and adding and removing voters. However, the way it works now apparently, the custom file replaces the whole snapshot data, which is not the requested functionality and which also doesn't work apparently.

The idea of this functionality is that the data in the custom file changes the snapshot accordingly and then the election prediction is run on this modified snapshot. Also, there is no point in assigning stake to candidates; this will be determined by the election.

In the original script, the JSON file had this structure:

{
  "candidates": ["Alice", "Bob"],
  "candidates_remove": ["Charlie"],
  "voters": [
    ["David", 1000000, ["Eve", "Frank"]]
  ],
  "voters_remove": []
}

I like you nominators structure better and it can be used to remove nominators if you set the targets to an empty array perhaps. But it needs a way to remove candidates also, candidates shouldn't have a predetermined "stake", and it shouldn't replace the existing snapshot.

Please make the custom file to work in this way and once we've tested it we'll be happy to award these two milestones.

[Ipsa11]

Hey @michalisFr
Thank you for your feedback
We have recently implemented all of these changes. The points you are referring to were applicable earlier, but the code has since been updated to support them. There may be some confusion due to this, so we kindly request you to please test the latest version again, as the changes you are suggesting have already been addressed on our side.

[filippoweb3]

@Ipsa11 I tried to test with the latest pushes and

• predictions work correctly for both Polkadot and Kusama, within and outside the snapshot window
• overrides work as intended

Left to do: API

[michalisFr]

Thanks @Ipsa11 for the update and @filippoweb3 for testing.

So, I believe at this point we can sign off on the first two milestones @semuelle!

[Ipsa11]

Hey @michalisFr @filippoweb3
Thank you for the feedback. As reviewed and confirmed by @filippoweb3, both the Core Election Engine and the Custom Data support are working as intended. Could we please proceed with the next steps, including merging the code and initiating the payment process?

[michalisFr]

Hey @michalisFr @filippoweb3 Thank you for the feedback. As reviewed and confirmed by @filippoweb3, both the Core Election Engine and the Custom Data support are working as intended. Could we please proceed with the next steps, including merging the code and initiating the payment process?

For the latter I already tagged @semuelle (we posted at the same time). For merging into staking miner, let me tag @sigurpol

[Ipsa11]

Hey @sigurpol @michalisFr
Thank you for the update. For the merging into the staking miner, thank you for tagging @sigurpol. Please let us know if any additional input is needed from our side to move this forward.

[sigurpol]

Hey @sigurpol @michalisFr Thank you for the update. For the merging into the staking miner, thank you for tagging @sigurpol. Please let us know if any additional input is needed from our side to move this forward.

I'll review last version today and provide a feedback - thanks all for the great work so far

[sigurpol]

@Ipsa11 I see you are still polishing the PR - let me know when you are done and it's good time for final review. Thank you

[Ipsa11]

@Ipsa11 I see you are still polishing the PR - let me know when you are done and it's good time for final review. Thank you

@sigurpol
We have completed the changes from our end. The PR is ready, and you may proceed with the final review at your convenience.

[sigurpol]

@Ipsa11 , I've added my review - couple of relevant issues I would like you to investigate / address + few minor issues not so important (but low hanging fruit so why not)

[Ipsa11]

@Ipsa11 , I've added my review - couple of relevant issues I would like you to investigate / address + few minor issues not so important (but low hanging fruit so why not)

Hey @sigurpol
We’ve addressed all the issues and suggestions you raised, including the minor ones. Could you please review the latest changes when you get a chance?

[filippoweb3]

@Ipsa11 let me know when I can test the API so that we got all Milestones covered (if you are still aiming for it)

[sigurpol]

@Ipsa11 , I've added my review - couple of relevant issues I would like you to investigate / address + few minor issues not so important (but low hanging fruit so why not)

Hey @sigurpol We’ve addressed all the issues and suggestions you raised, including the minor ones. Could you please review the latest changes when you get a chance?

Re-reviewed, thx for taking care of my comments. I have one very tiny cosmetic comment left but I can merge as it is once @filippoweb3 confirms results are OK

[Ipsa11]

Hey @filippoweb3 @sigurpol
Thank you for checking in. Yes, we are still aiming to cover all milestones.
Since we’ve been following a milestone-wise delivery approach, we planned to proceed with Milestone 3 (API) once Milestones 1 and 2 are completed. That said, could you please share any specific requirements or expectations you have for the API? This will help us align quickly and share the API for testing as soon as possible.
Looking forward to your inputs.

[filippoweb3]

@Ipsa11 it all checks, the changes did not affect how the prediction works.

Regarding the API:

The goal is to make election simulations and snapshot retrieval programmatically accessible for downstream tooling, dashboards, and research workflows, without requiring direct CLI or runtime integration.

Proposed API overview

The API would run as a standalone REST server backed by a configurable RPC endpoint.

Server startup

Default:

cargo run -- --rpc-endpoint wss://sys.ibp.network/asset-hub-polkadot server

Custom address:

cargo run -- --rpc-endpoint wss://sys.ibp.network/asset-hub-polkadot server --address 0.0.0.0:8080

REST API endpoints

POST /simulate

Simulate an election using a snapshot and configurable parameters.

Query parameters

• block (optional): block hash to use for the snapshot (defaults to latest)

Request body

{
  "desired_validators": 600,
  "algorithm": "seq-phragmen",
  "iterations": 10,
  "reduce": true,
  "max_nominations": 16,
  "min_nominator_bond": 1000000000,
  "min_validator_bond": 1000000000,
  "manual_override": {
    "candidates": [],
    "candidates_remove": [],
    "voters": [],
    "voters_remove": []
  }
}

Where

• algorithm: "seq-phragmen" or "phragmms" (default: seq-phragmen)

• iterations: number of balancing iterations

• reduce: apply reduce algorithm (default: false)

• Bond and nomination limits fall back to chain defaults if omitted

• manual_override matches the existing CLI override format

Success response (200 OK):

{
  "result": {
    "run_parameters": { ... },
    "active_validators": [ ... ]
    "nominators": [ ... ]
  }
}

Basically "active validators" and "nominators" are the equivalent of the nominators_prediction.json and validators_prediction.json via CLI

GET /snapshot

Retrieve the election snapshot used for simulations.

Query parameters

• block (optional): block hash for snapshot (defaults to latest)

Success response (200 OK):

{
  "result": {
    "validators": [ ... ],
    "nominators": [ ... ],
    "config": { ... }
  }
}

[sigurpol]

@Ipsa11 I have merged the changes for Milestone 1 and 2 into the miner repo main branch. Thanks for all the good work.

[Ipsa11]

@sigurpol @michalisFr @semuelle
Thank you very much for merging Milestone 1 and Milestone 2 into the miner repository’s main branch. We really appreciate the support and collaboration.

Could you please initiate the payment process for these two milestones when convenient? Also let me know if any input is required from my side.

[semuelle]

Hey @Ipsa11. You can find the evaluation notes here. The milestones are hereby accepted.

I don't see any invoices submitted yet. Please submit them using the instructions in the comment after mine.

[github-actions]

🪙 Please fill out the invoice form in order to initiate the payment process. Please make sure that you follow the instructions and requirements as laid out in the form as well as our Terms & Conditions. Thank you!

[Ipsa11]

Hey @semuelle,
The payment address mentioned in our application is the following:
Team Name: Starks
Payment (USDC): 0x4c2f3896E60B79ab3821cad311bD4c30Ae0a4693
Grant Level: 1

Could you please confirm if the payment can be processed to this address?

[Ipsa11]

Hey @Ipsa11. You can find the evaluation notes here. The milestones are hereby accepted.

Hey @semuelle
The evaluation notes contain only milestone 1. The PR merged contains both milestone 1 and milestone 2. Please have a look into this.

[semuelle]

The evaluation notes contain only milestone 1. The PR merged contains both milestone 1 and milestone 2. Please have a look into this.

The PR only contains one file. I see it titled as "Milestone 1 and 2", but the file in it clearly states Milestone Number: 1. Please check if the report for M2 is in another branch or fork.