[enhancement] crawl MSA IDs to find user migration jobs to execute #159
Description
Description
Currently, the app crawls blocks in the chain looking for new delegations to a provider in order to queue relevant graph update jobs. There are several significant drawbacks to this approach:
- Crawling the chain is slow & time-consuming
- Determining an appropriate block from which to start the scan requires off-line back-of-napkin calculations
- Usually requires an archive node in order to crawl historical blocks
One approach that might mitigate these drawbacks is:
- Determine the current max MSA ID and "crawl" MSA IDs sequentially from 1
- For each MSA ID, determine if delegations are in place, and if so, process the user's graph
Note, this approach does not necessarily eliminate the need to crawl the chain & watch for new delegations, but it does eliminate the need for an archive node--the chain scan can start from the current block as of the start of the service.