feat: Enforce syncronous procesing of blocks #118
Description
Description
Solve a potential problem on how blocks are processed.
2 potential issues:
- Blocks are executed in parallel (we could be processing multiple blocks at the same time), but our DB is modeled assuming synconizity (persist last block number --> possible to persist N+1 without N being processed)
- Within a block, we process orders synchronously (potentially, we can do them in parallel). I would avoid changing this into parallel processing if is not needed, but just saying cause might become a problem if we change into synchronous processing the blocks (if we can't process them fast enough).
Context
Im revisiting the persistance and block processing of watch tower.
I'm afraid we might have an issue.
We subscribe asynchronously to be notified about block production. And then we process the block, and persist in our registry our lastProcessedBlock
The issue i see is:
- Because we are being notified asynchronously for each new block, we are processing blocks in parallel
- This makes it technically possible that, the processing of block N+1 finishes before block N
- I believe this will be detected and labeled as REORG by this logic
- Further more, i think if there's a crash (like watch dog detecting a lagging node), I think it could skip one block
If we change into processing blocks synchronously, we should also be careful. I see processing a block can take a while (a lot of round trips), so changing to synchronous is also bringing their own problems
I think part of why is slow is because inside a block, the code is synchronous, so we handle one order at a time.
This might be fine, and is nicer for the RPC/api to give them a break, but in principle, we can do all orders in parallel (logs will be a bit more messy, so if its not needed i would avoid it)
https://cowservices.slack.com/archives/C05RMJB7ZFA/p1700593069144209