Skip to content

Commit b9b7164

Browse files
committed
docs: clarify multiple workers
1 parent a1e680e commit b9b7164

File tree

2 files changed

+14
-2
lines changed

2 files changed

+14
-2
lines changed

README.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,13 @@ Sources of latency:
2222
- Processing time of blocks.
2323
- DB delay. OnFinality has some caching which batches DB operations.
2424

25+
Mitigations:
26+
27+
- Index with multiple workers.
28+
- They allow up to five in **Change Compute Size**
29+
- In a recent indexing, the rate went from 14 to 50 blocks/sec (from 1 to 5 workers)
30+
- Diminishing returns are to be expected because the blocks are always processed in order, so workers are often waiting their turn.
31+
2532
## Developing
2633

2734
This should be the same as the [Agoric SubQuery project](https://github.com/Agoric/agoric-subql). See [Developing](https://github.com/Agoric/agoric-subql/blob/main/README.md#developing) in its README.

src/mappings/events/fastUsdc.ts

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,15 @@ export const transactionEventKit = (block: CosmosBlock, data: StreamCell, module
2121
const { height } = block.header;
2222
const time = block.header.time as Date;
2323

24-
// XXX this assumes that records will be processed in order,
25-
// so multiple indexer workers is not supported.
2624
const t = await FastUsdcTransaction.get(id);
2725
if (!t) {
26+
// NB: This really should not happen. The earlier entry in vstorage for any transaction
27+
// is always an OBSERVED. I put this condition here because the indexer
28+
// did fail at least once without it. I think it was when I started
29+
// indexing with multiple workers and for that reason I thought the
30+
// workers would proces blocks out of order. But the devs at OnFinality
31+
// said: The workers allow fetching blocks out of order but they are still
32+
// processed in order. (BLock 51 will always be processed after 50).
2833
if (payload.status !== FastUsdcTransactionStatus.OBSERVED) {
2934
console.error('new status ${payload.status} for ${id} without a previous OBSERVED');
3035
return [];

0 commit comments

Comments
 (0)