Skip to content

Conversation

magicxyyz
Copy link
Contributor

@magicxyyz magicxyyz commented Jul 9, 2025

Part of NIT-3330

This PR:

  • add config for number of threads used in tx indexer to rlp decode blocks (previously hardcoded to runtime.NumCPU)
  • adds configurable delay of tx indexer runs (minBatchDelay):
    • it mitigates the issue of tx indexer reading tx indexer tail from disk after each new head, what seemingly degraded node sync performance (experiments used pathdb, but probably applies also when hashdb is used)
    • delaying runs allows batching work for each run, so each run may be started for accumulated over the time block range
  • adds new txindexer metrics:
    • txindexer/tail gauge - tracks oldest indexed block, useful e.g. for verifying if we keep up with unindexing
    • txindexer/unindexed/blocks/count and txindexer/unindexed/transactions/count - those expose stats about blocks / transactions unindexed in single txindexer run, useful to monitor how changing minBatchDelay affects actual work done in each batch

@cla-bot cla-bot bot added the s CLA signed label Jul 9, 2025
@magicxyyz magicxyyz changed the title Add threads and batch delay configs to transaction indexer Add configs and metrics to transaction indexer Jul 9, 2025
@magicxyyz magicxyyz marked this pull request as ready for review July 10, 2025 16:47
@magicxyyz magicxyyz requested review from MishkaRogachev, amsanghi and diegoximenes and removed request for amsanghi July 10, 2025 16:50
Copy link
Contributor

@MishkaRogachev MishkaRogachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall it looks good, just are a few questions

Copy link
Contributor

@MishkaRogachev MishkaRogachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@magicxyyz magicxyyz requested a review from tsahee July 15, 2025 21:14
@magicxyyz magicxyyz assigned tsahee and unassigned magicxyyz Jul 15, 2025
Copy link
Contributor

@tsahee tsahee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tsahee tsahee merged commit 589dae4 into master Jul 16, 2025
15 checks passed
@tsahee tsahee deleted the tx-indexer-config branch July 16, 2025 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants