eth: batch eth_getLogs over block ranges during initial sync to reduce request volume

[cranycrane]

Problem

During EVM initial sync, Blockbook issues three RPC calls per block:

• eth_getBlockByHash — block body (bchain/coins/eth/ethrpc.go getBlockRaw)
• eth_getLogs with fromBlock == toBlock — one call per block (processEventsForBlock, ethrpc.go:1407/:1415)
• debug_traceBlockByHash — internal data (getInternalDataForBlock, ethrpc.go:1520)

The per-block eth_getLogs call is a large and avoidable share of the total request volume during initial sync. eth_getLogs supports fetching logs across a block range in a single call, so the entire log axis of initial sync can be collapsed from one call per block to one call per window of blocks.

This issue proposes doing that for the initial-sync path only, plus a secondary transport optimization to reduce HTTP round-trips and concurrent-connection pressure against rate-limited endpoints.

Scope

• Initial / bulk sync only — the deep backlog fetched by SyncWorker.BulkConnectBlocks (db/sync.go:773). Steady-state / tip sync and API random-access GetBlock are not touched.
• Generic EthereumRPC.GetBlock is left unchanged; the optimization is opt-in and coin-scoped.

Why it's safe/tractable here

BulkConnectBlocks workers (getBlockWorker, db/sync.go:654, spawned at :825) pull from a shared hash queue and write to unbuffered per-worker channels drained by a single writer in strict height order (db/sync.go:800). Because those channels are unbuffered, no worker runs more than ~syncWorkers blocks ahead of the write frontier — so the set of blocks in flight is always a small contiguous window, which is exactly what a range-log cache needs. Block hashes are already known up front from GetBlockHash (db/sync.go:851), so body and trace can be fetched by hash without fetching the body first.

Plan

Commit 1 — Range-log prefetcher (fewer requests)

Collapse per-block eth_getLogs into one range call per window of W blocks.

• Add SyncLogsBatchBlocks int to the eth ChainConfig, default 0 = disabled (byte-for-byte current behavior when off).
• New prefetcher in the eth package:
  • Forward-filling window keyed by block number: map[blockNumber] -> (map[txHash][]*RpcLog, []AddressAliasRecord).
  • Whole-window coverage marker so a block absent from the response is treated as zero logs, not a cache miss.
  • Singleflight per window fill — the ~syncWorkers workers hitting the frontier trigger only one range call; the rest wait.
  • Adaptive chunking — on "range too wide" / "too many results" provider errors, halve W and retry; remember the reduced width.
  • Truncation guard (correctness-critical) — if a window returns at/near the provider's result cap, sub-split and re-fetch; never trust a capped window. A silently truncated window = missing logs = a permanent index gap (this is exactly the failure mode behind past ERC20-not-found gaps).
  • Frontier eviction + hard size cap to bound memory.
  • Reuse getEnsRecord per log so EthereumBlockSpecificData.AddressAliasRecords is unchanged.
• Tip-tail safety — range-fetch only blocks < bestHeight - keep; the tail near the tip falls back to the existing per-block eth_getLogs path (eth_getLogs is number-only and cannot be pinned to a hash).
• Metrics via the existing observeSyncRPCLatency + a counter of range calls vs. blocks covered.

Commit 2 — Batch body + trace (throughput / round-trips)

Note: some providers bill/meter each method in a batch separately, so this may not reduce request volume on those. It is worth doing for throughput on rate-limited endpoints: one HTTP round-trip instead of two per block, and half the concurrent in-flight requests per worker (logs already served from the cache). Single-block latency is ~neutral (t(body) << t(trace)).

• Bundle eth_getBlockByHash + debug_traceBlockByHash into one BatchCallContext, reusing the existing type-assertion idiom from getNoncesRPC (ethrpc.go:2322), with a fallback to the current path when the client doesn't support batching.
• Per-element error mapping preserves current semantics: body error → fatal; trace error → tolerated → InternalDataError.

Scoping mechanism (keeps generic GetBlock untouched)

• The BulkConnectBlocks worker type-asserts w.chain to an optional interface{ GetBlockBulk(hash string, height uint32) (*bchain.Block, error) } and calls it when present, else falls back to GetBlock.
• Bitcoin-type coins and all API/random-access GetBlock callers are unaffected. Both features are dead code unless the flag is on and the client supports batching.
• Prefetcher lifecycle: lazy creation on first GetBlockBulk, frontier eviction, hard size cap (no explicit enter/exit wiring).

Request-count impact (trace enabled, N backlog blocks, window W)

Variant	Requests	vs. today
Today	3N	—
+ Commit 1 (range logs)	~2N (3N - N + N/W)	~33% fewer
+ Commit 2 (batch body+trace)	provider-dependent (batch may be metered per-method) — otherwise throughput only	round-trips halved

The request-count reduction comes entirely from Commit 1 (eliminates the eth_getLogs axis). Commit 2 is primarily a throughput / connection-pressure improvement.

Correctness checklist

• Truncation detection (result-cap) — must sub-split; no silent gaps.
• Empty-block windows treated as zero logs via coverage marker.
• Adaptive chunking on range/result-limit provider errors.
• Tip-tail fallback to per-block eth_getLogs.
• Flag off / non-batch client ⇒ identical to current behavior.
• ENS AddressAliasRecords output unchanged.
• Verify getBlockWorker is exclusive to BulkConnectBlocks (not shared with ParallelConnectBlocks) before wiring the hook.

Open questions

• Default and safe range for W (start conservative, e.g. 1000) — depends on the provider's block-range and result-count limits.
• Whether to expose W and the tip-tail depth as per-coin config.