fix: Eliminate top DB queries — hourly_statistics MV, balance N+1, VACUUM in transaction#672
fix: Eliminate top DB queries — hourly_statistics MV, balance N+1, VACUUM in transaction#672
Conversation
…regate table, batch balance lookups, fix VACUUM in transaction - Replace hourly_statistics MV with incremental aggregate table (migration 046): - MV refresh was top query at 201,290ms latency, running every 30min - Per-block upsert recomputes only the affected hour (~25ms vs 2,132ms locally) - 85x improvement; cost scales with 1 hour of data not 24h history - Add composite index (account_hash, asset_id, _id DESC) on balance table (migration 047): - Enables single index seek for IndexBalance lookups instead of multi-index scan - Batch balance lookups in IndexBalance: 1 query per block instead of N+1 per (account, asset) pair - Fix queryWithTimeout: VACUUM and REFRESH MATERIALIZED VIEW CONCURRENTLY cannot run inside a transaction block — detect these and use session-level SET instead of SET LOCAL
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryMedium Risk Overview Eliminates balance lookup N+1s during indexing. Reduces unnecessary DB load and fixes timeout execution for non-transactional commands. Migration Written by Cursor Bugbot for commit 23155aa. This will update automatically on new commits. Configure here. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Add migration 048 to remove the unused indexer.recent_account_transactions_mv. The migration deletes any scheduled database_jobs that reference the view and drops the materialized view with CASCADE. This MV was created in migration 030, is not queried anywhere in the codebase, and its 4-hour refresh appeared as a top load query despite having no consumers.
Use COALESCE for total_fee and total_gas_used when backfilling indexer.hourly_statistics_agg from indexer.hourly_statistics to avoid inserting NULLs. This ensures missing values are treated as 0 during the migration (packages/graphql/database/migrations/046_replace_hourly_statistics_mv_with_agg_table.sql) and preserves the existing ON CONFLICT update behavior.
Replace the multi-placeholder IN tuple query with an unnest($1, $2) approach and a LATERAL subquery to select the most recent balance per (account_hash, asset_id). Build separate accountHashes and assetIds arrays as query params instead of expanding pair placeholders. This simplifies parameter handling and reliably returns the latest balance (ORDER BY _id DESC LIMIT 1) for each requested pair.
Update migration 047 to clarify the need for a composite index on (account_hash, asset_id, _id DESC) including rationale and benchmark results, and keep the CREATE INDEX CONCURRENTLY statement. Add migration 049 to recompute hourly_statistics_agg from the blocks table, overwriting total_fee and total_gas_used to fix issues from the previous MV-based backfill (incorrect gas totals and missed blocks).
1 participant
Summary
hourly_statisticsMV with incremental aggregate table (migration 046): The MV refresh was the feat: add pre loaded data on fuel-core #1 DB query at 201,290ms latency, scanning all 24h of transactions every 30min. New approach upserts only the affected hour per block during indexing (~25ms vs 2,132ms — 85x improvement locally). Idempotent: recomputes the full hour fromindexer.transactionsso re-processing blocks never double-counts.IndexBalance: Was doing 1 SELECT per (account, asset) pair at 43.27 calls/sec (top load query). Now does 1 batchedDISTINCT ONquery per block usingWHERE (account_hash, asset_id) IN (...).balance(account_hash, asset_id, _id DESC)(migration 047): Single index seek instead of multi-index scan + sort.queryWithTimeoutfor VACUUM and REFRESH MATERIALIZED VIEW CONCURRENTLY: Both fail inside aBEGIN/COMMITblock. All VACUUM jobs were silently retrying forever with "VACUUM cannot run inside a transaction block". Fix: detect these queries and apply timeouts at session level withSET/RESETinstead ofSET LOCAL.Test plan
hourly_statistics_aggtable created, old MV and jobs deletedhourly_statistics_aggis populated per blockBlockDAO.getTotalFeereturns data from new tableManual testnet cleanup needed
Run this on testnet DB since migration 046 already executed there with the old exact-match DELETE: