perf(backend): index roms (platform_id, fs_size_bytes)#3780
Merged
Conversation
GET /api/platforms and GET /api/stats both sum roms.fs_size_bytes but the column was not in any index, forcing full reads of the wide, JSON-heavy roms table (5-11s on large libraries). A composite index on (platform_id, fs_size_bytes) turns both sums into index-only scans. Fixes #3769 Generated-By: PostHog Code Task-Id: e24ff248-6329-4b70-a826-9a9fee21411e
Contributor
Greptile SummaryThis PR adds a covering index for ROM size aggregation. The main changes are:
Confidence Score: 5/5This looks safe to merge.
Important Files Changed
Reviews (1): Last reviewed commit: "perf(backend): index roms (platform_id, ..." | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
GET /api/platforms(11.5s) andGET /api/stats(~5s) slow down badly as the library grows because both sumroms.fs_size_bytes(per-platform via thePlatform.fs_size_bytescolumn property, and whole-library for stats).fs_size_byteswas not part of any index, so the database had to read actual rows from the very wide, JSON-heavyromstable (964 MB / 92,800 roms in the reporter's case).rom_counton the same model is instant becauseCOUNTis satisfied by the existingplatform_idindex.This PR adds a composite index on
roms (platform_id, fs_size_bytes), turning both sums into index-only scans that never touch the wide rows. The optimizer picks the covering index for both query shapes on its own, so no query changes are needed.Changes:
backend/models/rom.py: addIndex("idx_roms_platform_fs_size", "platform_id", "fs_size_bytes")toRom.__table_args__.backend/alembic/versions/0097_roms_platform_fs_size_index.py: migration creating the index, usingbatch_alter_table+if_not_exists/if_existsfor MariaDB/MySQL/PostgreSQL compatibility (mirrors the existing0093index migration).Reported measurements (from the issue and a member's replica of a 92,800-rom production DB):
SUM(fs_size_bytes)+COUNT(/api/platformsshape)SUM(fs_size_bytes)(/api/statsshape)Index build took ~4.7s on the 964 MB table, so the migration is cheap even for large libraries.
Fixes #3769
AI-assistance disclosure: This change was written with AI assistance (PostHog Code / Claude). The diagnosis, index design, and migration were AI-generated; the timings in the issue and above were verified by a maintainer with read-only queries against a production-scale database.
Checklist
Screenshots (if applicable)
N/A (index-only change; see timing table above)
Created with PostHog Code