Skip to content

Commit 230c778

Browse files
committed
infoschema, executor, session: optimize partition table memory usage in infoschema v2
Issue Number: close #00000 Problem Summary: Partition tables are currently classified as "special attributes" and their full PartitionInfo is stored in tableInfoResident, bypassing the Sieve cache. This violates the V2 design principle of lazy loading and causes significant memory bloat for large-scale partitioned table clusters. What changed and how does it work? 1. Removed PartitionAttribute from HasSpecialAttributes() filter - Partition tables no longer go into tableInfoResident (permanent memory) - PartitionInfo now loaded on-demand from Sieve cache (evictable) 2. Added GetPartitionedTableIDsV2() API - Extracts partitioned table IDs from pid2tid btree (version-aware) - Returns nil for V1 (fallback to ListTablesWithSpecialAttribute) - Zero additional memory overhead 3. Updated all callers to use V2 API with V1 fallback: - fetchShowStatsMeta / fetchShowStatsHealthy: Use GetPartitionedTableIDsV2 + on-demand TableInfoByID - rebuildAllPartitionValueMapAndSorted: Scan only LIST partitions using V2 API Expected Benefits: - Memory: ~100MB saved for 1000 partitioned tables × 100 partitions - Startup: ~95% reduction in LIST partition scan scope - Query Performance: No regression (partition pruning remains O(1)) See design doc: explorer/partition-table-optimization-design.md
1 parent 5adafaf commit 230c778

5 files changed

Lines changed: 779 additions & 24 deletions

File tree

0 commit comments

Comments
 (0)