Skip to content

Commit e64d58c

Browse files
committed
hashmod issues
1 parent 78a9936 commit e64d58c

File tree

1 file changed

+22
-0
lines changed

1 file changed

+22
-0
lines changed

prometheus/prometheus-manage-performance.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,3 +80,25 @@ remote_read:
8080
- url: http://scraper3:9090/api/v1/read
8181
read_recent: true
8282
```
83+
84+
The disadvantages:
85+
86+
1. Operational
87+
88+
Hashmod sharding is static. Adding or removing Prometheus instances changes the modulus value and forces a full redistribution of scrape targets. This causes:
89+
90+
- Large-scale target churn
91+
- Increased scrape failures during transitions
92+
- Potential dual-scraping or gaps while the fleet stabilizes
93+
94+
2. Uneven Load Distribution With Heterogeneous Targets
95+
96+
Hashmod sharding provides uniform distribution by count, not by cost or cardinality. If some jobs are heavier (high cardinality, expensive exporters, long scrape durations), shards become imbalanced even if target counts are equal.
97+
98+
3. Difficulties With Global Querying and Federation
99+
100+
Because each shard stores only a portion of the metrics:
101+
102+
- Federation layers or external query engines must aggregate across shards.
103+
- Debugging becomes harder because a target’s data lives on only one shard.
104+
- Global alerting requires a separate aggregator or Thanos Ruler; local rules cannot be global.

0 commit comments

Comments
 (0)