Percentage Based Sharding for Compactors

[eeldaly]

Is your feature request related to a problem? Please describe.
Compactors shard size has to be increased manually for a tenant. This does not work well when scale up of compactors occurs since we are limited by a fixed value.

Describe the solution you'd like
Identical to: #6652 for compactors. Adding support for 0.2 = 20% of compactors for that tenant

[dsabsay]

I understand this makes scaling easier.

For some users, they move tenants between multiple clusters. I could see percentage-based sharding making it slightly more difficult to properly size the clusters. For example, you move a small tenant in a large cluster to a smaller cluster; the tenant was spread on lots of compactors before and now they have less. If you want predictable performance, you'd need to move tenants to only similarly sized clusters. Maybe this is an acceptable challenge; I'm just thinking.

What strategies do you see for properly sizing clusters during tenant moves?

[eeldaly]

I understand this makes scaling easier.

For some users, they move tenants between multiple clusters. I could see percentage-based sharding making it slightly more difficult to properly size the clusters. For example, you move a small tenant in a large cluster to a smaller cluster; the tenant was spread on lots of compactors before and now they have less. If you want predictable performance, you'd need to move tenants to only similarly sized clusters. Maybe this is an acceptable challenge; I'm just thinking.

What strategies do you see for properly sizing clusters during tenant moves?

For setups like this, the user can keep their current setup without changing to percentage based. This would be much more useful for clusters with a few large tenants where you can keep any of the smaller tenants configs the same and add percentage based for the large tenants especially those with spiky ingestion patterns.