-
Notifications
You must be signed in to change notification settings - Fork 129
Description
Is your feature request related to a problem?
We have a multi-tenant environment where most tenants have their own index. Our ingestion lambda handles creating daily indices for tenants as needed. Different tenants can have different primary counts based on volume. We are using an ISM policy to manage indices after they are created for the day.
From my understanding and what I've tested, we are unable to use the min_primary_shard_size
option on the rollover action. This is because the action blocks the transitions from running, waiting on a primary to cross the threshold, which may never happen after we create the tenant's next daily index. Indices get stuck on the rollover action.
To get around this, we use a min_size
transition and move the index into a "rollover" state that does the rollover without any conditions. This allows indices to move through all phases. But the min_size
transition here is not ideal because different tenants have different numbers of primary shards. We would really like all of the primary shards to have the same size limit.
What solution would you like?
I would like a transition condition for min_primary_shard_size
so we can move an index to our rollover phase when a single primary, not a total, crosses the threshold.
What alternatives have you considered?
We could have our lambda call the rollover api every now and then, offload the task from ISM. We could also split our ISM policy for the different tenants but this is not ideal.
Do you have any additional context?