What you would like to be added?
Add a flag to support workloads to use Queue priority to evict In quota preemptible workloads from lower priority queues
When higher priority Queue with in-quota workload needs resources that it can't find, it should be allowed to reclaim those resources from in-quota lower priority preemptible workloads (in-quota preemption). For over-quota workloads this is already implemented. This should be applicable for both reclaim and consolidation.
Points for consideration:
Allow only workloads that are within the queue deserved quota to reclaim workloads from queues that are in (strictly) lower queue priority even within quota
Need to adress and think about edge cases and how fairshare should be calculated to acheive this result
This behavior should not be enabled by default, consider adding a per shard configuration for that behavior
Need to consider if in all cases we want to use quota as a disruption budget or is there a mode where priority is strictly more important
Why is this needed?
We want higher priority in-quota queues to be able to use resources on the expense of lower priority queues, even if the lower priority queues are in-quota, as long as lower priority queue workloads are preemptible.
What you would like to be added?
Add a flag to support workloads to use Queue priority to evict In quota preemptible workloads from lower priority queues
When higher priority Queue with in-quota workload needs resources that it can't find, it should be allowed to reclaim those resources from in-quota lower priority preemptible workloads (in-quota preemption). For over-quota workloads this is already implemented. This should be applicable for both reclaim and consolidation.
Points for consideration:
Allow only workloads that are within the queue deserved quota to reclaim workloads from queues that are in (strictly) lower queue priority even within quota
Need to adress and think about edge cases and how fairshare should be calculated to acheive this result
This behavior should not be enabled by default, consider adding a per shard configuration for that behavior
Need to consider if in all cases we want to use quota as a disruption budget or is there a mode where priority is strictly more important
Why is this needed?
We want higher priority in-quota queues to be able to use resources on the expense of lower priority queues, even if the lower priority queues are in-quota, as long as lower priority queue workloads are preemptible.