feat: Implement an example CloudFormation for a custom Deadline Cloud Balancer #81
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Ahuge
force-pushed
the
ah/feature/custom-balancer
branch
from
cf2992f to
0962c7b
Compare
Ahuge
changed the title
Ahuge
force-pushed
the
ah/feature/custom-balancer
branch
2 times, most recently
from
1c282f6 to
28d9382
Compare
mwiebe
requested changes
Mar 5, 2025
cloudformation/auxiliary_templates/custom_balancer/custom_queue_job_balancer.yaml
Outdated
Show resolved
Hide resolved
cloudformation/auxiliary_templates/custom_balancer/custom_queue_job_balancer.yaml
Outdated
Show resolved
Hide resolved
cloudformation/auxiliary_templates/custom_balancer/custom_queue_job_balancer.yaml
Outdated
Show resolved
Hide resolved
cloudformation/auxiliary_templates/custom_balancer/custom_queue_job_balancer.yaml
Outdated
Show resolved
Hide resolved
cloudformation/auxiliary_templates/custom_balancer/custom_queue_job_balancer.yaml
Outdated
Show resolved
Hide resolved
cloudformation/auxiliary_templates/custom_balancer/custom_queue_job_balancer.yaml
Outdated
Show resolved
Hide resolved
cloudformation/auxiliary_templates/custom_balancer/custom_queue_job_balancer.yaml
Outdated
Show resolved
Hide resolved
Ahuge
force-pushed
the
ah/feature/custom-balancer
branch
from
28d9382 to
44955e9
Compare
|
I've noodled on a bit of refactoring of this, confirming the identitystore: IAM permission isn't necessary, trying to clean up naming, etc. Have pushed that here: https://github.com/mwiebe/deadline-cloud-samples/commits/ah/feature/custom-balancer/ |
ryyakobe
reviewed
Mar 21, 2025
cloudformation/auxiliary_templates/custom_balancer/custom_queue_job_balancer.yaml
Outdated
Show resolved
Hide resolved
Ahuge
force-pushed
the
ah/feature/custom-balancer
branch
from
dbb551e to
1878d78
Compare
This sample implements functionality similar to one of the Deadline 10 balancer options. Instead of a Priority/FIFO queue, this implements weighted balancing for jobs in the farm. There are some special edge cases here: - If there are jobs that are set to 100, the Balancer goes into a "High Priority" mode and the entire Fleet gets balanced only between jobs at 100 priority. Other jobs get paused. - Jobs at 0 priority do not get any workers UNLESS there are only jobs with priority 0 on the farm. These jobs are treated as "paused" The actual values for how many workers to assign to each job is done on a "squared weight" policy. That means that a job at priority 20 would get twice as many workers as a job at priority 10. Signed-off-by: Alex Hughes <[email protected]>
Signed-off-by: Alex Hughes <[email protected]>
Jobs at 100 priority should be the only ones that matter If there are no jobs at 100, only jobs that have priority above 0 should matter If there are only jobs with priority of 0, start allocating to those ones Refactor our edge case logic to filter out invalid jobs This simplifies logic to only operate on jobs that are valid in our current context Signed-off-by: Alex Hughes <[email protected]>
Signed-off-by: Alex Hughes <[email protected]>
Signed-off-by: Alex Hughes <[email protected]>
Credit @mwiebe Signed-off-by: Alex Hughes <[email protected]>
…event Signed-off-by: Alex Hughes <[email protected]>
Signed-off-by: Alex Hughes <[email protected]>
Ahuge
force-pushed
the
ah/feature/custom-balancer
branch
from
1878d78 to
fccbd53
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is this?
This is an example of implementing a Lambda function that acts as a custom Balancer by modifying the `maxCurrentWorkers on a job.
What was the problem/requirement? (What/Why)
We had an internal customer that wanted to modify how the default scheduler assigned workers to tasks. They wanted it to work in a similar way to the Deadline 10 Balancer options.
What was the solution? (How)
Implement a scheduled Lambda function that runs every minute and calculates the weighting and enforces it by setting
maxCurrentWorkersHow was this change tested?
This has been running in the customer account for just over a week now. The client reports it works as expected.
No extensive additional testing has been done.
Was this change documented?
New README and template section
Lambda code was initially written in collaboration with Michael Yuan (thanks!)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.