Since Buildkite now offers EventBridge integration, triggering the scaling lambda on job.* events may be more responsive than the polling method used currently - and without requiring a lambda to effectively be running 24/7.
One challenge might be preventing a large build that queues multiple jobs from firing the lambda repeatedly. However, solving that may be as simple as sticking a kinesis stream between the EventBridge and the lambda (and taking advantage of the built-in batching).
Since Buildkite now offers EventBridge integration, triggering the scaling lambda on
job.*events may be more responsive than the polling method used currently - and without requiring a lambda to effectively be running 24/7.One challenge might be preventing a large build that queues multiple jobs from firing the lambda repeatedly. However, solving that may be as simple as sticking a kinesis stream between the EventBridge and the lambda (and taking advantage of the built-in batching).