Skip to content

Commit 05c459f

Browse files
committed
fix(queues): widen worker lease margin to fix CI flake in worker recovery test
The "healthy long-running job" worker recovery test used a 600ms lease duration with only a 50ms safety buffer, leaving ~550ms for heartbeats to renew the lease. On a loaded CI runner a single delayed heartbeat can exceed that margin, causing the worker's own completion update to be rejected as lease-expired (JobRunAbortedError), which gets silently filtered out of jobStatus. Widen the margin to 5000ms/300ms so a single CI-scale scheduling stall can't exceed it, while keeping the same duration/safetyBuffer ratio.
1 parent b2379e8 commit 05c459f

1 file changed

Lines changed: 6 additions & 3 deletions

File tree

test/queues/int.spec.ts

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1429,8 +1429,11 @@ describe('Queues - Payload', () => {
14291429
expect(payload.config.jobs.processingLease.duration).toBe(20 * 60 * 1000)
14301430
expect(payload.config.jobs.processingLease.safetyBuffer).toBe(30_000)
14311431
payload.config.jobs.deleteJobOnComplete = false
1432-
payload.config.jobs.processingLease.duration = 600
1433-
payload.config.jobs.processingLease.safetyBuffer = 50
1432+
// Margin between duration and safetyBuffer must comfortably exceed any single
1433+
// heartbeat delay on a loaded CI runner - too tight and the heartbeat can lose
1434+
// the race, making the worker's own completion update look lease-expired.
1435+
payload.config.jobs.processingLease.duration = 5000
1436+
payload.config.jobs.processingLease.safetyBuffer = 300
14341437

14351438
const postTitle = 'created by the healthy worker'
14361439
const job = await payload.jobs.queue({
@@ -1451,7 +1454,7 @@ describe('Queues - Payload', () => {
14511454
)
14521455
.toBeTruthy()
14531456
// Wait longer than the original lease. Heartbeats must keep the first worker active.
1454-
await wait(900)
1457+
await wait(6000)
14551458

14561459
const secondWorkerResult = await payload.jobs.run({ silent: true })
14571460
const firstWorkerResult = await firstWorker

0 commit comments

Comments
 (0)