Bound workflow claims within a dequeue transaction

[jlalmes]

Summary

• Add a default dequeue-transaction claim cap for DBOS queues, with a public maxDequeuesPerPoll option and DB-backed getter/setter.
• Replace unbounded SELECT ... FOR UPDATE plus per-row updates with a bounded atomic claim query using a locked candidate CTE and UPDATE ... RETURNING.
• Persist and serialize the new setting through queue metadata, conductor protocol, generated schema typings, and migrations.

Problem

A queue dequeue transaction can currently select every eligible workflow when no effective concurrency bound is present. With a large backlog, that transaction can lock and update a very large candidate set before it commits. If it remains open, it holds row locks in workflow_status and blocks other DBOS state changes that need to update those workflows, such as cancellation or completion.

Solution

Each findAndMarkStartableWorkflows call now claims at most 100 workflows by default, or the queue-specific maxDequeuesPerPoll value. The cap is applied after available concurrency and remaining rate-limit capacity are calculated, so rate-limited queues do not lock more rows than they are allowed to start.

The claim is a single bounded SQL statement: select locked candidates with LIMIT, then update and return the claimed workflow IDs atomically. Existing partition polling and scheduler behavior are unchanged; each existing partition dequeue transaction is independently bounded.

This is not a concurrency limit. It limits the number of workflows claimed in one dequeue transaction, keeping transactions short while preserving existing queue concurrency semantics.

Tests

• npm run build
• npx jest ./tests/wfqueue.test.ts --runInBand --testNamePattern="bounded-queue-dequeue-claims" --silent
• npx jest ./tests/adminserver.test.ts ./tests/client.test.ts --runInBand --silent

[maxdml]

Thanks @jlalmes , we'll be looking into this. I think the batch makes sense. I'd be worried about moving to bounded dequeue + batch budget. It'd probably require a new, carefully tuned index on queue_partition_key and more tuning to the getQueuePartitions query, and other perf evaluation.

[jlalmes]

Thanks @maxdml, is there a similar implementation in the Python library already?

[maxdml]

No, we also dequeue each workflow sequentially in the other languages.