Skip to content

feat: Allow external semaphore for executor busy state tracking#14

Merged
lukekim merged 2 commits into
spiceai-51from
lukim/poll
Jan 29, 2026
Merged

feat: Allow external semaphore for executor busy state tracking#14
lukekim merged 2 commits into
spiceai-51from
lukim/poll

Conversation

@lukekim
Copy link
Copy Markdown

@lukekim lukekim commented Jan 29, 2026

This change allows the semaphore controlling task concurrency to be passed into the poll_loop instead of being created internally. This enables:

  1. Sharing the semaphore across multiple poll loops connected to different scheduler nodes
  2. External tracking of executor busy state by querying available_permits()
  3. Reporting busy state in scheduler shared state location metadata

The semaphore parameter is optional - if None is passed, the function creates one internally to maintain backwards compatibility.

This change allows the semaphore controlling task concurrency to be passed
into the poll_loop instead of being created internally. This enables:

1. Sharing the semaphore across multiple poll loops connected to different
   scheduler nodes
2. External tracking of executor busy state by querying available_permits()
3. Reporting busy state in scheduler shared state location metadata

The semaphore parameter is optional - if None is passed, the function
creates one internally to maintain backwards compatibility.
@lukekim lukekim self-assigned this Jan 29, 2026
@lukekim lukekim added the enhancement New feature or request label Jan 29, 2026
@lukekim lukekim merged commit 20ef1eb into spiceai-51 Jan 29, 2026
29 checks passed
@lukekim lukekim deleted the lukim/poll branch January 29, 2026 03:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants