Skip to content

Fix ramp-up worker scaling - #27

Merged
hemanth-asirvatham merged 1 commit into
mainfrom
fix-ramping-functionality-and-run-tests
Feb 7, 2026
Merged

Fix ramp-up worker scaling#27
hemanth-asirvatham merged 1 commit into
mainfrom
fix-ramping-functionality-and-run-tests

Conversation

@hemanth-asirvatham

Copy link
Copy Markdown
Collaborator

Motivation

  • The ramp-up logic wasn't actually increasing the active worker pool over time, so concurrency never reached the intended ramp target during get_all_responses runs.

Description

  • Added a worker_spawner loop and a helper _desired_worker_count() to gradually spawn new worker tasks as the ramp increases the parallel cap in get_all_responses (file src/gabriel/utils/openai_utils.py).
  • Make the spawner call _maybe_emit_ramp_complete() so ramp completion is emitted even when the periodic status_reporter is disabled.
  • Initialize an adaptive workers list and start an initial set of workers via _desired_worker_count() instead of creating a static pool upfront.
  • Ensure the spawner is cancelled and awaited during shutdown so spawned workers are cleaned up properly.

Testing

  • Ran pytest tests/test_parallelization_tuning.py::test_ramp_up_increases_parallelism, which passed.

Codex Task

@hemanth-asirvatham
hemanth-asirvatham merged commit 1007b93 into main Feb 7, 2026
1 check failed
@github-actions

github-actions Bot commented Feb 7, 2026

Copy link
Copy Markdown


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@github-actions github-actions Bot locked and limited conversation to collaborators Feb 7, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant