Skip to content

fix: use worker_id in request queue for ocncurrent load generation #347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jjk-g merged 2 commits into from
Feb 15, 2026
+3 −3
Merged

fix: use worker_id in request queue for ocncurrent load generation #347

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1ab94a1
fix: use worker_id in request queue for ocncurrent load generation
changminbark Feb 12, 2026
e13dc8a
chore: pdm validate/lint
changminbark Feb 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions inference_perf/datagen/shared_prefix_datagen.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ def _generate_prompts(self) -> None:

# Shuffle the generated prompts to ensure randomness if served sequentially by different workers
if self.enable_multi_turn_chat:
# no need to sync shuffles - multi-round initial prompt does not include system prompt
random.shuffle(self.user_sessions)
# no need to sync shuffles - multi-round initial prompt does not include system prompt
random.shuffle(self.user_sessions)
else:
random.shuffle(self.prompts)
2 changes: 1 addition & 1 deletion inference_perf/loadgen/load_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,7 @@ async def run_stage(
worker_id = request_data.prefered_worker_id
if worker_id >= 0:
worker_id = worker_id % active_workers
request_queue.put((stage_id, request_data, next(time_generator), lora_adapter), request_data.prefered_worker_id)
request_queue.put((stage_id, request_data, next(time_generator), lora_adapter), worker_id)

# Wait until all requests are finished processing
with tqdm(total=1.0, desc=f"Stage {stage_id} progress") as pbar:
Expand Down