Skip to content

[feat] support chat_scheduler_kwargs when init AsyncLLM #1538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[feat] support chat_scheduler_kwargs when init AsyncLLM #1538

wants to merge 2 commits into from

Conversation

llkn-2
Copy link
Contributor

@llkn-2 llkn-2 commented May 15, 2025

Checklist Before Starting

  • Search for similar PR(s).

What does this PR do?

When using a custom chat_scheduler, we need to use chat_scheduler_kwargs to pass additional parameters.

Additional Info.

  • Issue Number: none.
  • Training: none
  • Inference: none

Checklist Before Submitting

  • Read the Contribute Guide.
  • Apply pre-commit checks.
  • Add [BREAKING] to the PR title if it breaks any API.
  • Update the documentation about your changes in the docs.
  • Add CI test(s) if neccessary.

eric-haibin-lin
eric-haibin-lin reviewed May 15, 2025
View reviewed changes
verl/trainer/config/ppo_trainer.yaml
@@ -90,6 +90,7 @@ actor_rollout_ref:
name: vllm
mode: sync # sync: LLM, async: AsyncLLM
chat_scheduler: null # async chat scheduler, e.g examples.ppo_trainer.naive_chat_scheduler.NaiveChatCompletionScheduler
chat_scheduler_kwargs: { } # kwargs for custom chat scheduler
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also update config.rst under docs/examples/config.rst. thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I have updated the doc.

@vermouth1992 vermouth1992 requested a review from wuxibin89 May 16, 2025 08:46
@llkn-2 llkn-2 requested a review from eric-haibin-lin May 19, 2025 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wuxibin89 wuxibin89 Awaiting requested review from wuxibin89

@eric-haibin-lin eric-haibin-lin Awaiting requested review from eric-haibin-lin

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@llkn-2 @eric-haibin-lin