Skip to content

Conversation

@DongheJin
Copy link
Collaborator

No description provided.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes a bug where DeepSeek models fail when data parallelism (DP) is enabled. The fix involves remapping CHUNKED_PREFILL to PREFILL in the LLM engine when preparing inputs for DP ranks, which avoids an unstable code path. While the fix is effective, it's a workaround that lacks explanation. I've added a high-severity comment recommending the addition of a code comment to document this workaround for future maintainability. A minor log message improvement in the DeepSeek decoder is also included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant