support kimi 2.5/6 full param and lora#1057
support kimi 2.5/6 full param and lora#1057nanjiangwill wants to merge 1 commit intoradixark:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces support for Kimi VL and Kimi K2.5 models, including specialized weight conversion logic and multimodal token expansion for training. It significantly enhances the LoRA weight synchronization mechanism by implementing IPC staging buffers for faster transfers and a chunked streaming approach to handle large adapters with SGLang. Additionally, it adds support for shared-outer grouped-expert LoRA and updates various utility functions and quantization scripts to accommodate vision-tower and projector components. Feedback was provided to improve the robustness of rollout batch processing by using strict zipping.
| expanded_total_lengths = [] | ||
| expanded_response_lengths = [] | ||
|
|
||
| for i, (token_tensor, loss_mask_tensor) in enumerate(zip(tokens, loss_masks, strict=False)): |
There was a problem hiding this comment.
Using strict=True in zip is safer here to ensure that the number of token tensors and loss mask tensors match exactly, which is expected for a valid rollout batch.
| for i, (token_tensor, loss_mask_tensor) in enumerate(zip(tokens, loss_masks, strict=False)): | |
| for i, (token_tensor, loss_mask_tensor) in enumerate(zip(tokens, loss_masks, strict=True)): |
megatron-bridge patch from commit
3fd3768045422d0aa5c97e90a4e6c659aea9acb9sglang patch from commit
bb9223d7c51fa66092b3bcae566ef4ecff309dc6