- 
                Notifications
    
You must be signed in to change notification settings  - Fork 24
 
Open
Description
Checklist
- 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
 - 2. Please use English, otherwise it will be closed.
 
Requirement Description
We want role update associated in pd-disagg update scenario, during an update process:
(1) decode is updated before prefill, The prefill updates after the first decode-node finishes updating;
(2) gray-scale updates, update a subset of P/D nodes to test features, and remaining a group of P/D to provide services;
(3) associated updates, we need to control the update rates of prefill and decode to ensure update proportions are balanced.
The rollback operation is similar, where decode is rolled back before prefill.
Example
Shell                D (Decode)             P (Prefill)
  |                      |                        |
  |-- Trigger D update --|                        |
  |--   Fisrt D ready  --|                        |
  |                      | -- Trigger P update -------------> |
  |                      | <----------- P ready  ------------ |
  |---            P/D update continue     ------->|
  |---             P/D nodes reserve      ------->|
  | --      test      -- | --        test      -- |
  | --          P/D update continue         ----> |
  | <------       P/D update done         ------> |cheyang, ZYecho11 and loda13