Motivation
Drafter training often benefits from holding the peak learning rate for most of a run and concentrating annealing in a short final phase. The current trainer exposes linear and cosine schedules but cannot express warmup-stable-decay (WSD) without external patches.
Proposed interface
Add scheduler_type: wsd with:
- configurable warmup initial LR ratio
- configurable final minimum LR ratio
- configurable final-decay fraction
- linear, cosine, exponential, and minus-square-root final-decay curves
All ratios scale each optimizer parameter group relative to its own peak LR, including the multi-optimizer Muon path. Invalid or overlapping phase geometry fails closed.
Scope
This is an independent trainer feature. It does not change any draft-model architecture or add model-specific recipes. A focused implementation and scheduler tests are ready on qianlihuang:feat/wsd-scheduler.
Motivation
Drafter training often benefits from holding the peak learning rate for most of a run and concentrating annealing in a short final phase. The current trainer exposes linear and cosine schedules but cannot express warmup-stable-decay (WSD) without external patches.
Proposed interface
Add
scheduler_type: wsdwith:All ratios scale each optimizer parameter group relative to its own peak LR, including the multi-optimizer Muon path. Invalid or overlapping phase geometry fails closed.
Scope
This is an independent trainer feature. It does not change any draft-model architecture or add model-specific recipes. A focused implementation and scheduler tests are ready on
qianlihuang:feat/wsd-scheduler.