[Bug] 0.00 acc at the start of finetuning pretrained draft model from huggingface

[AxeCameron777]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
• 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/SpecForge/discussions/new/choose Otherwise, it will be closed.
• 5. Please use English, otherwise it will be closed.

Describe the bug

Thanks for the cool framework for training draft models. But I recently encountered a problem when I tried to fine-tune an existing draft model (lmsys/Qwen3-235B-A22B-EAGLE3) for my domain. I prepared a part of the sharegpt dataset and I also prepared hidden_states. Adding the already trained model to cache/model/epoch_0 and launching offline training script I saw acc 0.00-0.03 for a long time. Although I expected to see a high acc because the model accelerates the target model quite well in various frameworks. After training whole first epoch I saw ~1.2 mean_acceptance_length in vllm framework (model degradation).

I've tried to print model parameters loaded from .safetensors file and they are correct. Also I tried to change vocab_mapping cache on t2d, d2t from trained model. Nothing helps

Reproduction

Add lmsys/Qwen3-235B-A22B-EAGLE3 model to cache/model/epoch_0 directory.
launch scripts/prepare_data.py for some dataset
launch scripts/prepare_hidden_states.py for prepared dataset
launch train_eagle3_offline.py
And we see 0.00 acc at the start of training

Environment

SpecForge framework of the latest version

[uygnef]

I also encountered it and found that the root cause is the max_sequence_length being set lower than the actual length of the user's prompt.

[AxeCameron777]

I don't think this is the problem, because I've tried training the model from scratch with the same parameters and dataset, and the accuracy is increasing. Moreover, the mean acceptance length in vllm also looks good. However, I'll try to review the parameters again, thank you.

[hanq-moreh]

I don't think this is the problem, because I've tried training the model from scratch with the same parameters and dataset, and the accuracy is increasing. Moreover, the mean acceptance length in vllm also looks good. However, I'll try to review the parameters again, thank you.

Which dataset are you using?
If the token distribution of the fine-tuning data is very different from that of the pre-trained draft model, then you’ll need to create a new vocab mapping instead of relying on the existing one. In that case, having the acceptance rate stay around 0.00–0.03 for a while is expected.
Still, in my experience, it usually climbed to at least 0.2–0.3 before reaching the end of the first epoch.

[FrankLeeeee]

Which dataset is it? We trained one on 30B-A3B-Instruct and it works fine (acc len ~= 3 for 3-1-4 spec config). I used online training.

https://huggingface.co/frankleeeee/SGLang-EAGLE3-Qwen3-30B-A3B-Instruct-2507