Load model error in step3

[YingtongBu2]

When setting zero_stage=3, load my own ckpt in .pt format:

model_config = AutoConfig.from_pretrained(model_name_or_path)
model = AutoModel.from_config(model_config)

the shape of param in model is all torch.Size([0]), because the error message is:
size mismatch for model.decoder.layers.1.self_attn.k_proj.weight: copying a param with shape torch.Size([5120, 5120]) from checkpoint, the shape in current model is torch.Size([0]).
The ckpt I load is with correct shape of params, but the model build from config seems wrong.
With other zero_stages, the error will not occur.

I run the script line by line, the error does not occur either.
Anyone come across this problem?

[EthenZhang]

Same error. The code of deepspeed chat in step3 , critic and reward model are all load with zero_stage=0 ,So i can't find example that how to load model from checkpoint with zero_stage=3.

[gongyaguang-tal]

mark

[YingtongBu2]

solved, have to load from pytorch_model.bin of pretrained huggingface model when using zero3, otherwise after setting HfDeepSpeedConfig (a must for zero3), model = model_class.from_config(model_config) will make all parameters of model zero tensors.

[iamsile]

@YingtongBu2 Can you tell me in detail how you solved this bug? Also can you share this code?

[YingtongBu2]

@iamsile

Actually we don't need to modify any codes in model_utils.py, which is used to load model. The best solution is to convert model format to .bin for all steps

[iamsile]

@YingtongBu2 I used model.bin file in every step, but it doesn't work. I don't know where is wrong

[JumpingRain]

@YingtongBu2 I used model.bin file in every step, but it doesn't work. I don't know where is wrong

do you solve the problem now?

[BaenRH]

Have you solved the problem?