Skip to content

[ERROR]In Step3,load reward Model failed which trainged with zero-stage 3 #429

Open
@Clitost

Description

@Clitost

In step 3, I modified the zero-stage of the critic model and reward model from 0 to 3, and the Reward Model trained with zero-stage 3 in step 2 cannot be loaded。But we can load reward model with zero-stage 0 in rw_eval.py

err:size mismatch for rwtranrsformer.layers.0.self_attn.q_proj.weight: copying a param with shape torch.Size([4096, 4096]) from checkpoint, the shape in current model is torch.Size([0]).

modified the zero-stage of the critic model and reward model from 0 to 3:
image

Metadata

Metadata

Assignees

Labels

deespeed chatDeepSpeed Chatnew-configA modified config from the given example

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions