The vocab size of the sentence encoder is 50256, but the vocab size of the generator is 30522. This will cause the index out of range.
The code is follow:
cls.generator(mlm_input_ids, attention_mask) mlm_input_ids.min()=0 mlm_input_ids.max()=50264
How should I initialize them so that they can be consistent?