Missing key(s) in state_dict for unconditional #24
Description
Hi,
When I was trying to load the checkpoint, it gives the following error:
Missing key(s) in state_dict: Missing key(s) in state_dict: "bert.embeddings.position_ids", "bert.embeddings.word_embeddings.weight", "bert.embeddings.position_embeddings.weight", "bert.embeddings.token_type_embeddings.weight", "bert.embeddings.LayerNorm.weight", "bert.embeddings.LayerNorm.bias", "bert.encoder.layer.0.attention.self.query.weight", "bert.encoder.layer.0.attention.self.query.bias", "bert.encoder.layer.0.attention.self.key.weight",......
and a lot of other layer infos.
It looks like the state_dict has keys "module.bert...." rather than "bert..."as expected. Seems it's similar to issue #17 so please kindly help. How would I fix this issue? Thanks in advance.
P.S. I got the model checkpoints by running DDP_main.py. I saved earlier-stage checkpoints and stopped training as it took too long in eval mode with warnings "NAN encountered ... times". Does your training look the same?