-
Notifications
You must be signed in to change notification settings - Fork 337
Open
Description
# special_tokens_dict = {'additional_special_tokens': ['<ENTITY>', '<DESCRIPTION>']}
# tokenizer.add_special_tokens(special_tokens_dict)
# 模型加载
# model = AutoModelForSeq2SeqLM.from_pretrained(conf.model_folder, trust_remote_code=True)
**# model.resize_token_embeddings(len(tokenizer)) # 关键步骤,重新调整所有的向量数量,即vocab size**
# tokenizer.save_pretrained(conf.output_path)
报错resize_token_embeddings
[PyTroch报错NotImplementedError]
Metadata
Metadata
Assignees
Labels
No labels