Skip to content
This repository was archived by the owner on Mar 22, 2025. It is now read-only.
This repository was archived by the owner on Mar 22, 2025. It is now read-only.

"Some weights of the model checkpoint were not used" - is that intended? #167

@Jakobhenningjensen

Description

@Jakobhenningjensen

When running

from danlp.models import load_bert_base_model 
model = load_bert_base_model()
vectorizerd_text = [model.embed_text(sentence)[1] for sentence in df["TextColumns"]]

I get the following

Some weights of the model checkpoint at C:\Users\Jakob.danlp\bert.botxo.pytorch were not used when initializing BertModel: ['cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).

is that intended i.e should it be ignored or is it a bug?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions