Issue with saving and inference on a FastFit model I trained

[shervin0]

I have installed fastfit 1.2.1 from source.
I am exactly following the example on the fastfit read-me page (https://github.com/IBM/fastfit).
I can train the model, but I cannot save it. I try this to save the model:
model.save_pretrained("fast-fit")
Here's the error message when trying to save the model:
AssertionError: Config has to be initialized with encoder config

And here's where the error happens:

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/fastfit/modeling.py:259, in FastFitConfig.init(self, all_docs, num_repeats, rep_tokens, inference_type, clf_level, init_freeze, clf_dim, proj_dim, similarity_metric, encoder_dropout, encoding_type, clf_query, clf_doc, mask_zeros_in_query, mask_zeros_in_doc, sim_factor, clf_factor, mlm_factor, mask_prob, pretrain_mode, length_norm, scores_temp, inference_direction, symetric_mode, length_norm_at_inference, **kwargs)

    256 self.length_norm_at_inference = length_norm_at_inference
    258 assert inference_direction in ["query", "doc", "both"]
--> 259 assert "encoder" in kwargs, "Config has to be initialized with encoder config"
    260 encoder_config = kwargs.pop("encoder")
    261 encoder_model_type = encoder_config.pop("model_type")

Also, trying to use the trained model for inference, I use this code:

from transformers import AutoTokenizer, pipeline

tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/paraphrase-mpnet-base-v2")
classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)
print(classifier("I love this package!"))

And get this error on the last line:

TypeError: FastFitTrainable.forward() got an unexpected keyword argument 'input_ids'

Am I doing something incorrectly?
Thanks.

[bayadsi]

I got the same error running on my own dataset. The problem for me was that my dataset object did not have a validation dataset. Managed to resolve it by having a train, validation and test sets in the dataset object passed to the trainer.

[gokulrejith]

Same issue. Also don't know why I need a validation dataset to save a model. Also fastfit does not seem to be compatible with many other models.

[gokulrejith]

@shervin0 As @bayadsi mentioned, adding the validation dataset solved the first issue for me. All these are dependency issues and the authors (@Taytay) haven't mentioned the specific version of the libraries they used.

For solving your second issue try this snippet

from transformers import AutoTokenizer, pipeline
import torch

model = FastFit.from_pretrained("fast-fit")
tokenizer = AutoTokenizer.from_pretrained("/path/to/model")

def custom_classifier(text):
    # Tokenize without 'token_type_ids'
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
    inputs.pop("token_type_ids", None)
    outputs = model(**inputs)
    logits = outputs.logits
    probabilities = torch.nn.functional.softmax(logits, dim=-1)
    predicted_class = torch.argmax(probabilities, dim=-1).item()
    return predicted_class, probabilities

predicted_class, probabilities = custom_classifier("I love this package!")
print(f"Predicted Class: {predicted_class}")
print(f"Probabilities: {probabilities}")

[shervin0]

Thank you @bayadsi and @gokulrejith for your help.
I have a validation dataset and it didn't fix my problem.

I was able to fix the second problem, but not the first one.
The problem is that model was a FastFitTrainable object, not a FastFit model.
I used the following code to convert it to a FastFit model:
ff_model = FastFit( model.config, encoder=model.encoder )
This solved my second problem (being able to use it for inference) but still have issues when trying to save my trained model.

Still when I want to save my model, I get this error:
"AssertionError: Config has to be initialized with encoder config"