Model Training and backprop

[apratim-mishra]

Is training possible on this model - sapiens?
I download the checkpoint and loaded it with RobertaModel.from_pretrained; however, it is a Roberta hub instance.
how would I go about tokenizing a sequence and running the forward pass on it?

[prihoda]

Hi @apratim-mishra inference code can be found here: https://github.com/Merck/Sapiens/blob/main/sapiens/roberta.py

As for training, we used fairseq-train script described here: https://fairseq.readthedocs.io/en/latest/command_line_tools.html

Please let me know if you need more info.